Purpose of Review Color provides important information about the identity of the objects we encounter. After early processing stages in the retinal cones, thalamus, and occipital cortex, retinal signals reach the ventral temporal cortex for high-level color and object processing, which links color perception with top-down expectations and knowledge. In the language-dominant hemisphere, some of these regions communicate with the language systems; by assigning verbal labels to percepts, these circuits speed up stimulus categorization, and permit fast and accurate inter-individual communication. This paper provides a review of color processing deficits, from dysfunction of wavelength discrimination in the retinal photoreceptors to deficits of high-level processing in the ventral temporal cortex. Recent Findings Neuroimaging evidence defined the existence and localization of color-preferring domains in the ventral occipito-temporal cortex. Evidence from the performance of a brain-damaged patient with color anomia but preserved color categorization demonstrated the independence of color categorization from color naming in the adult brain. Summary Evidence from patients with brain damage suggests that high-level color processing may be divided into at least three functional domains: perceptual color experience, color naming, and color knowledge.