Nature Neuroscience
- 9, 1367 - 1368 (2006)
Published online: 15 October 2006; | doi:10.1038/nn1794
Memory modulates color appearanceThorsten Hansen, Maria Olkkonen, Sebastian Walter & Karl R GegenfurtnerAbteilung Allgemeine Psychologie, Justus-Liebig-Universität Giessen, Otto-Behaghel-Str. 10F, 35394 Giessen, Germany.
Correspondence should be addressed to Karl R Gegenfurtner gegenfurtner@uni-giessen.de We asked human observers to adjust the color of natural fruit objects until they appeared achromatic. The objects were generally perceived to be gray when their color was shifted away from the observers' gray point in a direction opposite to the typical color of the fruit. These results show that color sensations are not determined by the incoming sensory data alone, but are significantly modulated by high-level visual memory.
Visual perception is inherently constructive and creative in nature, and frequently, additional assumptions and constraints are necessarily involved in generating a specific visual sensation1. Color is a particularly ill-posed problem, because the light entering our eyes is the product of the illumination and the surface reflectance of the object and therefore changes considerably when the illumination changes. Still, we are able to reliably perceive the color of objects. There are several known mechanisms involved in this process. Some mechanisms act on a local scale, computing cone excitation ratios across edges, whereas others act on a more global scale, computing the average color over larger image regions2. What these mechanisms have in common is that they are driven solely by the visual signals arising from the retina.
Here we investigated whether the known color of objects also affects color appearance. Many natural and man-made objects have a typical or diagnostic color, which is termed their memory color. This knowledge could be used to guide our visual system in the process of assigning colors to objects and in discounting the illuminant. Previous studies have demonstrated that the memory color can be distinct from the measured color of the objects and is typically more saturated (refs. 3,
4,
5,
6,
7,
8,
9; and A.Hurlbert & Y.Ling, J. Vis. 5, 787a, 2005). However, the results of these studies were highly variable, and, because the subjects in these studies could never directly change the colors of the objects under investigation, it is not clear whether the effects were of a perceptual nature.
We presented digitized photographs of natural fruit objects on a uniform gray background. The color of the fruit objects could be interactively manipulated to produce, for example, bananas of any arbitrary color. This, of course, poses the question: what does a blue banana look like? We used an adjustment method in which we started with the original color image and then allowed the subject to interactively scale and rotate the whole distribution of pixels in color space (Fig. 1 and Supplementary Methods online). This method satisfies the constraint that a gray banana under a neutral illumination should be achromatic: all pixels vary only along a luminance axis but not in their chromaticity. For all other settings, the banana was characterized by a whole distribution of chromaticities, and the relationship between relative saturation and luminance was kept stable.
 | | |
Our subjects had to adjust the color of the fruit objects until they appeared gray. In a different set of experiments, we asked the same subjects to adjust the color of the fruit objects until they appeared natural. It was evident that the settings for the banana (Fig. 2a) deviated from the neutral gray adaptation point at the origin of the color space, in the direction opposite to the typical setting (t13 = 6.915, P < 0.001). In actual fact, subjects adjusted the banana to a slightly bluish hue—its opponent color—in order for it to appear neutral gray. At the point where the banana was actually achromatic, at the origin of the color space, it still appeared yellowish. A similar effect was obtained for the other fruit objects we investigated (Fig. 2b). As a control, we asked our subjects to adjust uniform spots of light and random noise patches (Supplementary Methods), which do not have an association with a typical color. The settings for these stimuli (Fig. 2b) did not differ significantly from the neutral gray background (t13 = 1.39, P > 0.05), but the difference between the controls and the fruit setting was significant (t13 = 4.22, P < 0.001). To quantify the memory color effect, we determined how far from the neutral control stimuli the subjects adjusted the fruits in the direction away from their typical setting. We then normalized the memory color index by dividing through the distance between the control settings and the fruit's typical setting (Supplementary Methods). The indices varied between 4– 13% for the different fruit objects (mean = 8.23%, t13 = 6.195, P < 0.001). This amounts to an effect that is approximately three to five times above the threshold of discrimination. The effect was highly reliable even for individual observers. All 14 subjects had positive memory color indices, indicating that they all adjusted the fruits in the direction opposite to their typical settings.
| | |
In previous studies on memory color, subjects had to either view an object or recall its color from memory, and then pick a matching color from a set of colors. In our study, observers could actually manipulate the color of the object while viewing it, without the need for a cognitive retrieval of memory during the adjustment procedure. Rather, memory seemed to have a direct top-down effect that continuously modulated the incoming sensory data and changed basic color sensations.
Our results show that natural fruit objects tend to be perceived in their typical color. Objects with a typical color might seem to constitute only a small percentage of the objects surrounding us, but one has to keep in mind that our visual system evolved in a world that was not filled with man-made objects of arbitrary color. Another question posed by our results is whether they are applicable to the type of lighting conditions occurring naturally. Notably, natural illumination during the course of the day varies mostly along the blue-yellow dimension10. Natural daylight contains a high proportion of short-wavelength energy in the morning and gradually shifts toward energy of longer wavelengths during the course of the day. Most fruit objects are yellowish rather than bluish; therefore, they might appear grayish in the morning, as color constancy is known to be imperfect even in realistic situations11. The appearance shifts described here may contribute toward color constancy under these conditions.
Our results show a high-level cognitive effect on low-level perceptual mechanisms. Our knowledge of the world affects our perception, very much in the way that Bayesian modeling has predicted in many settings1. Just as we apply prior knowledge that the light usually comes from above12, we apply prior knowledge about the natural color of fruit objects. This knowledge is used together with other local and global information about the scene to determine color appearance. This allows the visual system to function and perform even under reduced conditions when only single objects are shown under an unknown illuminant, as in our experiments. The mechanisms that determine color appearance act throughout all processing stages of the visual system, from the retina to visual cortex including visual memory. Modulatory feedback is a candidate neural mechanism underlying the integration of bottom-up incoming data and top-down expectations13,
14.
Note: Supplementary information is available on the Nature Neuroscience website.
Received 1 September 2006; Accepted 27 September 2006; Published online: 15 October 2006.
REFERENCES
- Knill, D.C. & Richards, W. Perception as Bayesian Inference (Cambridge Univ. Press, Cambridge, Massachusetts, 1996).
- Smithson, H.E. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 1329–1346 (2005). | PubMed | ChemPort |
- Duncker, K. Am. J. Psychol. 52, 255–265 (1939). | Article |
- Bruner, J.S., Postman, L. & Rodrigues, J. Am. J. Psychol. 64, 216–227 (1951). | Article | PubMed | ChemPort |
- Fisher, S.C., Hull, C. & Holtz, P. Am. J. Psychol. 69, 546–560 (1956). | Article | PubMed | ChemPort |
- Bolles, R.C., Hulicka, I.M. & Hanly, B. Can. J. Psychol. 13, 175–185 (1959). | PubMed | ChemPort |
- Bartleson, C.J. J. Opt. Soc. Am. 50, 73–77 (1960). | PubMed | ChemPort |
- Delk, J.L. & Fillenbaum, S. Am. J. Psychol. 78, 290–293 (1965). | Article | PubMed | ChemPort |
- Jin, E.W. & Shevell, S.K. J. Opt. Soc. Am. A 13, 1981–1991 (1996). | ChemPort |
- Taylor, A.H. & Kerr, G.P. J. Opt. Soc. Am. 31, 3–8 (1941).
- Kraft, J.M. & Brainard, D.H. Proc. Natl. Acad. Sci. USA 96, 307–312 (1999). | Article | PubMed | ChemPort |
- Ramachandran, V.S. Nature 331, 163–166 (1988). | Article | PubMed | ISI | ChemPort |
- Grossberg, S. Psychol. Rev. 87, 1–51 (1980). | Article | PubMed | ISI | ChemPort |
- Hupé, J.M. et al. Nature 394, 784–787 (1998). | Article | PubMed | ISI | ChemPort |
Acknowledgments
We thank L. Kaim and W. Kirchner for technical assistance, and A. Hurlbert and L. Maloney for helpful discussions. This work was supported by the Deutsche Forschungsgemeinschaft (Ge 879/5-2) and by a Giessen University dissertation fellowship to S.W.
Competing interests statement:
The authors declare that they have no competing financial interests. |
