Humans perceive the properties of a surface by interpreting visual input. When estimating gloss and lightness, it seems that neural discrimination of simple image statistics plays a large part.
How do you tell the difference between peaches and nectarines, or between unfinished and polished wood? Many visual attributes help us to distinguish different surface materials, including lightness, colour and texture. The salient attribute shared by nectarines and finished wood is a mirror-like (specular) component of its reflectance, which is perceived as gloss or shininess. On page 206 of this issue, Motoyoshi and colleagues1 describe a surprising discovery concerning surface perception: a simple characteristic of image statistics — the distribution of luminance values in an image, or the 'skew' — is highly correlated with judgements of gloss and lightness Footnote 1. The principle can be illustrated by the manipulation of a picture in which a nectarine has been visually transformed to look more like a peach by removing a highlight (Fig. 1).
Motoyoshi et al. took calibrated photographs of stucco-like materials varying in albedo (dependent on the amount of black pigment in the material) and gloss (the amount of clear acrylic coating), and found that as gloss was increased, or as albedo was reduced for a glossy surface, the luminance distribution became positively skewed (see Fig. 2a of Motoyoshi et al.1). In other words, images of glossy materials are predominantly dark, with occasional bright highlights (Fig. 2). They found that human visual judgements of glossiness and lightness were correlated with histogram skew for the stucco images, as well as for photographs of other natural materials. More importantly, simply skewing the histogram of a photograph of a material caused the surface to appear glossier and darker. Finally, they found that if observers adapted to an image with positive skew, a subsequently viewed surface appeared less glossy (with the opposite result for adaptation to negative skew), indicating that humans extract something like luminance skew from images.
This finding is consistent with other work showing that humans are sensitive to image statistics for a variety of judgements. In addition to gloss, perceived surface roughness and translucency also depend on image statistics2,3,4. Skew is an example of one statistic derived from the luminance histogram. But humans are sensitive to at least three statistics of the histogram5,6. Perceived brightness and contrast correspond roughly to the mean and variance of luminance7,8. In early work9,10, luminance statistics were found to be insufficient to account for the discriminability of texture patterns. More recent studies, however, indicate that humans are sensitive to the statistics of responses of bandpass filters — for example, simple cells in the primary visual cortex — for both texture discrimination11 and texture appearance12,13.
How might the visual system compute statistics such as histogram skew? The initial coding involves spatial linear filtering, which is carried out by various parts of the visual system: the centre-surround receptive fields of ganglion cells in the retina; cells in the lateral geniculate nucleus region of the brain; and the orientation-tuned receptive fields of simple cells in the primary visual cortex. Histogram statistics, and skew in particular, could be recovered from the cells with centre-surround receptive fields, for which darkness and brightness information are separately represented by 'off' and 'on' channels. Motoyoshi and colleagues1 simulated such a model. Alternatively, such statistics might be recovered from the responses of orientation-selective simple cells in primary visual cortex5.
Why should positive histogram skew result in both an increased perception of gloss and an apparent darkening of the surface? Many perceptual capabilities are described in terms of 'discounting'. For example, colour constancy refers to the ability, albeit incomplete, of observers to estimate surface colour independently of the spectral power distribution of the illuminant, thus discounting the illuminant in the interpretation of the retinal signal14. When a histogram is positively skewed, apparent glossiness is increased. Thus, pixels in the positive tail of the luminance distribution are interpreted as highlights (mirror reflections of the illuminant), and then discounted in interpreting surface lightness15. Lightness then becomes a function of the remaining, darker pixel values. This explains why an increase in perceived glossiness is often associated with decreased lightness.
There is, however, another possible explanation of the correlation of image skew with judgements of both gloss and lightness. Parameters of a luminance histogram (mean, variance, skew and so on) are convenient mathematically, but might not correspond precisely to the computations used in making perceptual judgements. In fact, luminance variance is not the form of nonlinearity used by humans for estimates of image contrast7. If the impact of different luminance levels on judgements of glossiness were directly measured, one might find that a different nonlinearity (other than skew) is computed, such as the 'blackshot mechanism'5,6 — which was, by design, orthogonal to the computation of mean luminance, and hence should not correlate with judgements of lightness. It remains to be seen how one can determine the perceptually relevant quantity for estimation of gloss.
Histogram statistics are not the whole story for the perception of lightness, contrast or gloss. Perceived lightness and contrast of a surface depend in a complex way on the surrounding surfaces16. For an image to appear glossy, it has to first look like a surface. As Motoyoshi et al.1 point out, the mere presence of a positively skewed histogram is not enough. If an image is modified by randomly permuting its pixels, or by giving random phase values to its sine-wave components, the resulting image may have positively skewed luminance statistics, but will not look like a surface, so the rare, bright pixels will not look like highlights.
For a surface to appear glossy, not only must it include a specular reflectance, but the surroundings must result in a pattern of illumination consistent with the statistics of natural scenes17,18. There are many physical dimensions of gloss that affect the perception of surface material. The one studied by Motoyoshi and colleagues is the percentage of ambient light that is reflected in the mirror direction. Another is the degree to which the specular reflection is point-like or blurred (for example in the case of polished versus brushed metal). Its effect on perception has not been studied systematically. But although histogram skew does not explain everything about the perception of surface material, or even of gloss, it is a major step towards a theory of the perception of surface materials.
This article and the paper concerned1 were published online on 18 April 2007.
Motoyoshi, I., Nishida, S., Sharan, L. & Adelson, E. H. Nature 447, 206–209 (2007).
Ho, Y.-X., Landy, M. S. & Maloney, L. T. J. Vis. 6, 634–638 (2006).
Ho, Y.-X., Maloney, L. T. & Landy, M. S. J. Vis. 7, 1–16 (2007).
Fleming, R. W. & Bülthoff, H. H. ACM Trans. Appl. Percept. 2, 346–382 (2005).
Chubb, C., Econopouly, J. & Landy, M. S. J. Opt. Soc. Am. A 11, 2350–2374 (1994).
Chubb, C., Landy, M. S. & Econopouly, J. Vision Res. 44, 3223–3232 (2004).
Chubb, C. & Nam, J. H. Vision Res. 40, 1677–1694 (2000).
Nam, J. H. & Chubb, C. Vision Res. 40, 1695–1709 (2000).
Caelli, T. & Julesz, B. Biol. Cybern. 28, 167–175 (1978).
Julesz, B., Gilbert, E. N. & Victor, J. D. Biol. Cybern. 31, 137–140 (1978).
Bergen, J. R. & Adelson, E. H. Nature 333, 363–364 (1988).
Heeger, D. J. & Bergen, J. R. Proc. ACM SIGGRAPH 1995, 228–238 (Assoc. Comput. Machinery, New York, 1995).
Portilla, J. & Simoncelli, E. P. Int. J. Comput. Vision 40, 49–71 (2000).
von Helmholtz, H. Treatise on Physiological Optics, Vol. II (transl. Southall, J. P. C.) 287 (Dover, New York, 1962).
Todd, J. T., Norman, J. F. & Mingolla, E. Psychol. Sci. 15, 33–39 (2004).
Gilchrist, A. L. Seeing Black and White (Oxford Univ. Press, New York, 2006).
Fleming, R. W., Dror, R. O. & Adelson, E. H. J. Vis. 3, 347–368 (2003).
Dror, R. O., Willsky, A. S. & Adelson, E. H. J. Vis. 4, 821–837 (2004).
About this article
Operative Dentistry (2020)
Synthesis of an Aqueous Self-Matting Acrylic Resin with Low Gloss and High Transparency via Controlling Surface Morphology
Estimating mechanical properties of cloth from videos using dense motion trajectories: Human psychophysics and machine learning
Journal of Vision (2018)
Journal of Vision (2018)