Introduction

Digital displays are increasing in ubiquity and complexity. Traditional movie theaters and televisions presented a sequence of closely related images at 48–60 Hz. Stereo 3D television presents a coded sequence of frames intended for the left and right eyes at a total of 120 Hz. DLP projection technology creates full color images from a temporal sequence of dozens of very brief monochrome sub-frames, each lasting only microseconds1. Rather than a simple sequence of frames, each of which is a natural image, display designers and researchers now think in terms of coding the temporal output of a display. Applications include using three or more sub-frames to allow simultaneous 3D and 2D television2, embedding imperceptible codes into normal images3,4 and making private displays viewable only with the appropriate glasses5,6.

The light output of modern displays may at no point of time actually resemble a natural scene. Instead, the codes rely on the fact that at a high enough frame rate human perception integrates the incoming light, such that an image and its negative in rapid succession are perceived as a grey field. This paper explores these new coded displays, as opposed to the traditional sort which show only a sequence of nearly identical images.

The key question that must be answered to build these devices is “What framerate is necessary to provide the illusion of a stable picture?” This question has been the subject of research for more than 50 years and nearly all articles and textbooks on the subject contain a statement similar to the following, The critical flicker fusion rate is defined as the rate at which human perception cannot distinguish modulated light from a stable field. This rate varies with intensity and contrast, with the fastest variation in luminance one can detect at 5090 Hz5,6,7,8,9,10,11,12,13.

These primary perceptual findings have been incorporated into international standards for display ergonomics14 and a belief that “…a frame rate of 72 Hz for computer displays is sufficient to avoid flicker completely.”15.

In the present study, we find that viewers can distinguish between modulated light and a stable field at up to 500 Hz, much higher than the widely reported rate. We hypothesize that unconscious rapid eye movements across high frequency edges in the displayed image are responsible.

Most existing studies have been carried out using a spatially uniform light source16, however digital displays provide a spatially varying image. Spatio-temporal sensitivity has also been explicitly measured, with reports of a maximum perceivable rate approximately equivalent to spatially uniform lighting17,18. These studies have sometimes attempted to measure retinal stabilized response, often with special equipment to insure that eye movement does not affect the measurements19,20. The relationship between eye movement and flicker perception has received mixed reports. Some researchers have reported that eye movement can enhance threshold perception21,22, while others have concluded that eye movement do not substantially affect the visibility of motion artifacts or flicker on spatial displays8,23.

The effect measured in this paper is likely due to saccades and thus related to the phantom array, a repeated pattern observed with high frequency modulation of a bright point or bright line24,25,26. Unfortunately, none of the existing work on phantom arrays addresses the question of the maximum perceivable modulation rate on displays with 2D spatial extent, the information required by display designers.

The work presented here attempts to clarify “the rate at which human perception cannot distinguish between modulated light and a stable field.” We allow for a spatially varying light source and do not attempt to constrain natural eye movements of our subjects. We follow prior work by measuring the viewer's contrast sensitivity, the ratio between background illumination and modulated illumination which is perceivable. However, rather than varying the brightness of the modulated signal as is done in most perceptual research, we mimic real world situations in which displays have constant brightness and the surrounding ambient light level varies. In our tests the display modulates between an image and its inverse at a constant brightness level, while the subject adjusts the level of ambient illumination until flickering artifacts are just noticeable.

Results

Uniform Light vs. Spatial Edge

We presented users with a modulated light source and asked them to determine the level of ambient illumination under which flicker was just noticeable. We performed experiments both with spatially uniform light resembling most prior studies on the critical flicker fusion rate, as well as with a spatially varying image as would be common on display devices such as TVs and computer screens.

In our experiments, uniform modulated light was produced by a DLP projector and consists of a solid “bright” frame followed by a solid “black” frame. The high spatial frequency image is first “bright on the left half of the frame and black on the right” and then inverted. We observed the effect described in this paper whenever we displayed an image containing an edge and its inverse in rapid succession. The effect was even stronger with more complex content that contained more edges, such as that in natural images. We chose a simple image with a single edge to allow our experimental condition to be as repeatable as possible.

The median contrast sensitivity curve as well as the individual curves for ten subjects is shown in Figure 1. When the modulated light source is spatially uniform, we obtain a contrast sensitivity curve that matches that reported in most textbooks and articles. Sensitivity drops to zero near 65 Hz. However, when the modulated light source contains a spatial high frequency edge, all viewers saw flicker artifacts over 200 Hz and several viewers reported visibility of flicker artifacts at over 800 Hz. For the median viewer, flicker artifacts disappear only over 500 Hz, many times the commonly reported flicker fusion rate.

Figure 1
figure 1

Data collected from ten subjects, together with median sensitivity marked as a darker line.

(a) Temporal contrast sensitivity for human observers has previously been reported to drop to zero (the critical flicker fusion rate) near 65 Hz. We duplicate those findings when the modulated light is spatially uniform. (b) When the modulated light contains a high frequency edge in the spatial domain, we measure sensitivity above 500 Hz, much higher than the previously reported rate.

Colored light

Our projector allowed for the use of a red, green, or blue light source. In the experiment above, only the green light source was used. To test that the findings were not specific to a particular color, we repeated the experiments with each light source. We used two subjects for this experiment, with all six measurements shown in Figure 2. Note that the subjects reported flicker at high modulation rates with all colors, but were most sensitive to green and least sensitive to blue, consistent with past literature on sensitivity to different wavelengths.

Figure 2
figure 2

Contrast sensitivity of two subjects when spatial edges were present, using red, green and blue LED illumination sources.

Note that the subjects reported flicker at high modulation rates with all colors, but were most sensitive to green and least sensitive to blue, consistent with past literature on sensitivity.

Spatial frequency

We hypothesize that the effect measured in this paper is due to saccades across high frequency spatial edges. If this is true, then we would expect that lower frequency spatial edges would result in lower sensitivity since for a given eye motion at a given retinal receptor the change in modulation pattern would occur at a lower magnitude. We thus repeated our experiments with the projector lens out of focus. This resulted in the edge in the projected image falling from full brightness to black over a distance of 6 mm. We used two test subjects for this experiment, with all four measurements shown in Figure 3. The subjects showed reduced sensitivity to flicker for content with lower spatial frequencies.

Figure 3
figure 3

Contrast sensitivity of two subjects when the displayed spatial frequency is reduced by defocusing the projector.

Notice that temporal sensitivity is greater for higher frequency spatial content.

Discussion

The user studies presented in this work provide a critical insight for modern display designers. The flicker fusion rate is much higher than was previously thought for some types of content and existing international display standards will need to be revised. Existing measurements of critical flicker fusion were driven by the needs of traditional displays. Traditional TVs show a sequence of images, each of which looks almost like the one just before it and each of these images has a spatial distribution of light intensities that resembles the natural world. The existing measurements of a relatively low critical flicker fusion rate are appropriate for these displays.

In contrast, modern display designs include a sequence of coded fields which are intended to be perceived as one frame. This coded content is not a sequence of natural images that each appears similar to the preceding frame. The coded content contains unnatural sequences such as an image being followed by its inverse. As one example, consider a display designed to provide 3D perception to viewers wearing glasses and 2D perception to viewers not wearing glasses. A very simple display might be built by displaying a sequence of fields which show [Left, Right, 1-Right]. Active shutter glasses that synchronize with the display ensure that the 3D viewer receives the Left and Right sub-fields to the appropriate eyes, with the third field blocked from both eyes. The 2D viewer sees the integral of light from all three fields, equivalent to a contrast reduced version of the Left image. While there are many details of this design neglected here, it contains this new sort of content with images followed by their inverse, so sort of images studied in this work. Our conclusion is that these displays require a higher refresh rate for flicker free display.

It should be noted that in addition to complex coded patterns of content, modern displays have high frequency variations due to modulated backlights and overdrive technology27,28. These high frequency variations may themselves be content dependant and interact with the effect described here, but are not the focus of this study.

The user studies in this work were carried out with a DLP projector because it provided a variable frame rate for experimentation. However we have observed the effect on every display we have available, including commercial LCD and OLED TV panels. Indeed, the authors first noticed the effect while designing a coded display pattern and started the present research study in order to explain the unexpected observations. The results presented here now serve as design guidance for our work designing better displays.

Methods

Modulated illumination was created using a TI DLP Lightcrafter. This projector allows monochrome images to be switched at a user specified rate, up to 4000 Hz. DLP projectors use time multiplexing to produce white light from RGB sub-frames. Since this would be a confounding source of time modulation we enable only one light source in each experiment. Similarly grey-levels are formed from time multiplexing, so we limit our patterns to monochrome 1-bit patterns. The DLP switching rate is on the order of microseconds, much faster than the modulation rates measured here. This effectively produces square wave temporal modulation at the specified rate.

Ambient illumination, the background level in measuring modulation contrast, is created using a Toshiba TLP-X3000 LCD projector. LCD projectors produce grey-levels by physical attenuation of light rather than time modulation. Similarly color is produced by mixing, not time-multiplexing. Our projector uses 3 LCDs so does not contain a spatial color mosaic. We use all three RGB color channels and project grey level images at different intensities. We calibrate the display for each of 255 levels of illumination. We use a darkened room so that the only ambient light is that generated intentionally.

The viewer sits 187 cm from a white wall. The modulated light is projected on an area measuring 22.9 cm × 12.4 cm. The ambient illumination extends beyond the modulated illumination, covering an area 47.7 cm × 35.9 cm. The rest of the room is dark. Figure 4 illustrates this configuration.

Figure 4
figure 4

The subject views the modulated illumination projected onto a white wall on top of unmodulated ambient illumination produced by a second projector.

The level of ambient illumination is adjusted by the subject until flicker in the modulated area is just noticeable.

All light is measured in terms of luminance in units of candela per square meter, cd/m2. We use a Sekonic L-758 photometer to measure luminance levels. The modulated light has a measured luminance of 2 cd/m2 in the “black” state and 470 cd/m2 in the “bright” state. The average luminance is thus 236 cd/m2, similar to many commercial displays. The ambient background illumination was user adjustable among 255 possible settings from 20 cd/m2 to 2700 cd/m2. Thus ambient illumination ranges from 10× dimmer to 10× brighter than the modulated display. Contrast is defined as (Lmax-Lmin)/(Lmax+Lmin), with values in the range 0.95 to 0.09.

Ten subjects are presented modulated light starting from 20 Hz and increasing to 1000 Hz, specifically [20 Hz, 33 Hz, 42 Hz, 50 Hz, 56 Hz, 63 Hz, 71 Hz, 83 Hz, 100H z, 250 Hz, 400 Hz, 500 Hz, 556 Hz, 625 Hz, 714 Hz, 833 Hz, 1000 Hz]. At each tested frequency the subjects were asked to adjust the ambient illumination using computer controls until flicker was just noticeable. The subjects were not given a fixation target and were allowed to look around if desired. A typical user adjusted the ambient level both until it was bright enough that flicker was not visible as well as dark enough that flicker was easily visible before settling into their estimate of the boundary between these conditions. Each subject is tested first with spatially uniform light and second with spatially varying light. We report the aggregate curve as well as each of the individual curves. The aggregate curve is calculated at each frequency such that 50% of viewers would notice flicker at the specified contrast ratio. Subjects reported in this work were aged 20-29, of both genders and included two of the authors. This work was approved by the UCSC IRB and was performed in accordance with the approved guidelines. The eight non-authors were informed of and consented to the task, but were not informed of the goals of the experiment until after providing observations. All subjects have normal or corrected to normal vision.