Vision in the eternal present

When we are searching our visual environment, it might be expected that we accumulate records or memories of the locations that have been scanned. But a new study indicates that visual search can operate without the guidance of such memories.

As quickly as you can, decide whether or not Fig. 1 contains the letter T. For decades¹, it has been thought that visual-search tasks such as this proceed in much the same way as a well-organized person might search a pile of puzzle pieces for the one that fits a particular hole. A piece is first selected and compared to the target. If the piece fits, the search is over; if not, the piece is laid to one side so that it won't be selected again, and a new piece is selected. By putting the pieces aside, the searcher can remember which ones have already been examined. But a less organized or more easily frustrated person might simply throw the pieces back into the pile if they don't fit. This search could go on indefinitely, returning time and again to pieces that have already been assessed. Such an ‘amnesic’ approach to finding something appears to be strikingly inefficient. Yet, as reported by Horowitz and Wolfe² on page 575of this issue, the visual system seems to operate in just this way.

Figure 1: Visual-search task.

Horowitz and Wolfe² suggest that, contrary to expectations, in searching this figure for the letter T, your visual system does not remember which positions in the figure have already been scanned.

The method used by Horowitz and Wolfe is simple and clever. In difficult visual-search tasks, such as that shown in Fig. 1, the time taken to find a target increases more or less linearly with the total number of items in the display. The speed of perceptual processing, in terms of average time per display item, is measured by the slope of the line that relates the search time to the number of items (the ‘search function’). Horowitz and Wolfe measured this slope under two conditions. In the ‘static’ presentation, all items remained in the same location — as is typical in search experiments. But in the ‘random’ presentation, the target and all of the other display items changed position roughly every 100 ms.

If search performance depends on remembering which locations have already been examined, the continual changes in the random presentation should be disastrous. The effects of this disruption will be relatively small with few items in the display, and get increasingly worse as more items are added. So, the standard, memory-based model of search predicts that there will be much greater search slopes with random presentations than with static ones. But, in three experiments, Horowitz and Wolfe found equivalent search slopes for both types of presentation. In other words, an amnesic model of search seems to be correct. As the authors describe it, the visual system as exposed in these tasks appears to operate in an ‘eternal present’.

The results pose a puzzle. In these tasks, the visual system operates with no regard for or memory of previous visual states, even when this could be beneficial. Yet visual and memory systems are clearly not entirely separate — we can form memories of what we see, and can guide visual processing based on remembered locations and forms. So how can we explain the dissociation? Perhaps the visual system is designed to take nothing for granted. In the real world, where previously unoccupied locations may suddenly contain a bear or parking-meter attendant, it could be a mistake to rely too heavily on memories from previous visual analysis.

Not only can assumptions based on memories of a changing world be potentially misleading, but there is also a computational cost to constructing and maintaining these memories. Even if a visual stimulus is static and unchanging, the corresponding images projected on the retina are certainly not, as the observer moves their eyes, head and body. Loosely speaking, two strategies are available to deal with this changing series of images. One is to continually compute transforms that allow the stimulation received within a retina-centred frame of reference to be mapped onto a set of coordinates centred on some stable external point of reference³. The amnesic strategy might be to forego this computation and get updates, as needed, from the environment itself.

For some tasks the amnesic approach may be sufficient. Consider Herbert. He's a robot whose sole function is to roam through a crowded office building and collect empty soft-drink cans⁴. Herbert has what might be called a profoundly amnesic visual system. If Herbert senses a can, he moves towards it; if he senses an obstacle, he moves around it. Herbert does not maintain memories of where cans or obstacles can be found, and would have no basis for realizing that a can or obstacle had moved or even changed form. Nevertheless, Herbert is reasonably effective at his simple job, responding to unforeseeable changes in the environment without the computational costs of memory.

Of course, human behaviour — and the demands on our visual system — is much more varied than Herbert's. Unlike Herbert, we may have basic, opposing tensions on the need to recognize change. On the one side there is the necessity to respond to the present visual state, regardless of what might have occurred in previous states. A system that operates solely in the present is well suited to meet this challenge. On the other side, change in itself often indicates where and how behaviour should be directed. It may be prudent to investigate the sudden appearance of new objects. Further research may show how the visual system balances the pressures to live in the present while remembering what is useful about the past.