Visual crowding is a combination of an increase of positional uncertainty, source confusion, and featural averaging

Abstract

Although we perceive a richly detailed visual world, our ability to identify individual objects is severely limited in clutter, particularly in peripheral vision. Models of such “crowding” have generally been driven by the phenomenological misidentifications of crowded targets: using stimuli that do not easily combine to form a unique symbol (e.g. letters or objects), observers typically confuse the source of objects and report either the target or a distractor, but when continuous features are used (e.g. orientated gratings or line positions) observers report a feature somewhere between the target and distractor. To reconcile these accounts, we develop a hybrid method of adjustment that allows detailed analysis of these multiple error categories. Observers reported the orientation of a target, under several distractor conditions, by adjusting an identical foveal target. We apply new modelling to quantify whether perceptual reports show evidence of positional uncertainty, source confusion, and featural averaging on a trial-by-trial basis. Our results show that observers make a large proportion of source-confusion errors. However, our study also reveals the distribution of perceptual reports that underlie performance in this crowding task more generally: aggregate errors cannot be neatly labelled because they are heterogeneous and their structure depends on target-distractor distance.

Introduction

Throughout the entire visual field, vision is constrained by multiple bottlenecks in visual processing that limit the information reaching our awareness. Initially, information is lost to physiological factors such as the eyes’ optics and retinal nerve fiber density, and neural selective sensitivity to spatio-temporal patterns^1,2. However, our ability to identify even a simple object, such as a letter or an oriented grating, is far worse than predicted from these factors when the object is surrounded by clutter^3,4. These identification failures, referred to as “crowding”, occur even though adaptation after-effects demonstrate that the object’s features have been encoded, at least in primary visual cortex^5,6,7. Thus, our ability to consciously access the identity of an object is constrained by information processing capacity, not simply by retinal physiology or sensitivity limitations of the visual system.

Crowding, the inability to recognise an object in visual clutter, influences many aspects of vision. It is generally agreed to occur across the entire visual field⁸, although it is markedly more difficult to measure at the fovea⁹. As discussed in a review by Pelli and Tillman⁴, crowding affects all basic object recognition tasks, predicts reading speed and dyslexia, and is diagnostic of foveal deficits present in amblyopia¹⁰. Furthermore, it limits visibility of naturalistic images^11,12,13, and interacts with saccadic and smooth pursuit eye movements in non-trivial ways^{14,15,16,17,18,19} (but see ref. 20). There have been several recent reviews of crowding that summarize very well its ubiquity^4,21,22,23, as well as examples in which object recognition is seemingly unaffected by parameters that cause crowding in other instances e.g. ref. 24.

The spatial extent of crowding is quite similar across paradigms. Perceptual errors increase with eccentricity and decrease as the distance between target and distractor increases. The precise target-flanker distance at which crowding is alleviated at a given eccentricity, often referred to as Bouma’s constant, is somewhat variable across studies⁸, and changes dynamically according to the duration and relative timing of target and distractors^{19,25,26,27,28,29}. Despite this variability, the term “crowding” is typically taken to refer to any target identification interference that depends on target-distractor proximity³⁰. However, there are also several studies demonstrating that a metric of target-distractor distance alone fails to predict target visibility. For example, the extent of crowding varies according to: colour, shape, or polarity differences between target and flankers³¹; the duration of the crowded display²⁸; and perceptual grouping of the flankers^32,33. To the best of our knowledge, no model has been advanced to attempt to account for all of these effects.

Although there are many fascinating aspects of crowding, we focus on fundamental findings. By viewing Fig. 1, the reader can experience firsthand three phenomenal aspects of crowding that arise with stimuli similar to those used in our experiments. Following standard convention, we term these phenomena: 1) positional uncertainty^34,35, 2) feature averaging³⁶, and 3) source confusion^37,38. In this figure we present the same target stimulus, a modified Landolt C, in a series of distractor conditions. An observer’s goal is to locate the orientation of the gap section as accurately as possible. If the reader fixates the spots in succession from top to bottom, they may note that the apparent clarity of the correspondingly-coloured target orientation is affected differently in each condition. The target gap is clearest in the top row, but its position is less clear when fixating the yellow spot below, perhaps because the solid ring distractor adds noise to the positional mechanisms encoding the target orientation³⁹. When fixating the pink spot, the target and distractor gaps may perceptually blend together, shifting the perceived target orientation toward the distractor orientation e.g. ref. 40. When fixating the green spot, it is not immediately clear which of the multiple gaps is the target, and it may be easy to confuse a distractor gap for a target gap⁴¹. The changes in target visibility while viewing the yellow, pink, and green stimuli demonstrate, in order, positional uncertainty, feature averaging, and source confusion.

Figure 1: Examples of stimuli in our experiments that produce qualitatively different perceptual outcomes.

From top to bottom, rows represent the unflanked, no-gap flanker, one-gap flanker, and two-gap flanker conditions. Stimuli are drawn to scale, but were white on a grey background in the experiment. Data in later figures follow the colour conventions shown here.

The perceptual phenomena experienced in cluttered displays are typically revealed across studies employing different methodologies. Changes in positional uncertainty and feature averaging are found in experiments in which an observer is required to make a spatial judgment about a continuous property of the target, such as its orientation or relative position, for examples, see refs 40 and 42. Feature substitutions – mistaking a distractor element for a target – are mostly found in paradigms in which the observer is required to report the categorical identity of target such as a letter; trials in which the observer reports a distractor identity instead of the target reveal source confusions^37,38,43, which may be independent of an increase in positional uncertainty^44,45,46.

It is important to note that distinct categories of errors, such as averaging and substitution errors, are descriptors of results, not descriptors of a mechanism per se. Indeed, even the term crowding refers to the result of some visual process and not a mechanism. The underlying cause of crowding has previously been explained by various computational models^{11,23,40,41,42,47,48,49} and higher-level mechanistic hypotheses^24,50. Population code models, in which all visual features probabilistically contribute to perceptual reports, can produce a wide variety of data⁵¹, including so-called averaging and substitution errors⁴¹. We have thus argued that the different classes of errors reported across the crowding literature are actually arbitrary categories of the output of a single mechanism. In the present report, therefore, we use the terms “substitution” and “averaging” as a convenient way to describe patterns in our data, but not to indicate hypothesised mechanisms. Our aim in the present study is to shed further light on the causes of crowding using a single paradigm that produces multiple perceptual phenomena. Here we use experiment and modelling to quantify changes in positional uncertainty, averaging, and source confusions in visual clutter.

Methods

This experiment accorded with the protocols reviewed and approved by the Northeastern University Institutional Review Board. We tested three highly experienced psychophysical observers, including the two authors, all of whom gave informed consent. In figures, we refer to the participant naïve to the specific purposes of this experiment as N1, and to the authors as A1 (PJB) and A2 (WJH). All observers previously participated in two similar crowding experiments⁴¹.

An observer sat 57 cm from the display with their head stabilised by a chin and headrest. The display was a CRT monitor (1280 × 1024 resolution, 85 Hz). We programmed the experiment with the Psychophysics Toolbox Version 3^52,53 in MATLAB (MathWorks). Stimuli were white (100 cd/m²) on a gray (50 cd/m²) background. The target was centered 10° to the right of the fixation spot, had a 2° diameter, and a line width of 0.4°. The gap width, measured at the midpoint of the line width, was 0.4°. In the one-gap and two-gap flanking conditions, the size of the gaps remained constant across flanker diameters. For all flanking conditions, the line width remained constant (0.4°). A flankers’ outer edge was separated from the target’s outer edge by 0.4°, 0.92°, 1.62°, 2.58°, 3.9°, or infinity (ie. no flanker). We express each target-flanker condition as the flanker radius as a proportion of the eccentricity of its centre, giving 0.14 ϕ, 0.19 ϕ, 0.26 ϕ, 0.36 ϕ and 0.49 ϕ, where ϕ is the flanker eccentricity (10°). The flanker condition was selected randomly from trial-to-trial.

The orientations of flankers were constrained in the following way to produce maximal crowding effects⁴¹. In the one-gap and dual-gap flanker conditions, the flanker orientation was drawn from a normal distribution, centred on the target orientation and with a standard deviation of 22.5°. Within this range, we expect maximum levels of crowding. For the dual-gap flanker condition, a second flanker gap was drawn from a normal distribution centred 180° from the first flanker gap, with a standard deviation of 22.5°.

Each trial began when an observer pressed the space bar, which triggered the display of a small spot in the centre of the screen and the target (with or without flanks) for 500 ms. Immediately following the offset of the target, a Landolt C was presented in the centre of the display, and observers could rotate this clockwise or anti-clockwise by pressing the right or left arrow key respectively. To report an orientation, they pressed the space bar, and the next trial would begin. Observers were instructed to be as accurate as possible without rushing. They could take a break at any time by withholding a report. All observers completed 320 trials per session (20 repetitions of each target-flanker combination), for five sessions, giving a total of 1600 trials, or 100 trials per target-flanker condition. Each session took approximately 15–20 minutes.

Alternative analyses

Prior to performing the main analyses for the one-gap and two-gap flanker conditions described in the Results, we performed three other analyses in order to categorise errors. First, we performed simple linear regression, as described in the Results below. Second, we applied a mixture model used in recent visual working memory papers^54,55. This model uses maximum likelihood to estimate the proportion of trials in which an observer reported a target versus a distractor, as well as the precision with which reports are made. Third, we added two additional parameters to the mixture model in order to estimate reports corresponding to a weighted average of target and flanker⁴⁰, in addition to pure target and substitution reports. Finally, we used a Monte Carlo analysis as described in Supplementary Figure A2. Before applying these analyses to our data, we tested them on simulated datasets with known distributions. When we simulated data consisting of reports corresponding to the target orientation and substitution errors corresponding to the flanker orientation (plus Gaussian noise; 100 trials as per the experiment), both the mixed modelling and Monte Carlo analyses returned approximately the correct underling proportions of report types. However, the weighted average model was unreliable, likely due to the number of free parameters. The standard mixture model and Monte Carlo analyses estimating target and substitution reports are further outlined in Supplemental Figure A2.

Results

For all conditions, report error is defined as the difference between the reported orientation and the actual target orientation, with positive errors indicating a report that was more clockwise than the target. Because in our experiments increasing the flanker radius increases target-flanker separation, we use the terms “flanker radius” and “target-flanker separation” interchangeably. For consistency with our previous paper, figures show the flanker radius as a proportion of the eccentricity of the target center.

No-gap flanker condition

Because the no-gap flanker has no features that could be substituted or averaged with the target, results from this condition allow us to examine clearly changes in positional uncertainty. We rule out the contribution of more classic forms of masking in the Discussion. In the no-gap flanker condition, observers’ report errors clustered around 0° for all target-flanker separations (Fig. 2A). We used the Circular Statistics Toolbox (http://uk.mathworks.com/matlabcentral/fileexchange/10676-circular-statistics-toolbox-directional-statistics-) to find the circular standard deviation of observers’ reports. We refer to this measure as perceptual error, which is plotted in degrees separately for each observer in Fig. 2B–D as a function of the flanker radius. Consistent with the crowding literature, all participants’ perceptual error was higher than the unflanked condition (dashed lines) for the two smallest flanker radii. Observers N1 and A2 in particular show the characteristic relationship between performance and increasing flanker radius. Performance in the farthest flanker condition was similar to unflanked performance for all observers.

Figure 2: Results from the no-gap flanker condition.

(A) Average frequencies of each report error size (25 bins; bin width = 15°), fit with circular distributions. To improve visibility, the data have been offset slightly along the x-axis, and the x-axis has been truncated to ±90°. Only a total of 4 errors greater than ±90° were made by all observers across all no-gap flanker conditions. The inset legend shows the target-flanker separation condition. Error bars show one standard error. (B–D) Perceptual error (the circular standard deviation of report errors) plotted as a function of flanker radius for each participant. Target-flanker separation is expressed as flanker radius divided by the eccentricity of its centre (i.e. in units of Bouma’s constant). The dashed line indicates perceptual error for the unflanked condition. Error bars show 95% bootstrapped confidence intervals.

One-gap flanker condition

In the one-gap flanker condition, we have previously shown that observers’ reports correspond to the distribution of weighted responses to the target and flanker orientations⁴¹. Thus, reports may include a proportion of responses at the target orientation, the flanker orientation and their mean orientation. Quantifying reports with a unimodal circular standard deviation measure as used for the no-gap flanker condition above, is inappropriate in such a case. We express report errors as a function of target-flanker orientation difference (see Fig. 3A and Appendix Figure A1A). Under a first analysis (see Appendix A), we fit linear models to the data: reports at the target orientation have a slope of zero, reports at the flanker orientation fall on the diagonal with unity slope, and reports at the average orientation fall on the diagonal with a slope of 0.5. For all observers for the two most crowded conditions, the slopes were close to 0.5 and reduce to 0 at larger target-flanker separations. However, if these data were composed of noisy target and noisy flanker reports as described above and as has been argued previously⁴⁵, the slope of a linear fit to crowded data may give a spurious interpretation favoring the averaging model. We further applied maximum likelihood mixture modelling as described by Bays et al.⁵⁴, as well as Monte Carlo simulations, but these alternative analyses failed to return the true proportions of underlying report types of simulated data with known distributions (see Alternative analyses in Methods). We thus used a simplified approach that labels each datum according to its distance from each model prediction, as described below.

Figure 3: Results from the one-gap flanker condition.

(A) Raw report errors from the naïve participant for the smallest flanker radius condition. Solid, dashed, and dotted lines show target, averaging, and substitution models, respectively. Data are attributed to each model according to proximity. (B–D) Proportion of report type as a function of flanker radius for all observers. Report type is shown on the ordinate across panels. Data for different observers have been horizontally offset slightly for clarity.

To quantify report errors in the one-gap flanker condition, we measured the distance of all report errors from each of three underlying model predictions that correspond to target reports, averaged reports, or substitution reports, and labelled each datum according to the nearest model (see Fig. 3A). We then quantified report types as a proportion of all trials from each condition. Note that this analysis assumes that three components are necessary to quantify crowding; in Supplementary Figure A2 we show the results from mixture modelling and Monte Carlo analyses assuming reports consist of only target and substitution errors e.g. ref. 45. These proportions are shown for all target-flanker separations in Fig. 3B–D with symbols indicating observers as per the legend. The ordinate labels on the different panels correspond to different model predictions. Note that this analysis fits data simultaneously to all model predictions, and so summing across panels for one flanker radius for a single observer gives 1. The pattern is very similar for all observers: with increasing flanker radius, the proportion of target reports increases, the proportion of average reports is relatively stable, and the proportion of substitution reports decreases. Note that this pattern of results indicates that the response distribution is multi-modal, since the proportions of data for each error type changes non-monotonically. Although the proportion of target reports saturates around 0.6 (Fig. 3B), this is likely an underestimate due to a limitation of our analysis for distributions when the response standard deviation is large and the target-flanker orientation difference is small: our simulations revealed that the modelled proportion of target reports is accurate when the proportion of other model components is high (greater than ~0.3), but the proportion of target reports is underestimated when the contribution from other model components is minimal. For small flanker radii, the conditions under which we expect relatively poor performance, the proportion of each report type is likely more accurate than for larger flanker radii. Based on our previous work and the crowding literature, it is likely that observers’ reports are barely, if at all, influenced by the flanker for the largest flanker condition (for example, see Fig. 2). Indeed, our alternative analyses shown in Supplementary Figure A2 show that the proportion of substitution errors is 0 for the largest flanker radius for all observers when assuming reports consist of only target and substitution reports. Note that mixture modelling⁵⁴ also estimates the perceptual error, which we show in Figure A3 in comparison with the no-gap flanker condition. When modelled in this way, perceptual error is vastly different across flanker conditions for observer A2, but fairly consistent for N1 and A1. Finally, mixture modelling provides an estimate of the random guess rate, which was most often 0, but never exceeded 7%, in line with standard lapse rates in psychophysical testing e.g. ref. 56.

Two-gap flanker condition

In the two-gap flanker condition, one flanker gap orientation was normally distributed around the target orientation (“near gap”; s.d. = 22.5°) and the second flanker gap orientation was distributed 180° from the first flanker gap (“far gap”; s.d. = 22.5°). Because of the relatively narrow report error distributions even in the presence of a single flanker gap (e.g. Fig. 3A), we can with some confidence delineate which errors are associated with the near gap and which with the far gap. In Fig. 4A, we show the raw report errors for the naïve observer. Report errors form two clusters: one cluster centred on the y-axis at approximately 0° and another at approximately ±180°. We arbitrarily defined reports with an absolute error greater than 90° as far-gap reports. These reports are shown above and below the top and bottom dashed lines, respectively, of Fig. 4A. The proportion of far gap reports for each flanker radius and each observer are shown in Fig. 4B–D. For all observers, the proportion of far-gap flanker errors is greatest for the smallest flanker radius, and gradually decreases as the flanker radius increases.

Figure 4: Report errors in the two-gap flanker condition.

(A) Raw report errors from the naïve participant. Report errors greater than 90° or less than −90° (dashed lines) were classified as trials in which observers misreported the far-flanker gap rather than the target or near-flanker gap. (B–D) The proportion of such far gap reports as a function of flanker radius for all observers. Errors bars are 95% confidence intervals.

We next divided the two-gap flanker condition data into two subsets for further analysis. First, we examined only those report errors within 90° of the target orientation. We performed the same analysis as we did for the one-gap flanker condition. Results are shown in Fig. 5, and are highly similar to the results from the one-gap flanker condition in Fig. 3. We also performed simple linear fits to the raw data, the results of which (misleadingly) favour an averaging model (see Appendix Fig. A4).

Figure 5: Results from the two-gap flanker condition.

Data are shown as per Fig. 3. (A) Raw report errors from the naïve participant, focusing on only those errors within 90° of the target. (B–D) Proportion of report type as a function of flanker radius.

Second, we performed the same modelling on report errors that were greater than 90° from the target orientation. Due to the small number of observations in this analysis (Fig. 4), we pooled observers’ data. We re-centered this subset of data by subtracting 180° from the orientation difference between the target gap and far flanker gap, as well as from the report error. The pooled errors corresponding to the far flanker gap are shown in Fig. 6A. Because we re-centered these data, an error of 0° corresponds to a report of the target’s polar opposite orientation, whereas data falling on the line of unity are reports following the far flanker gap orientation. As in the results above, with increasing flanker radius, the proportion of target reports increases and the proportion of substitution reports decreases. The proportion of average reports is relatively stable for the three smallest flanker radii, but the proportions vary greatly for the two largest flanker radii, likely because there were only four trials in total for these conditions. In contrast to the results above, the proportion of target reports at the largest flanker radius reached 1. However, there was only a single trial that had an error greater than 90° for the largest flanker condition, so this proportion necessarily had to be one or zero. Similarly, there were only two trials in the condition with the second largest flanker radius, greatly restricting the possible proportions of report types.

Figure 6: Results from the two-gap flanker condition for report errors greater than 90° from the target with re-centered data (see text).

Data are shown as per Fig. 3. (A) Raw report errors pooled across observers. (B–D) Proportion of report type as a function of flanker radius. Error bars show 95% bootstrapped confidence intervals. Note that in the condition with the second largest flanker radius, there were only two trials. No error bars are shown for the largest flanker condition because there was only a single trial.

Discussion

We used a method of adjustment to quantify perceptual error in peripheral vision under novel crowded conditions. In all conditions, performance depended on the distance between the target and flanker, in line with the vast crowding literature⁴. Based on the phenomenological responses and appearance of crowded stimuli, three general classes of mechanism have been advanced to account for crowding: 1) positional uncertainty³⁴, 2) feature averaging³⁶, and 3) source confusion³⁷. These models are not necessarily mutually exclusive and image processing based approaches have been advanced that incorporate elements of each of these mechanisms (refs 11 and 13), but it has been difficult to reconcile which best accounts for the data because of the use of different methodologies and stimuli across studies supporting each account. Furthermore, with image processing based models that produce foveal performance deficits with synthetic images that simulate peripheral vision, it is not clear which combinations of these underlying processes accounts for perceptual performance^11,57,58. The experimental design and complementary modelling employed here provides a novel way to classify the frequency of each error type with the same stimuli and observers. Our results reveal that all error types characterise crowding with Landolt Cs, but their proportions vary with the distance between the target and flanking stimuli.

In the no-gap flanker condition, the crowding we observed at relatively small target-flanker separations is likely caused exclusively by an increase in orientation uncertainty (for example, ref. 34). In this condition, there are no flank features for averaging or substitution to occur, yet, as shown in Fig. 2, we found a reliable increase in the circular standard deviation of perceptual errors with decreasing target-flanker separation, see also ref. 41. This result is unlikely due to a form of classical masking, such as meta-contrast masking, for two reasons. First, the target duration was much greater than required for such masking⁵⁹. Second, meta-contrast masking increases random guessing⁶⁰, but our observers’ perceptual reports were not randomly distributed in the presence of close flankers (e.g. Fig. 2A). It is also difficult to account for these errors with an attentional account of crowding, in which observers’ attentional resolution is too coarse to individuate the target gap⁵: even in the smallest flanker condition, observers’ reports cluster around the actual target position, indicating that they could indeed attend to the target gap, albeit with greater perceptual error that is directly attributable to an increase in orientation variance (Fig. 2). We suggest that this orientation noise can be attributed to the solid flanking ring increasing the bandwidth of a population code that encodes the target orientation (see below and ref. 41).

The results of the one-gap and two-gap flanker conditions also support recent proposals that crowding may best be accounted for by a population code. Rather than conforming neatly to a single report error type, we found errors could be accurate, follow the flanker gap, or some average of the two. These data are thus difficult to reconcile with simple averaging or substitution models. Van den Berg et al.⁵¹ showed that many hallmarks of crowding can be explained by a biologically inspired model that simulates the responses of populations of neurons tuned to orientation within a fixed region of space (ie. a receptive field). Crowded stimuli create systematic shifts in the population code, so that when the population code is decoded, the decoded signal is prone to error. We further showed that an idealised population code can, for a given crowded stimulus, produce accurate, averaged, or substituted report errors in a probabilistic fashion⁴¹. We thus argued that there is no averaging or substitution mechanism per se, but instead that perceptual reports are drawn from the population response to the stimuli. Such a process is distinct from any single phenomenon such as source confusion, averaging or substitution but instead results from the broad spatial bandwidth of early stage filters. It is important to note that this model does not predict instances in which the magnitude of crowding is greatly modulated by certain configurations of flankers^33,61. We are not convinced, however, that these limitations argue against a population code; they instead require reconsideration of the way in which the features are weighted within the population code. We are optimistic that a texture-processing model front-end may be sufficient to model configurational effects⁴⁹, but further efforts are required.

Critically, the results from the present study show that averaging of features is not compulsory, in contrast to previous work³⁶. On a number of trials in which a flanker gap is present, observers can recover the target orientation with a precision similar to that observed with no flanker gap (Fig. 3). With only a single report, it is impossible to know if, on a single trial, an observer perceived both the target orientation and the flanker orientation, their average, or if the frequencies of these percepts varied across trials. We previously asked participants to report both the target and flanker orientations in an experiment similar to the present report, and found that participants were generally capable of reporting both elements, though they often reversed feature positions⁴¹ (see also ref. 61). Taken together, these data reveal that perceptual reports in clutter are probabilistic, but relatively fine detail can be recovered. Our results are in line with a conceptually similar earlier study by Põder and Wagemans⁶², who had observers report a target Gabor defined by spatial frequency, colour, and orientation. Flankers were Gabors with various combinations of colours, spatial frequencies, and orientations, allowing the authors to quantify the extent and category of error types in crowding. In brief, they found observers often mistook an entire flanker for a target, or reported some flanker features as belonging to the target, as would be expected from frequent substitution errors as per our study.

It is clear from our data that observers often report an orientation closer to the flanker orientation instead of the target orientation (Fig. 4). However, our findings suggest the proportion of substitution-type errors is substantially greater than the proportion of substitution errors that occur when whole letter stimuli are used⁴³. It is likely that this discrepancy can be explained by the task differences. Letter report paradigms limit the response range and so the observer is forced to select the most similar letter, even if the perceived stimulus does not match any of the possible responses. Our response method does not suffer from this limitation.

Since submission of this article, Agaoglu and Chung⁶¹ published a study using a paradigm motivated by our earlier paper that investigates the validity of the present design. They levelled multiple criticisms of the continuous report approach and population response models of crowding. In short, they had two main aims: to test predictions made by an assumption of the population code model and to determine whether the relationship between the target’s orientation and its eccentricity affected behavioural reports. Like previous crowding models, our model assumed that target and flanker orientations were weighted according to their mutual distance. This assumption predicts stronger crowding for flankers inside the target than outside. Agaoglu and Chung tested this prediction and found the opposite result, and therefore concluded that this choice of weighting field is incorrect. This striking observation challenges the many models that predict crowding based on the centre-to-centre spacing of target and flankers⁴. As discussed above, this finding does not negate the appropriateness of a population code model, but it does suggest that interference zones in crowding may be nonlinear. Agaoglu and Chung also found that observers’ perceptual errors varied according to the absolute orientation of the target and flankers, consistent with eccentricity effects. We reported a similar finding in our original study, and measured these differences across a range of target-flanker separations to model the variable interference zone around the target (see Supplemental Fig. 1 of ref. 41). How these dependencies affect the present data requires future testing. However, they do not undermine the merit of continuous report designs nor do they render the paradigm inappropriate for measuring crowding. In fact, their data demonstrate how this paradigm can be used to provide a rich analysis of perceptual experiences under crowding. These observations raise many important questions about the data obtained in many previous studies that employed similar stimuli but different response methods and analyses.

Our results thus provide new evidence revealing that the component features of visual objects can be individuated even far in the periphery, although their relative positions and orientations may appear noisy and confusable across trials. The level of detail made available by the visual system has been heavily debated both within the crowding literature and more generally. Our data suggest one possible reason for this apparent conflict in the literature. Note that our analyses suggest all observers have very high rates of substitution-type reports under crowded conditions: combining the proportion of far-flanker gap substitutions (Fig. 4) and proportion of near-gap errors (Fig. 5), observers made approximately 50–70% substitution-type errors in the most crowded conditions. Such performance may be misinterpreted in alternative forced choice experiments (AFC), a more common psychophysical paradigm in which an observer is forced to categorise a target as being one of usually two to four alternatives. In these experiments, such a high proportion of substitution-type reports could render performance at or close to chance, leaving it unclear if a participant was randomly guessing, reporting an average stimulus or reporting what they thought was the target on some trials and the flanker on other trials. This is especially true in experiments with simple stimuli such as lines or Gabors, though it is less problematic with letter stimuli e.g. ref. 38, 43.

In conclusion, our findings show that relatively fine featural detail is not necessarily lost during early visual processing, but the precision of each perceptual report is corrupted. The aggregate of errors made when viewing crowded displays cannot be characterised as simply being accurate, averaged or substituted. The variety of report error types that occur within the same paradigm, as demonstrated here, provides a continued challenge to models of visual crowding.