Neural representation of visual concepts in people born blind

How do we represent information without sensory features? How are abstract concepts like “freedom”, devoid of external perceptible referents, represented in the brain? Here, to address the role of sensory information in the neural representation of concepts, we used fMRI to investigate how people born blind process concepts whose referents are imperceptible to them because of their visual nature (“rainbow”, “red”). Activity for these concepts was compared to that of sensorially-perceptible referents (“rain”), classical abstract concepts (“justice”) and concrete concepts (“cup”), providing a gradient between fully concrete and fully abstract concepts in the blind. We find that anterior temporal lobe (ATL) responses track concept perceptibility and objecthood: preference for imperceptible object concepts was found in dorsal ATL, for abstract (non-object, non-referential) concepts in lateral ATL, and for perceptible concepts in medial ATL. These findings point to a new division-of-labor among aspects of ATL in representing conceptual properties that are abstract in different ways.


Supplementary Figure 1: Main effects for abstract and concrete concepts in each group
The whole-brain, random effect contrasts of abstract > concrete concepts (left panels) and concrete > abstract (right panel) are depicted for the entire group of subjects (n=26, A,D) sighted subjects only (n=14, B, E) and blind subject group (n=12, C, F). Data from Exp. 1.

Supplementary Figure 2: Group effects in the comparison between abstract and concrete concepts
A 2-way ANOVA was computed for Abstractness (A; abstract vs. concrete concepts) and Group effects (B; blind, sighted; data from Exp.1). While the abstractness effect is found in left ATL across groups, neither group effect or abstractness X group interaction (C) is found in this region.

Supplementary Figure 3: No group X imperceptibility interaction in the abstract concept network outside ATL
A. Replica of Fig. 1C: A contrast of typical abstract words (e.g. "freedom") with concrete everyday objects (e.g. "cup") in conjunction with abstract words significant activation in the combined subject group (n=26; data from Exp. 1). Clusters outside dorsolateral ATL are numbered, and data was sampled from Exp. 2 from each of these, none showing significant group X imperceptibility interaction.
E. Data sampled from Cluster 4 in the ATL pole in Exp. 2 shows no significant group X imperceptibility interaction (F(1,22)=0.18, p=0.68). Error bars reflect mean squared error.

Supplementary Figure 4: Neural pattern in dorsal ATL reflects imperceptibility in the blind
A-B. Multivariate representational similarity analysis (RSA) was computed based on a behavioral matrix based on ratings of the sensory perceptibility/accessibility of the astral and scene concepts collected from an independent group of congenitally blind subjects, who did not participate in the fMRI experiment (RDM in panel A, n=6). These ratings were highly correlated to those of the fMRI participants (r 2 =0.81, p<0.001). Neural pattern correlation (B) to these ratings (data from Exp.2) was also found in the dorsal ATL.

C-D.
RSA was computed also based on a behavioral matrix based on ratings of the "visualness" (visual dominance) of the astral and scene concepts collected from an independent group of sighted participants, who did not participate in the fMRI experiment (panel C, n=45). These ratings were negatively correlated to those of the sensory perceptibility of the blind participants (r 2 =0.45, p<0.001). Neural pattern correlation (D) to these ratings was also found in the dorsal ATL of the blind (data from Exp.2).

E-G.
RSA was computed also based on ratings of the blind subjects of the sensory perceptibility/accessibility of the concepts (panel E, replica of Fig. 3A), with the neural patterns in a searchlight manner across the brain while using behavioral ratings of abstractness, imaginability, manipulability, emotional valence and emotional arousal as nuisance regressors. Neural pattern correlation (F) was found in the dorsal ATL, overlapping the effects of imperceptibility X group interaction and abstract concepts preference, controlling for any collinearity between sensory perceptibility ratings and other factors (data from Exp.2). The same is found when also including referentiality as a nuisance regressor (panel G), indicating that the effect of imperceptibility is separable from this effect.

H-J.
RSA was computed also based on ratings of a group of blind subjects of the referentiality of the concepts (n=15, panel H), with the neural patterns in a searchlight manner across the brain alone (panel I) or while using behavioral ratings of perceptibility, abstractness, imaginability, manipulability, emotional valence and emotional arousal as nuisance regressors (panel J). Referentiality/objecthood was defined as the extent to which each concept describes something that could be pointed out in the external world. Referentiality neural pattern correlation (I,J) was not found in ATL in either the blind group (presented here; data from Exp. 2) or sighted group. This may be due to an underpowered analysis, as only the abstract concepts show reduced referentiality as compared to the other concepts (data from Exp. 2).

Supplementary Figure 5: Group effects in the comparison between abstract and imperceptible astral concepts (objecthood)
A 2-way ANOVA was computed for Objecthood (A; abstract vs. imperceptible astral concepts) and Group effects (B; blind, sighted; data from Exp.2). While the Objecthood effect is found in left ATL across groups, neither group effect or abstractness X group interaction (C) is found in this region.

Supplementary Fig. 6: Imperceptibility and domain interaction
A. Data sampled from the imperceptibility X group interaction cluster pdATL shown in Fig. 2A shows that this area does not show a consistent imperceptibility effect across content domains in the blind group (F(2,11)=1.7, p<0.21; data sampled from Exp. 2). Instead, it shows an interaction between imperceptibility and content domain in the blind (F(2,22)= 5.73, p< 0.01). Error bars represent standard error of the difference between means for the perceptible and imperceptible words in each content domain. B. Data sampled from pdATL in the sighted group (from Exp. 2). The interaction of imperceptibility with group, evident in pdATL in Fig. 2A appears to result from a bias in the sighted group, towards concepts which are imperceptible to the blind. This cannot be explained in terms of perceptibility, as all these concepts are similarly perceptible to the sighted. Error bars represent standard error of the difference between means for the perceptible and imperceptible words in each content domain. C. We computed a 2-way ANOVA for imperceptibility and content domain in the blind to inspect the interaction between them separately from the group effect (data from Exp.1). The imperceptibility effect in dorsal ATL replicates the findings of the contrast of imperceptible vs. perceptible concepts in the blind (Fig. 2C), showing that this effect is found regardless of content domain and depends on imperceptibility of the concepts only. D. Content domain effects in the blind are found in the inferior ATL as well as in the IFS (data from Exp.1). E. Whole-brain imperceptibility X domain interaction in the blind (data from Exp.1). Despite the significant interaction between imperceptibility and domain in the pdATL ROI, this effect does not survive a stricter whole-brain analysis. In ATL, interaction is found only in the uncus, in medial ATL. However, this region shows insignificant or negative BOLD in response to our experimental conditions in Exp. 2, making the interaction problematic to interpret. Weaker interaction is found more posteriorly, in the parahippocampus, potentially due to the scene category 1,2 .

Supplementary Figure 7: Functional connectivity of ATL in the blind
Functional connectivity of the ATL peaks was computed from pairs of ATL seeds reflecting differential processing: the dorsal vs. lateral ATL (A; adATL and lATL) and dorsal vs. medial ATL (B; adATL and mATL). Although the dorsal and medial sites seem to share more connectivity in the blind (B) than in the sighted (Fig. 6B), no large-scale RSFC group effects are found for any of the ATL seeds (C,D,E).

Supplementary Figure 8: Temporal signal to noise ratio in imaging ATL
the averaged temporal signal-to-noise ratio (tSNR, the ratio of the average signal intensity to the signal standard deviation) is presented for each group (A: sighted group, B: blind group). The maps show high quality signal coverage over ATL, although values are lowest in the ATL pole. It is possible that in the ATL pole relatively lower SNR could lead to low detection power, which may be improved using additional measures in future studies.

Supplementary Tables
Supplementary 1 The overall imperceptible vs. perceptible design includes the comparison between the imperceptible and perceptible words in the three tested domains: astral/weather phenomena (e.g. "rainbow" vs. "rain"), scenes ("island" vs. "beach") and object features (colors vs. shapes, e.g. "red" vs. "square"). We first tested for the difference between the imperceptible and perceptible concepts using a mixed-effects ANOVA.