Interaction of surface pattern and contour shape in the tilt after effects evoked by symmetry

Integration of multiple properties of an object is a fundamental function of the visual cortex in object recognition. For instance, surface patterns and contour shapes are thought to be crucial characteristics that jointly contribute to recognition. However, the mechanisms of integration and corresponding cortical representations have not been fully clarified. We investigated the integration of surfaces and shapes by examining the tilt after effects (TAEs) evoked by the symmetry of patterns and contours. As symmetry in both pattern and contour evokes TAEs, we can directly measure the interaction between the two. The measured TAEs exhibited mutual transfer between the symmetry of the pattern (SP) and that of the contour shape (SS), i.e., adaptation by SP (SS) evoked TAEs when tested by SS (SP), suggesting the existence of an integrated representation. Next, we examined the interaction between SP and SS when both were simultaneously presented in adaptation. Congruent adaptors wherein their symmetry axes aligned evoked compressive interaction, whereas incongruent adaptors wherein the axes of SP and SS tilted to the opposite directions evoked subtractive interaction. These results suggest the existence of a cortical representation that integrates the properties of the surface and shape with suppressive interactions, which can provide crucial insights into the formation of object representation as well as the integration of visual information in the cortex.


Supplement 1. Blank condition
To examine the exact amount of TAE evoked by adaptation, we measured the perceptual tilt of the symmetry axis in the test stimuli without adaptation (blank condition: TABB). A blank screen with midgray was presented during the initial and top-up adaptation periods. The stimuli and experimental procedure were identical to those used in the main text. The measured tilts of the P, S, and P+S test stimuli were 0.398 o , 0.435 o , and 0.018 o , respectively, as shown in Fig. 2 (P and S test) and Fig. 5 (P+S test). The measured tilt was not significantly different from zero (one-sample t-test; P test: t(59) = 1.75, p = 0.086; S test: t(59) = 1.35, p = 0.182; P+S test: t(59) = 0.098, p = 0.922). These results indicate that the test stimuli evoked the perception of tilt, similar to the geometrically defined tilt.

Supplement 2. Statistical analysis
p, adjusted: p-value obtained using a pairwise t-test with Bonferroni correction. p, random: p-value obtained by a pair-wise randomization test with Bonferroni correction. A randomization test was performed when normality and/or equal variance were violated.

Supplement 3. Control experiment for TAE evoked by shape symmetry
To examine whether the stimuli with SS in fact evoked TAE, we measured TAE for two control conditions. First, we tested the effect of randomly placed dots within the egg-shaped contour by removing the dots from the SS stimuli used in Exp. 1 (TAES-ND, S-ND). Second, we tested the effect of shape afterimage by alternatively presenting two stimuli (the original SS stimulus and its upside-down image; (TAES-SW, S-SW)).
The procedure was identical to those in Exp.1. Four participants (one female and three male) carried out the experiment; all of them were different from those performed Exp. 1 to 3, but the procedures and attributes were identical to those described in the Participants section in the main text.
The mean measured TAE magnitudes across the participants are shown in Fig

Supplement 5. The responses of the model
We propose a conceptual model comprising the representations of pattern, shape, and their integration with mutual suppression between SP and SS through the formation of the integration, as described in the Discussion section. This supplementary note describes the responses of the model that qualitatively reproduces the observed TAEs under all conditions.

Reproduction of TAES, S ---control condition
When an adaptor with SS (S adaptor, the red bar at the top of panel (a) ) is presented with the symmetry axis tilted to q, neural circuits responsible for the shape representation and that responsible for the integrated representation are activated (as illustrated by the red bars next to the representations) and fatigued. When a test stimulus with SS (S test, the green bar at the top of the panel (a) ) is then presented, both the shape and integrated representations exhibit TAE in the opposite direction to the symmetry axis of the adaptor (the light-blue bars, tilted to -q ). The perception of the tilt (the blue bar; TAES, S) depends on the tilt of the integrated representation. For the sake of simplicity, we describe here that the TAE (-q ) is the opposite to the tilt of adaptor (q ). TAEP, P is explained similarly (refer to the panel (c) ) but with a smaller magnitude of TAE because of a weaker connection from the pattern representation to the integrated representation.

Reproduction of TAES, P ---transfer condition
We consider transfer condition with the S adaptor and P test (the red and green bars at the top of panel (b), respectively). With the S adaptor, the shape and integrated representations are activated and adapted (the red bars next to the representations) but not the pattern representation. When the P test is then presented, no TAE is evoked in the pattern representation. Although the pattern representation is not adapted, the integrated representation is adapted, so that a smaller degree of TAE is observed compared to that measured with the S test (TAES, P < TAES, S), indicating the partial transfer from SS to SP.

Reproduction of TAEP-S, P
With the P test stimuli, the measured magnitude of TAEP-S, P was significantly smaller than that of TAEP, P in Exp. 1. The model reproduces this observation as shown in panel (a). Under the condition of TAEP-S, P, the directions of TAE in the pattern and integrated representations are opposite because the directions of adaptation are opposite in SP and SS, and the SS is dominant through the integration of SP and SS. When the P test stimulus is presented, the pattern representation yields TAE with a magnitude similar to TAEP, P.
However, it is then canceled and the TAE in the integrated representation is rather biased toward the SS, resulting in TAEP-S, P < TAEP, P.

Reproduction of TAEP-S, S
With the S test stimuli, the measured magnitude of TAEP-S, S was smaller than that of TAES, S in Exp. 1, but the difference was not statistically significant. The model reproduces this observation as shown in panel (b).
Since TAEP-S, S and TAES, S share the S test stimuli and the adaptation in the shape and integrated representations (compare the panel (b) and S5-1(a)), similar degrees of TAEs are expected (TAEP-S, S = TAES, S). Note that the adaptation by the P-S adaptor in the integrated representation is considered to be similar to or slightly weaker than the adaptation by the S adaptor despite the incongruent adaptation. This is because the mutual suppression between the SP and SS through the formation of the integrated representation acts as disinhibition.

Reproduction of TAEP-S, P+S
With the P+S test stimuli, the measured magnitude of TAEP-S, P+S was significantly smaller than that with the S test stimuli (TAEP-S, S; refer to the panel (b) ). The model reproduces this observation as shown in panel (c).
When the P+S test is presented, the pattern and shape representations produce TAEs similar to TAEP, P and TAES, S, respectively, in opposite directions. Therefore, they cancel each other in the integrated representation. The cancelation leads to a smaller magnitude in TAEP-S, P+S than in TAEP-S, S (TAEP-S, P+S < TAEP-S, S).

Reproduction of TAEP+S, P and TAEP+S, S
With the S test stimuli, the measured magnitude of TAEP+S, S was not significantly greater than TAES, S (refer to Fig. S5-1(a)). The model reproduces this observation as shown in panel (b). Since the pattern and shape representations share the same direction of adaptation (q ; as illustrated by the red bars next to the representations), it might be argued that the degree of adaptation in the integrated representation could be increased. However, the mutual suppression between the SP and SS in our model prevents the increase of adaptation in the integrated representation as illustrated by the red bar next to the integrated representation.
Note that the degrees of adaptation in the shape and integrated representations are identical to those evoked by the S adaptor. Therefore, when the S test is presented, a similar magnitude of TAEP+S, S is evoked compared to TAES, S. Note also that, during the presentation of test stimuli, the suppression operates effectively only when both SP and SS are present as with the P+S test but not with the S test. The mutual suppression through the formation of the integrated representation contributes to the reproduction of TP+S, S = TS, S. Similarly, the model reproduces TAEP+S, P = TAEP, P, as shown in the panel (a).

Reproduction of TAEP+S, P+S
With the P+S test, the measured magnitude of TAEP+S, P+S was significantly smaller than that with the S test, TAEP+S, S. The model also reproduces this observation as shown in panel (c). When the P+S test is presented after the adaptation, unlike the S test, the signals from both pattern and shape representations come into the integrated representation, so that the suppression operates effectively; thus, a smaller degree of TAE is evoked compared to that with the S test (TAEP+S, P+S < TAEP+S, S). The red bar also illustrates adaptation in the shape representation, i.e., neurons responsive to this tilt are activated and fatigued. A light-blue bars on the right of the shape representation illustrates the expected TAE in the representation (-q ) resulting from the adaptation. The light-blue bar at the bottom of the integrated representation also illustrates the expected TAE in the representation. The blue bar at the bottom shows the TAE measured by psychophysical experiments. (b) Transfer condition with the S adaptor and P test. This panel is identical to that shown in Fig.6a. (c) Control condition with the P adaptor and P test. As illustrated by the thickness of the lines between the representations, the shape representation has a greater influence than the pattern representation through the formation of the integrated representation; thus, TAEP, P is smaller