Introduction

Orientation is considered one of the most basic features in vision1. It is extracted very early in visual processing by specialized neurons in the primary visual cortex2 and plays a vital role as one of the essential building blocks of visual perception. The receptive fields of neurons in the primary visual cortex (area V1) are narrowly tuned to the orientation of contours at a particular spatial scale. Collectively, the population of V1 neurons encodes all orientations over a wide range of spatial scales3. Similar to vision, neurons in the early somatosensory cortex (areas S1 and S2) show an orientation selectivity. Although not as specialized or narrowly tuned for orientation as V1 neurons, the population of all orientation-tuned somatosensory neurons covers all orientations at a range of scales4,5,6. Given these similarities and considering the value of efficient sensory coding, it is natural to wonder whether there is a common processing of orientation between the visual and somatosensory modalities.

Various findings have accumulated in support of the view that orientation processing in visual and somatosensory modalities is linked. First, a number of studies using various brain imaging methods have reported occipital lobe (visual cortex) involvement during purely tactile tasks with both sighted and visual-impaired individuals7,8,9,10,11,12,13,14,15. These findings all point to the involvement of extrastriate visual cortex during the processing of a variety of tactile tasks. More importantly, they reveal that the spatial scale of the task feature regulates the degree of visual involvement: large features such as size, shape and orientation (macrospatial tactile scale) tend to activate the visual cortex whereas fine-grained or microspatial tasks (e.g., texture and roughness, for which somatosensory neurons are specialised) do not. One interpretation is that macrospatial tactile information is passed onto the visual cortex because vision is more specialized for such information; and that this occurs in a top-down manner, possibly as visual imagery7,9,13,15,16,17.

Several neuroimaging studies have shown the involvement of extrastriate visual cortex during tactile orientation discrimination (macrospatial task) which is absent during fine tactile spatial frequency discrimination (microspatial task). Evidence for this comes from a positron emission tomography (PET)16 study and a functional magnetic resonance imaging (fMRI)15 study which found that orientation discrimination produced a significant task-specific activation in extrastriate visual cortex compared with a tactile texture discrimination. Stronger evidence comes from a transcranial magnetic stimulation (TMS)13 study which showed that TMS applied over the extrastriate visual area significantly impaired participants’ ability to discriminate grating orientation (macrospatial) but did not affect grating texture (microspatial) discrimination. In an interesting asymmetry, TMS applied over somatosensory cortex had a severe impact on both tasks—suggesting the visual cortical component for the grating orientation task originated from somatosensory cortex. These results suggest differing processing pathways for micro- and macro-spatial tactile tasks: while both require the somatosensory cortex, macrospatial tasks such as orientation also require the involvement of visual cortex. Behavioural data complementing these findings come from a perceptual learning study of grating orientation which showed tactile-to-visual transfer of orientation learning18. Together, these findings support the hypothesis that tactile orientation information is passed on to the extrastriate visual cortex, presumably to exploit vision’s finer orientation acuity.

There is another line of evidence implicating connections between the somatosensory cortex and early visual cortex, including area V1. Neuronal recordings in rats have found primary somatosensory cortex (S1) and primary visual cortex (V1) carry similar amounts of information during purely tactile tasks, and V1 responses correlate strongly with task performance19. In humans, one study found activation in V1 and deactivation in extrastriate visual cortex during a tactile dot spacing task20. Complementing this, psychophysical studies show that visual contrast sensitivity for grating stimuli is improved when paired with a tactile grating that is congruent with the visual stimulus. Critically, varying the relative orientation between the gratings reveals the crossmodal facilitation has a very tight orientation tuning, implying an interaction at the earliest cortical stage where orientation tunings are tightest21. Another study found a similarly tight orientation tuning between touch and vision in a visual suppression paradigm in which visual gratings were less suppressed when paired with a congruent or near-congruent tactile grating22. These findings support the possibility of direct connections between primary visual and somatosensory cortices, such as those documented in rodent brain23,24, and hence an early integration of visual and tactile orientation information. Thus, while there is a considerable accumulation of evidence documenting visual-tactile interactions for orientation tasks, it is not clear at which cortical stage these interactions occur.

Adaptation and aftereffects often serve as valuable tools for investigating underlying neural mechanisms through behavioural experiments. In multisensory research, numerous studies have utilized the adaptation paradigm to investigate various combinations of visual, tactile and auditory modalities25,26,27,28,29. Specifically, motion aftereffects have been shown to be transferable between vision and touch, offering behavioural support for the hypothesis of a shared representation of motion29. Therefore, establishing a similar crossmodal adaptation effect for orientation perception could provide additional behavioural evidence for the proposed common processing mechanism of orientation information between touch and vision. The tilt aftereffect (TAE) is an old and well-known visual aftereffect that has revealed much about visual orientation mechanisms30. The TAE occurs when a tilted stimulus presented for a prolonged period causes nearby orientations to appear repelled away from the adaptor’s orientation. These perceptual repulsion effects are linked to corresponding changes in the cortical responses of V1 neurons31,32. These adaptation-induced neural changes, similar to many other visual aftereffects33, are traditionally explained by a simple fatigue model in which neurons preferring the adaptor orientation show reduced responsivity after adaptation, increasing the relative response strength to nearby orientations. In recent years, several studies have tried to explore orientation adaptation in somatosensation. A tactile TAE has been demonstrated on the palm of the hand using a two-point stimulation method34, and visual adaptation to tilted gratings biases perception of subsequent two-point tactile orientation stimuli on the forehead35. In addition, combining tactile and visual surround gratings is found to increase the visual tilt illusion36. These exciting findings all point to the similarity in adaptation effect between the two modalities, which further supports the hypothesis of a close connection between the two modalities in orientation processing. However, a few important aspects have not been comprehensively investigated in the above-mentioned studies which motivate our current study. To date, there has been no systematic demonstration of tactile and crossmodal orientation aftereffects within the same paradigm. By testing whether the same tactile stimulus could lead to a tactile TAE and a crossmodal TAE when tested in vision, and whether a visual TAE can be transferred between vision and touch, it could help demystify whether the orientation adaptation effect is just a result of separate but analogous processing in different cortical areas, or whether it arises from the same process (i.e., tactile orientation information being processed in the visual cortex).

Moreover, besides the negative adaptation effect, another form of bias induced by the previous stimulus, namely serial dependence, has also been reported in visual orientation perception37. Unlike neural adaptation, which is believed to be low-level and repulsive, serial dependence in visual orientation has been reported to be mostly attractive, with the stage at which it occurs still under extensive debate38,39,40,41,42,43,44,45,46,47. Serial dependence has not yet been reported in tactile orientation perception, nor compared to its visual counterpart. Given the close connection between vision and touch in orientation perception, it would be hypothesized that tactile orientation perception might also exhibit similar serial dependence. By including a test sequence in our studies, we aim to further explore the intricate mechanism of temporal influence on orientation perception in both modalities.

Given the ample evidence pointing to the common orientation processing mechanism between the two modalities, we hypothesized that tactile orientation perception on the finger pad would exhibit a similar TAE to its visual counterparts. Additionally, we posited that if tactile orientation information is indeed processed in the visual cortex, adaptation in one modality would be transferable to the other. We conducted two experiments: experiment 1 tested TAE in touch, while experiment 2 examined crossmodal TAE by testing for transfer in both directions. Our results reveal a robust tactile TAE and also show that adaptation effects can transfer between modalities in an asymmetrical way, where tactile adaptors bias subsequent visual perception, but not vice versa. These results provide concrete evidence of touch and vision engaging common orientation mechanisms, while the asymmetricity of adaptation transfer is informative about the links underlying visual-tactile orientation interactions. In addition to the main findings, we also took a closer look at how the adaptation effect changes after the adaptation phase by further examination of the trial-by-trial variability during the test phase. We found a similar asymmetrical intramodal serial dependence effect between the test trials, which provides additional insights into the potential source of this asymmetricity.

Figure 1
figure 1

(a) Experimental setup. The tactile stimuli were presented via a device positioned in front of the screen and the instructions and visual stimuli were presented on the screen. (b) Examples of adaptation (left image) and test (right image) visual stimulus used in Experiment 2. Both types of stimulus were orientation-filtered pink noise with different RMS contrast levels (adaptation: 0.300, test: 0.018), oriented at \(-27^\circ\) or \(27^\circ\). (c) Illustration of the tactile grating used in both experiments. (d) The three sets of tactile stimulus orientations used in Experiment 1. To deal with the large variability in orientation sensitivity among different participants, three sets of stimulus orientations with the same mean and varying range were used in Experiment 1 as shown in the figure (\(-16.2^\circ\) to \(16.2^\circ\), \(-21.6^\circ\) to \(21.6^\circ\) or \(-27^\circ\) to \(27^\circ\)), each set consisted of six different orientations (\(5.4^\circ\) gap, \(7.2^\circ\) gap or \(9^\circ\) gap, respectively). For each participant, one of the three sets would be selected based on their performance in the preliminary experiment, such that their performance would be in the reasonable range for psychometric function fitting. In Experiment 2, only the largest set (\(-27^\circ\) to \(27^\circ\) with \(9^\circ\) gap) was used.

Full size image
Figure 2
figure 2

Procedures for Experiment 1 and Experiment 2: (a) Experiment 1: the experiment consisted of two phases, an adaptation phase (initial: 30 s, top-up: 5 s) and a test phase. During the adaptation phase, participants were cued to passively feel the orientation of a tilted grating (\(-27^\circ\) or \(27^\circ\)) without making a response cued by the “No Response” text on the screen. During the test phase, one of the six test stimulus orientations was presented for 1.5 s. Participants were then cued by the “Response” text on the screen to respond using the arrow keys on the keyboard to indicate whether the perceived orientation was left or right of the vertical (proximal-distal) axis. (b) Experiment 2, Condition 1: tactile adaptation, visual test (TV): during the adaptation phase (initial: 60 s, top-up: 10 s), participants were cued to perform multiple tactile orientation discrimination tasks, with the pseudo-randomised tactile stimulus that sits around either \(-27^\circ\) (leftward adaptation condition) or \(27^\circ\) (rightward adaptation condition), after which participant went through the test phase that consisted of a sequence of six visual test stimuli, with each stimulus presented for 150 ms. (c) Procedure for Experiment 2, Condition 2: visual adaptation, tactile test (VT): during the adaptation phase (initial: 60 s, top-up: 10 s), participants were instructed to fixate on the orientation-filtered pink noise pattern (\(-27^\circ\) or \(27^\circ\)), during which they were also instructed to perform a dot location task, to ensure their fixation was on the visual adaptor. After this, participants went through the test phase which consisted of a sequence of two tactile test stimuli, with each test stimulus presented for 1.5 s.

Full size image

Results

Two experiments were conducted to assess the tactile tilt aftereffect (TAE) and the transfer of TAE between vision and touch. Figure 1 illustrates the experimental setup and example stimuli. Participants underwent an adaptation phase in which tilted stimuli (Experiment 1: tactile; Experiment 2, Condition 1: tactile, Condition 2: vision) were presented for a prolonged duration (Experiment 1: 30 s, Experiment 2: 60 s). Following this, test stimuli were presented either in the same modality (Experiment 1: tactile) or different modalities (Experiment 2, Condition 1: visual; Condition 2: touch). Participants then indicated their responses by judging the orientation of the stimulus with respect to the vertical axis (proximal-distal direction in touch). Psychometric functions were fitted for each participant under each adaptor orientation (left vs. right). The point of subjective equality (PSE) was obtained from the fitted psychometric functions and was used as the measure of adaptation-induced orientation bias, with positive values indicating a leftward bias and negative values indicating a rightward bias. The TAE is then quantified by the difference in PSE (\(\Delta PSE = PSE_{right} - PSE_{left}\)) between the two adaptor orientations (left vs. right), where positive values indicate a repulsive TAE and negative values indicate an attractive shift. See Methods for more details.

Robust repulsive tilt aftereffect (TAE) in touch

Figure 3
figure 3

Experiment 1: tactile TAE results: (a) Group average of PSEs following leftward (\(-27^\circ\), red bar) and rightward (\(27^\circ\), blue bar) tilted tactile adaptor, PSEs for individual participants are shown as grey dots. Error bars denote standard errors. (b) Psychometric functions for each adaptor orientation (super-subject: bold lines, individual participants: faint lines). (c) A permutation test was used to assess the difference between PSEs under each adaptor orientation. The histogram of the t-statistics from 10,000 permutations of the PSEs is shown in blue, while the observed t-statistic is represented by the red dashed line, \(p_{perm} < .0001^{***}\).

Full size image

Experiment 1 investigates whether adaptation to somatosensory orientation will produce a TAE, similar to what is observed in the visual system. The experiment consists of two phases: an adaptation and a test phase (Fig. 2a). During the adaptation phase, participants were instructed to passively feel the orientation of a tilted tactile grating (\(-27^\circ\) or \(27^\circ\)) without making any response. After adaptation, a test stimulus was presented (a rotated version of the same tactile grating), and participants indicated whether the test grating was oriented left or right with respect to the proximal-distal direction.

A permutation test on the PSE difference between leftward and rightward adaptor conditions (Fig. 3) revealed a robust repulsive perceptual shift from the adaptor orientation (Mean \(\Delta PSE = 11.11^\circ\), \(SD = 5.70\), \(t_{obs}(13) = 7.30\), \(p_{obs} <.001\), Cohen’s \(d_{obs}= 1.83\), \(p_{perm} <.0001^{***}\)). The tactile TAE was strong and reliable and was present at the individual level for all 14 participants (Fig. 3a, grey dots and lines represent individual participants). A post hoc power analysis was carried out using G*Power, based on an observed effect size of 1.83 and a sample size of 14, yielded an estimated power of 99.99%.

Crossmodal transfer of orientation adaptation from touch to vision

Figure 4
figure 4

Experiment 2, Condition 1: tactile adaptation, visual test (TV) results: (a) Group average of PSEs in visual test following leftward (average \(-27^\circ\), red bar) and rightward (average \(27^\circ\), blue bar) tilted tactile adaptor, individual participant PSEs were shown as grey dots. (b) Psychometric functions for each adaptor orientation (super-subject: bold lines, individual participants: faint lines). (c) A permutation test was used to assess the difference between PSEs under each adaptor orientation. The histogram of the t-statistics from 10,000 permutations of the PSEs is shown in blue, while the observed t-statistic is represented by the red dashed line, Holm-Bonferroni correction was applied to the permuted p-value, \(p_{Holm;perm} =0.0006^{***}\).

Full size image

In the previous experiment, we successfully demonstrated a robust TAE in tactile orientation perception, similar to its visual counterpart. Given the close connection established by prior research in orientation perception between the two modalities, we aimed to investigate whether adaptation in one modality would induce TAE in the other modality.

Participants were tested in two conditions in Experiment 2: tactile adaptation, visual test (TV) and visual adaptation, tactile test (VT); In the TV condition, similar to Experiment 1, participants were first adapted using the titled tactile grating during the adaptation phase. In order for the participant to focus their attention on resolving the tactile orientations, which is thought to be critical in deploying the visual cortical process in tactile contexts12, participants were adapted through a series of pseudo-randomized two-interval-forced-choice (2IFC) tactile grating orientation discrimination trials with stimuli centred around the adaptor orientation (\(-27^\circ\) or \(27^\circ\)), alternating between blocks and counter-balanced between subjects (see Method section for more details). After the adaptation period (initial: 60 s, top-up: 10 s, see Fig. 2b), participants were then tested in vision using orientation-filtered pink noise stimuli (Fig. 1b). After each tactile adaptation phase, a series of six visual test trials were presented in the test phase. Psychometric functions were fitted separately for each trial (i.e., all first-trial responses, all second-trial responses, etc) for each participant. The PSEs from the first-trial data (immediately after adaptation) were used to assess the adaptation effect (as shown in Fig. 4), and the remaining five trials in the test phase were used to examine how the adaptation effect changed over time (discussed in more detail in the last part of the “Results” section).

Similar to Experiment 1, a significant difference in PSE was observed between leftward and rightward tactile adaptation conditions (Mean \(\Delta PSE =.64\),\(SD =.49\), \(t_{obs}(15) = 5.29\), \(p_{obs} <.001\), Cohen’s \(d_{obs} = 1.32\), \(p_{holm;perm} =.0006^{***}\), see Fig. 4), which clearly shows that the TAE can transfer crossmodally from somatosensation to vision. Again, the repulsive effect was robust and present at the individual level for all 16 participants (Fig. 4a). A post hoc power analysis was carried out using G*Power, based on an observed effect size of 1.32 and a sample size of 16, yielded an estimated power of 99.85%.

Asymmetry in crossmodal tilt aftereffect: visual adaptation does not bias tactile orientation perception

Figure 5
figure 5

Experiment 2, condition 2: visual adaptation, tactile test (VT) results: (a) Group average of PSEs in the tactile test following leftward (\(-27^\circ\), red bar) and rightward (\(27^\circ\), blue bar) tilted visual adaptor, individual participant PSEs were shown as grey dots. Error bars denote standard errors. (b) Psychometric functions for each adaptor orientation (super-subject: bold lines, individual participants: faint lines). (c) A permutation test was used to assess the difference between PSEs under each adaptor orientation. The histogram of the t-statistics from 10,000 permutations of the PSEs is shown in blue, while the observed t-statistic is represented by the red dashed line, Holm-Bonferroni correction was applied to the permuted p-value, \(p_{Holm;perm} = 0.20\).

Full size image

In the second condition of Experiment 2, we tested the crossmodal transfer of orientation adaptation in the other direction—from vision to touch. Participants were adapted using either a leftward tilted (\(-27^\circ\)) or rightward tilted (\(27^\circ\)) orientation filtered pink noise pattern (see Fig. 1b for an example). A dot discrimination task on the stimulus (see Fig. 2c) was employed in the adaptation phase to make sure the participant fixated on the visual adaptor. After the adaptation phase, participants were then tested in touch using the tactile grating. Each test phase consists of two trials, and the responses of the first trial were used to evaluate the crossmodal TAE here. And surprisingly, different from the previous condition, no significant difference in PSE was found between the two adaptor orientations (Mean \(\Delta PSE = -1.34^\circ\), \(SD = 4.06^\circ\), \(t_{obs}(15) = -1.32\), \(p_{obs} = 0.20\), Cohen’s \(d_{obs} = -0.33\), \(p_{holm;perm} =0.20\), see Fig. 5). A post hoc power analysis was carried out using G*Power, based on an observed effect size of -0.33 and a sample size of 16, yielded an estimated power of 23.54%.

Trial-by-trial variability in crossmodal tilt aftereffect and asymmetrical serial dependence

Figure 6
figure 6

Experiment 2 results: (a) Condition 1, tactile adaptation, visual test (TV): after each adaptation phase, a sequence of six test trials were performed, the detailed analysis for the first trial immediately after adaptation was shown in Figure 4, similar permutation test were performed for all six trials, and the permuted p-values were corrected using Holm-Bonferroni procedure. A significant repulsive TAE were found for the 1st trial (\(p_{Holm;perm} =.0006^{***}\), see Fig. 4 ), while subsequent trials yielded non-significant p-values of \(p_{Holm;perm} =.58\), \(p_{Holm;perm} = .95\), \(p_{Holm;perm} =.73\), \(p_{Holm;perm} = .07\), \(p_{Holm;perm} = .58\), respectively. Error bars denote standard errors. (b) Condition 2, visual adaptation, tactile test (VT): no significant TAE were observed for either the 1st or the 2nd trial after adaptation, \(p_{Holm;perm} =.20\), \(p_{Holm;perm} =.15\) respectively.

Full size image

In Experiment 2, the adaptation phase had the same durations for both TV and VT conditions. However, a notable difference was that the test stimulus presentation time for tactile was 1.5 s—10 times longer than that for visual test (150 ms). This difference in test duration can be attributed to the higher sensitivity of vision (mean orientation bandwidth \(\sigma_{vision} = 1.29^\circ\), obtained from the fitted cumulative Gaussian psychometric functions) compared to somatosensory perception (\(\sigma_{touch} = 7.84^\circ\)). To account for the difference in stimulus presentation time, we decided to incorporate a sequence of test trials in the test phase following each adaptation phase. This approach allows us to examine the time course of changes in TAE after adaptation more comprehensively (See Fig. 2b, c).

Similarly, psychometric functions were fitted for each of the remaining trials after adaptation, and permutation tests were performed to examine the TAE for each trial after adaptation, with p-values adjusted using the Holm-Bonferroni procedure. In condition 1 (TV), as stated earlier, a significant repulsive TAE was observed in the first visual test trial after tactile adaptation, however, this effect seems to diminish quickly after the first test trial, with no significant biases observed in the remaining trials (See Fig. 6a). In condition 2 (VT), no significant biases were observed for either the first test trial after adaptation or the second one (see Fig. 6b).

Table 1 GLMM model statistic for remaining test trials (2nd to 6th following adaptation), in Condition 1 (TV) of Experiment 2.
Full size table
Table 2 GLMM model statistic for the second test trial (following adaptation), in Condition 2 (VT) of Experiment 2.
Full size table
Figure 7
figure 7

Super-subject psychometric functions as a graphical illustration of serial dependence and TAE in Experiment 2. Psychometric functions were differentiated by adaptor orientation (colour-coded, with red indicating a leftward adaptor and blue indicating a rightward adaptor) and previous stimulus orientation (denoted by solid and dotted lines, where solid lines represent a leftward previous stimulus and dotted lines represent a rightward previous stimulus). Previous stimuli were categorized based on their direction for this plot (left vs. right), while the actual orientation in degrees was used in the generalized linear mixed model (GLMM) analysis. (a) Condition 1: tactile adaptation, visual test (VT): a repulsive serial dependence (indicated by the horizontal shift in psychometric function between solid and dotted lines, i.e., the difference in PSEs between leftward and rightward previous stimulus conditions). However, no crossmodal TAE was observed (no visible shift between red and blue lines within each previous stimulus condition, i.e., similar PSE for leftward and rightward adaptors), as confirmed by GLMM results. (b) Condition 2: visual adaptation, tactile test (VT): an attractive serial dependence was observed from the plot, but no TAE was observed, as also confirmed by GLMM results.

Full size image

In Condition 1 (TV), the absence of a TAE in the second and subsequent trials could simply be due to recovery from adaptation over time. However, it could also be a result of the adaptation effect being mediated by the percept of the previous test trial, a serial dependence37,47,48 effect might be operating in addition to the adaptation (especially given that we tested a sequence of test trials).

To evaluate the potential biases caused by serial dependence using PSEs, the trials need to be further divided based on 1-back stimulus orientation, which would lead to insufficient data points for proper psychometric function fitting, hence we instead used the Generalized Linear Mixed Effect Model (GLMM) to evaluate the potential influence of serial dependence49. In Condition 1 (TV), the GLMM was fitted with data from the 2nd to 6th visual test trials from all participants. For Condition 2 (VT), GLMM was fitted for the 2nd tactile test trial. See the Methods section for details on the GLMM.

The results indicate that both the current stimulus and previous stimulus were significant predictors for the remaining test trials in Conditions 1 and 2 [see Table 1 for Condition 1 (TV) and Table 2 for Condition 2 (VT)]. However, no significant difference in response was observed between the two adaptor orientations, for both TV and VT conditions. For graphical illustration purposes, we fitted a super-subject psychometric function for each combination of adaptor orientation and 1-back stimulus orientation direction (see Fig. 7). The significant serial dependence (difference in PSE between solid lines and dotted lines) and non-significant TAE (difference in PSE between blue and red lines) can also be clearly visualized in the figure. However, surprisingly, both the GLMM and the super-subject psychometric function have shown an asymmetry in the direction of the serial dependence between conditions, with TV showing repulsive serial dependence (\(\beta = -0.10\), \(t(7196) = -7.90\), \(p < .001^{***}\) ) and VT indicating attractive serial dependence (\(\beta = 0.01\), \(t(2876) = 5.57\), \(p < .001^{***}\)).

Discussion

In this study, we used the classic tilt aftereffect paradigm to test adaptation to orientation in the somatosensory system. Our first goal was to determine whether a TAE would occur in the touch domain (Experiment 1) and whether it would exhibit the repulsive tuning that is observed in the visual TAE. Our second goal, motivated by accumulating cortical and behavioural evidence for visual-somatosensory interactions7,8,9,10,11,12,13,14,15, was whether any adaptation to orientation would transfer crossmodally between vision and touch (Experiment 2). In Experiment 1 we successfully demonstrated a robust tactile TAE using a tactile grating which displayed the typical repulsive effect observed with the visual TAE. The tactile TAE was robust and observable reliably at the level of the individual in all participants (Fig. 3). Experiment 2 tested whether there is a crossmodal transfer of adaptation effect between the two modalities, and revealed an interesting asymmetry in which tactile adaptation to orientation induced a repulsive TAE when tested in vision, yet adaptation to visual orientation did not generate a tactile TAE. We also examined the time course of crossmodal adaptation and found the repulsive tactile-to-vision TAE diminished quickly and was no longer significant after the first trial in the visual test sequence. Finally, we examined whether there were serial dependence effects in the test sequences used in the crossmodal conditions (Experiment 2). In another curious asymmetry, serial dependence was found in both VT and TV conditions, however, it was a repulsive effect for visual test sequence and an attractive effect for tactile test sequence.

The significant tactile TAE found in Experiment 1 is in agreement with a very recent report using a different paradigm to study tactile orientation. Hidaka et al.34 presented orientation on the palm of the hand using a two-point stimulation task. The alignment of two points was sufficient to produce a percept of orientation and adaptation to this stimulus produced a robust TAE. Somewhat surprisingly, these are the first two published studies to document a purely tactile TAE (i.e., adapting and testing with tactile stimuli). While our study is focused primarily on crossmodal transfer of adaptation, Hidaka et al. made important observations relating to somatosensory processing of orientation by showing that the TAE did not depend on hand rotation and that adaptation transferred from the palm to the back of the hand (although not between hands), suggesting it is not coded in an external reference frame. Our results complement Hidaka et al.’s report by adding that orientation on the fingertip also produces a TAE that is very robust and apparent at the individual level in all participants. Importantly for our purposes, the robust tactile TAE we observed confirms the feasibility of using the same tactile stimulus settings to explore crossmodal TAEs between touch and vision in Experiment 2.

Our second experiment examined crossmodal transfer of orientation adaptation between touch and vision, testing for both vision-to-touch and touch-to-vision transfer. As shown in Fig. 4, there was a very clear effect of tactile adaptation to orientation on the fingertip transferring to vision and causing orientation repulsion on visual test stimuli. The effect was very reliable and was present at the individual level in all participants. This result is consistent with the literature reviewed in the Introduction showing behavioural evidence of common processing mechanisms for visual and tactile orientation. For example, a tactile grating combined with a visual surround grating increases the magnitude of the tilt illusion36, showing a clear orientation interaction, and visual-tactile orientation interactions have been shown to exhibit surprisingly tight orientation tunings21,22, possibly due to the increased perceptual precision that arises from integration of multisensory cues50. Despite these behavioural findings, and a host of neural findings supporting visual-tactile interactions12,13,16, the crossmodal transfer of adaptation did not occur in the reverse direction. As shown in Fig. 5, there is no evidence of transfer of visual orientation adaptation to tactile orientation perception. There is no definitive explanation for this null result (which we consistently observed during several pilot experiments testing various parameters), although we consider several possibilities below.

Potential explanation for asymmetry in crossmodal TAE and implications for crossmodal connections

As a first observation, it is clear that orientation signals from the two modalities in our study were not simply combined in the manner of Bayesian maximum likelihood estimation (MLE) integration. MLE fusion of multisensory signals does occur and is well documented in the perception of audio-visual spatial location51 and visual-tactile size52, but it does not appear to happen for visual-tactile orientation. The reason for this may relate to the very different orientation sensitivities in touch and vision, with orientation discrimination in touch being much poorer than in vision. Behavioural studies show that human orientation discrimination in vision is typically well below \(1^\circ\) for a wide range of spatial frequencies and can be as low as \(0.3^\circ\)–\(0.5^\circ\) in the most sensitive range53,54,55. This is around ten times better than human orientation discrimination in touch where the best-reported estimates are \(4.2^\circ\)–\(5.4^\circ\) for static and passively applied stimuli on the finger pad56. Our behavioural observations are consistent with these neural findings in that we found a mean bandwidth for visual orientation of \(1.29^\circ\) (average value of \(\sigma\) from the fitted psychometric functions ) and a mean bandwidth for tactile orientation of \(7.84^\circ\). These perceptual results reflect single-unit neurophysiological data recorded from primates. In the primary visual cortex, most neurons are orientation-tuned and the full bandwidth is typically in the range of \(30^\circ\)–\(40^\circ\)57. In the somatosensory cortex, neurons selective for orientation on individual finger pads are found in S1 and across multiple fingers in S26. An example of an S1 neuron with a “sharp tuning” is shown in Hsiao et al.6 and has a full bandwidth of about \(50^\circ\). In the second somatosensory cortical area (S2) the average orientation tuning bandwidth is \(63^\circ\)58. Given these marked differences in orientation precision, an MLE framework would always predict that vision would dominate visual-tactile orientation perception. Thus, in the TV condition (where we found a strong transfer of tactile orientation adaptation to vision), a Bayesian fusion model would predict little change in post-adaptation orientation as vision would be by far the higher weighted input. Thus, vision would be expected to dominate orientation perception regardless of the adapted state of tactile orientation neurons, yet against this, we found a strong TAE in the TV condition. Conversely, the MLE fusion model would predict strong a TAE in the VT condition, as the adapting stimulus was visual and thus in a vision-dominated fusion model the adapted state of visual neurons should be clearly evident, yet we did not observe a TAE.

Another consideration is that there may have been a genuine TAE in the VT condition but it was simply missed. Neural and methodological factors could both contribute to this possibility. As mentioned earlier, Konkle et al. demonstrated the shared representation of motion in vision and touch through a similar crossmodal adaptation paradigm, where the adaptation effect was transferable in both directions29. Compared to Konkle et al., one noticeable limitation in the current study is the unmatched spatial localization of the visual and tactile stimuli. Although the tactile devices were placed right in front of the screen to keep them as close as possible spatially, the grating disk was in a different plane than the visual stimulus. In contrast, Konkle et al. decided to place the tactile stimulus behind the monitor, superimposing the visual stimulus on top of it. Hence, one potential source of the asymmetry could be the different planes of stimulus in space, leading to different populations of neurons being adapted and tested. One potential improvement in future experiments would be to recreate the task in a virtual reality environment and then spatially collocate the visual and tactile stimuli.

Besides, given the markedly different orientation bandwidths, detecting evidence of orientation repulsion following visual adaptation could be challenging. Visual adaptation typically produces a visual TAE on the order of \(1^\circ\) - \(2.5^\circ\)30,59. In the tactile system, where the level of orientation precision is much lower (\(\sigma_{touch} = 7.84^\circ\)), the effect of visual adaptation might simply be too small to be reliably detected. This is also compounded by the fact that aftereffects are relatively brief and decay over time, returning continuously towards their unadapted baseline. This makes it difficult methodologically to measure the tilt aftereffect as measurements in touch generally take more time than in vision. In vision, it is easy to flash a brief test grating and quickly probe the adapted state of the orientation system but in touch, this is usually a much slower process. In our experiment, the tactile grating was presented for a duration of 1.5 s in the test condition and it is possible there may have been a transfer of orientation adaptation from vision to touch in the VT condition but it dissipated and fell below the threshold during test trials as participants felt the stimulus and arrived at their decision about its orientation. If the decay of the aftereffect was a critical factor in our failure to observe the transfer of orientation adaptation from vision to touch, it remains possible that it could be obtained under different test conditions using briefer tactile test trials. Supporting the possibility that we simply missed a genuine TAE in the VT transfer condition is the fact that this same condition has been successfully demonstrated by Krystallidou and Thompson35, although they tested tactile orientation on the forehead and used a two-point test stimulus. It remains to be seen whether the same effect would be obtained when testing as we did, on the finger pad with oriented gratings and we propose that brief tactile test stimuli would likely improve the probability of detecting the effects of visual orientation adaptation in touch.

By contrast, the transfer of orientation adaptation from touch to vision was robust. How this would arise neurally is not well established. One possibility is that tactile-visual interactions for orientation are mediated by imagined or visualized orientation when engaged in touching tactile orientation, a suggestion made by Sathian and Zangaladze12. Indeed, multiple studies have shown visual imagery is capable of inducing aftereffects through top-down recruitment of the visual cortex60,61,62. Particularly, Mohr et al63,64 showed that mental imagery alone was sufficient to produce orientation-specific adaptation in the extrastriate visual cortex (V3-V4) and elicit a consequent visual TAE. Thus if tactile orientation were to interact with vision through visualizing the felt orientation12, it would still provide a possible means for the transfer of orientation adaptation from touch to vision that we observed in the visual test condition.

Interestingly, Ganis and Schendan showed that mental imagery and perception produce opposite-signed adaptation effects in face perception60 (with the imagery effect being attractive). Mohr et al. studied TAEs with imagined visual stimuli and found a similar result: while imagined lines produced the classic direct TAE (repulsion of orientations near the adapted orientation), they found the indirect TAE (where orientations far from adaptation are tested) had the opposite sign to the standard attractive effect elicited directly from visual adaptation63. This also seems to coincide with our results in the visual-to-tactile condition, which though not significant, did point to an attractive TAE while testing in touch (Fig. 5), although further work would be needed to establish this concretely. It seems that the top-down recruitment of the visual cortex might lead to the opposite adaptation effects (attraction), but the cause of this is not clear yet, possibly it has to do with the differences in neural mechanisms between bottom-up and top-down recruitment of the primary visual cortex.

Another possibility is that the adaptation effect transfers via direct connections between primary cortices for touch and vision. This notion is appealing and sits well with recent challenges to the traditional view that the primary cortices are strictly unimodal and well segregated from each other65,66. Direct interactions between primary sensory cortices have been well documented in rodents67,68,69, however corresponding studies in primates show that direct projections between primary cortices are quite sparse70,71,72,73,74 and primates probably rely more on multisensory interactions in areas beyond primary cortices. This view squares with evidence for tactile input to the extrastriate visual cortex during tactile orientation discrimination such as Sathian et al.’s16 PET study and Zhang et al.’s15 fMRI study showing orientation discrimination elicited significant task-specific activations in the extrastriate visual cortex, and a TMS study by Zangaladze et al.13 showing the same area was critical to discriminating tactile grating orientation.

Adaptation and serial dependence acting as competing mechanisms

In Experiment 2, the GLMM analysis on the remaining test trials (excluding the initial trial) reveals contrasting intramodal serial dependence effects in the two conditions. These serial effects were present while no crossmodal TAE was found in these remaining trials by the GLMM modal. Serial dependence has been widely studied in vision (for review, see Pascucci et al.45, Cicchini et al.46 and Manassi et al.44), with orientation being one of the first visual features that have been looked at37. In condition VT, we were able to show an attractive intramodal serial dependence in the tactile orientation task, which is first up to date, and also seems to coincide with our hypothesis that the tactile orientation judgment shares a very similar processing mechanism as vision. In Condition TV, we did not observe a classical attractive serial dependence, on the contrary, we found a repulsive intramodal serial effect, which is very interesting.

Although the serial dependence effect has been widely reported in a wide variety of visual perceptual tasks, ranging from low-level features such as orientation37 and numerosity47 to high-level features such as faces75, aesthetic perception76, and attractiveness77, there are still heated debates regarding the stage of the perceptual process from which it originates and its direction. On the one hand, many believe that the serial dependence effect is an attractive perceptual bias (as opposed to a repulsive neuronal adaptation effect), without the need for a previous response41 or decision42,43, and is tuned for low-level properties such as location37 and feature similarity47. On the other hand, there is also another line of arguments suggesting that the observed attractive serial bias cannot be a low-level process78 and originates from post-perceptual processing, modulated by memory38, decision39, and confidence40. On top of these, various studies also reported a mixture of attractive and repulsive effects38,39,79,80,81,82,83. One hypothesis that attempts to unite these mixed results is that the observed serial bias originates from two competing mechanisms: a lower-level repulsive sensory adaptation effect presented right after the previous percept, and an attractive serial dependence that manifests throughout higher-order post-perceptual processing. The observed direction of the perceptual biases is considered to be a combination of these two competing mechanisms38,45. Previous studies also indicated that the strength of attractive serial dependence is dependent on the reliability of the current percept, in particular, the attractive effect is greater when the current precept lacks reliability (hence more weight is put on the recent perceptual history to optimize the current percept47). And in our experiment, orientation perception is much more reliable in vision (\(\sigma_{vision} = 1.29^\circ\)) compared to touch (\(\sigma_{touch} = 7.84^\circ\)). In condition TV, with the more reliable visual orientation judgment, the weight on the previous percept should be lower, which should lead to a weaker attractive serial effect. Besides, the immediate response cue after the brief presentation of visual test stimuli (150 ms) might be too early for the attractive serial dependence to manifest in the higher-order post-perceptual processing, which together could lead to the observed classical sensory-driven repulsive effect. It would also be able to explain the quick diminishing of crossmodal TAE in Condition TV: the visual neurons that were crossmodally adapted by tactile adaptor could be re-adapted intramodally by the visual test stimuli, and hence diminishing the touch-to-vision crossmodal TAE in the subsequent test trials.

In condition VT, tactile orientation judgment would potentially rely more on the previous percept due to the lack of reliability, and the much longer presentation time (1.5 s) would provide more time for higher-order post-perceptual processing, hence showing a stronger attractive intramodal serial effect at the time of cued response. The absence of vision-to-touch crossmodal TAE in this condition can also be attributed to this lack of reliability in tactile orientation judgment, as the effect of the weak repulsive crossmodal sensory adaptation being overpowered by the stronger attractive influence of perceptual history due to high uncertainty in perception, and fits with our suggestion that the hint of an attractive effect in our vision-to-touch TAE condition (Fig. 5), and provides a good theoretical framework to motivate a reinvestigation of this condition in future studies to clarify whether there is a genuine attractive effect or not.

Divisive normalization between modalities leads to a combination of repulsive and attractive effects

Figure 5 shows a mix of attractive and repulsive biases among participants for condition VT. These effects might simply reflect noise in a genuine null condition, but they might also indicate a genuine mixture of adaptation-induced attractive and repulsive biases. Some studies in vision have reported attractive adaptation shifts with certain paradigms84,85, although repulsive effects are more common. One flexible model used to explain adaptation effects is the divisive normalization model and it can explain the difference in the direction of adaptation-induced bias across many different sensory adaptation paradigms85,86,87. At the neuronal level, divisive normalization is proposed to work by taking a given neuron’s activation from within its own receptive field (Classical Receptive Field—CRF) and dividing it (i.e., normalising it) by the sum of activation from neurons arranged in a larger suppressive surround field (normalization pool). The net effect is that stimulation in the normalization field down-regulates the neuron’s response. Phenomena such as surround suppression observed in V1 can be explained by divisive normalization as extending the stimulus extent into the surround field reduces a neuron’s response86. Divisive normalization has been used to model many perceptual and neurophysiological effects and is also thought to play a role in multisensory integration88. Hence it may also play a role in how orientation information is transferred from somatosensory to the visual cortex. We tried to simulate a divisive normalization model based on these principles where an orientation-selective visual neuron has a normalization field with a broader orientation range compared to the CRF. We were able to show that the sign of the adaptation effect—whether attractive or repulsive—could vary based on the relative strength of the adaptation drive to the CRF and normalization field (See Fig. 8). It has been shown that during tactile processing there are both a top-down suppressive pathway through the visual hierarchy and an ascending excitatory pathway that directly influences V120. This supports the possibility of a normalization arrangement between these two types of signals as in this study, and as found more generally, top–down projections tend to be diffuse and provide surround suppression89,90. Additionally, Sathian has shown the significant variability in connection strength between visual and somatosensory cortical areas across individuals based on their visual history, potentially attributed to crossmodal plasticity91. Hence, the mixed attractive and repulsive adaptation effect observed in condition VT could potentially be attributed to experience-related differences in the relative strength of suppressive and excitatory signals among participants. However, the model and simulation here were only for illustrative purposes to demonstrate the possible explanation of the observed behaviour, future research with actual neuronal recording might help confirm this hypothesis.

Furthermore, despite the common perception of normalization as a static computation in our model, recent studies have demonstrated that a dynamic normalization model with temporally changing normalization weights can better capture the dynamic nature of transient behaviour in visual neurons92,93. This insight could potentially contribute to explaining the mixed results observed in our experiment. Specifically, the temporal contingency of adapting stimuli has been shown to influence whether the normalization signal weakens or strengthens94. The adaptation paradigm in our experiment, involving multiple testing, differs in temporal contingencies between conditions (varying in the number of test trials and test trial durations between conditions VT and TV). These differences might be the source of the asymmetrical TAE and serial dependence effects observed between the touch and vision modalities. However, further investigation is required to comprehensively understand these dynamics.

Figure 8
figure 8

Illustration of the Divisive normalization model.(a) CRF and normalization pool prior to adaptation. (b,c) Stronger adaptation to the CRF (d,e) stronger adaptation to the normalization signal (f) Stronger adaptation to the CRF lead to classic repulsive TAE with suppression effect on the population response, while stronger drive to the normalization signal lead to attractive adaptation and facilitating effect in the population response, which could potentially explain the asymmetrical crossmodal TAE results in our experiment.

Full size image

Conclusion

In the current study, we found that adaptation to a tactile grating presented to the finger pads produced a robust, repulsive TAE, analogous to what is observed in vision after adaptation to a visual grating. We also explored the transfer of orientation adaptation between vision and touch. Adaptation to tactile orientation produced a very robust TAE when tested in vision that was reliable at the level of individual participants. This result is consistent with mounting neural evidence of common processing of visual and tactile orientation in the extrastriate cortex. However, in an asymmetrical result, we found no evidence of transfer of orientation adaptation in the other direction, from vision to touch. We propose that the failure to find an effect of visual orientation adaptation when test in touch is inconclusive and should be interpreted with caution. It is inconsistent with a behavioural report of vision-to-touch transfer of orientation adaptation (albeit for test stimuli presented on the forehead rather than the finger pad). We conjecture that our rather long tactile stimulus presentation time may have prevented effective measurement of transferred orientation adaptation and recommend future studies employ brief test trials to better capture the ephemeral and decaying nature of orientation adaptation. It is also possible that the poor precision of tactile orientation perception relative to vision adds to the difficulty of capturing vision-to-touch transfer of orientation adaptation. We also looked at the intramodal serial dependence effect within the test sequence and found a similar asymmetrical serial dependence effect. Which could be a competing mechanism that mediates the crossmodal TAE, and provides another alternative explanation for the asymmetrical crossmodal transfer of TAE. Finally, we propose that a divisive normalization model could explain our inconclusive findings in the case of vision-to-touch transfer as both attractive and repulsive shifts can be induced by the adaptor depending on the nature of the surround field. Thus our null result might reflect roughly equivalent numbers of subjects showing a repulsive or an attractive effect.

Despite these caveats, we have demonstrated two key findings: (i) for the first time we have shown a purely tactile TAE on the finger pads, and (ii) we show clear evidence of crossmodal transfer of adaptation from touch to vision, complementing an earlier report of transfer from vision to touch. The first result complements neural work in primates showing orientation selectivity in S1 neurons and indicates that neural interactions among S1 cells with adjacent orientation preferences interact similarly to those observed in the visual cortex. The second result adds to mounting neural and behavioural evidence of common processing of visual and tactile orientation.

Methods

Participants

Participants were all young adults (first/second-year psychology students from the Univesity of Sydney) recruited through the Sydney University psychology SONApsych research participation system, and were given course credit for their participation. Informed consent was obtained from all participants prior to the commencement of the experiment. 14 naïve participants participated in Experiment 1, 12 females and two males. 22 naïve participants participated in Experiment 2, 16 females and six males. All participants reported being right-hand dominant with no recent history of damage to the right index finger and no history of damage or diseases of the nervous system. Participants also reported having a normal or corrected-to-normal vision, which is collected as part of the SONA pre-screening process. This research was approved by the University of Sydney Human Research Ethics Committee (HREC 2021/048), and all methods were carried out in accordance with relevant guidelines and regulations.

Apparatus & stimulus

The tactile stimulus used in all experiments was a 3D printed (Ultimaker 3D printer) disk with a grating pattern on its upper surface whose orientation could be rotated with an Arduino Uno microcontroller. The model of the grating disk was designed using FreeCAD. It has a diameter of 30 mm and is composed of a triangular wave with a 6 mm spatial period and a peak-to-trough amplitude of 2.5 mm (See Fig. 1c). The microcontroller is controlled by an Arduino Uno Board via MATLAB and can rotate the disk with a resolution of \(1.8^\circ\)/step. The disk was housed in a 3D printed box on which the subject rested their hand. The disk was presented to the subject through an aperture on top of the box and the presentation period was controlled by raising the disk into position for a fixed period and then lowering it back inside. In Experiment 1, three sets of tactile stimulus orientations were employed (refer to Fig. 1d). This approach was adopted to account for the substantial variability in orientation acuity among participants, and the selection of the specific set was based on each participant’s performance in the preliminary test. In Experiment 2, only the largest set was used to simplify the process.

Visual stimuli used in experiment 2 were generated using a MATLAB image processing toolbox95, and presented using Psychophysics Toolbox extensions96 on a Dell OptiPlex 7440 Display (\(1920\times 1080\) pixels, \({60}\,Hz\) refresh rate). Visual adaptation and test stimuli in these experiments (see Fig. 1b) were orientation-filtered noise patterns (pink noise, filtered with a Gaussian orientation filter with bandwidth (\(\sigma = 5^\circ\)), with a 2D Gaussian kernel applied to the noise pattern to generate a smooth roll-off at the edge. In experiment 2, condition VT, the root-mean-squared (RMS) contrast for the visual adaptor was 0.3, oriented at \(-27^\circ\) and \(27^\circ\). In experiment 2, condition TV, the RMS contrast for the test stimuli was 0.018, with six different orientations around vertical (\(-1.8^\circ\) to \(1.8^\circ\), with \(0.4^\circ\) gap, excluding \(0^\circ\)).

Data analysis

Exclusion criteria

As the bias estimation depends on reliable psychometric function fitting for each participant under all conditions, it is crucial for participants to perform within a reasonable percentage correctness range. Hence, we excluded participants with a low percentage of correct performance (<65%). In Experiment 1, the criterion was applied to the overall performance of the participant since only one condition was presented in the experiment, all 14 participants passed this assessment and were included in the analysis. For Experiment 2, the criterion was applied for each condition separately, and six out of the 22 participants were excluded, as their performance in at least one of the conditions was below the 65% cutoff.

Psychometric function fitting

Psychometric functions were fitted for each participant and each adaptor orientation separately using the Bayesian hierarchical model from Palamedes toolbox97:

$$\begin{aligned} \Psi (x;\mu ,\sigma ,\gamma ,\lambda ) = \gamma +(1-\gamma -\lambda )F(x;\mu ,\sigma ) \end{aligned}$$
(1)

The function \(F(x;\mu ,\sigma )\) is a cumulative Gaussian function where \(\mu =\) mean and \(\sigma =\) standard deviation. And \(\gamma\) & \(\lambda\) represent the guess rate and the lapse rate. These parameters correspond to the disparity between the asymptotes of the psychometric function and the ideal performance levels of 0% and 100%, respectively. Point of subjective equality (PSE) was obtained as the mean of the cumulative Gaussian function (\(\mu\), 50% rightward response), which was used as a measure for bias in orientation judgement (positive: leftward bias, negative: rightward bias).

Permutation test

We used a non-parametric permutation test to evaluate the biases in the PSEs. The observed statistics (\(t_{obs}\)) were obtained through paired-sample t-tests comparing the PSEs of the actual participants’ data under different adaptor orientations. Then, 10,000 permutations of the empirical PSEs were generated by randomly swapping the PSEs under different adaptor conditions. The p-value of the permutation test is calculated as the proportion of t statistics from the permuted distribution that is more extreme than the observed statistics(\(|t_{permutation}| \ge |t_{obs}|\)).

For each condition, as multiple trials were tested after the adaptation, and similar permutation tests were performed for each test trial, we used the Holm-Bonferroni method98 to correct for family-wise error rate. The adjusted p-value is calculated by sorting the permuted p-value in each condition in ascending order, and multiplying the p-value with the number of tests (C) in the condition minus the rank (i) and plus one, the formula is as follows:

$$\begin{aligned} p_{holm} = (C - i +1)p_{perm} \end{aligned}$$
(2)

where an adjusted p-value larger than one is set to one.

Generalized linear mixed effects model (GLMM)

The serial dependence effects were analysed using GLMM, as further dividing each participant’s data based on previous stimulus in addition to adaptor orientations would result in insufficient data for proper psychometric function fitting, and lead to unreliable PSE estimates.

The model was fitted with the raw trial-by-trial data, following the principle of Moscatelli et al.49 The first test trial after each adaptation phase was excluded in this analysis as there is no 1-back test trial (the previous test trial was prior to the preceding adaptation phase, which is separated with the test trial by 60 s/10 s for initial and top-up adaptation, and would not be appropriate for serial dependence analysis).

We employed MATLAB’s Statistics and Machine Learning Toolbox to fit the GLMM. The model was fitted using a binomial distribution and probit link function, applying the maximum likelihood method with Laplace approximation. We included the three fixed effects, to account for the previous and current stimulus, as well as the adaptor orientation. We also included the subject as a random effect to account for the differences in sensitivity and overall internal biases for each participant. The GLMM model is presented in Wilkinson notation below:

$$\begin{aligned} Response \sim 1 + Current\, Stimulus+ Adaptor\, Orientation + Previous\, Stimulus + (Current\, Stimulus | Subject) \end{aligned}$$
(3)

Notice the previous stimulus orientations used in the actual GLMM were in degrees. However, for clarity, in the graphical illustration (Fig. 7) the previous stimulus was simplified to left or right.

Design & procedure

Experiment 1: tactile tilt aftereffect

Participants sat in front of a screen with their right hand resting on top of a box containing the tactile device. A schematic plot is shown in Fig. 1a. Each session began with a preliminary test lasting approximately five minutes to determine for each participant which of the three orientation ranges was best suited to their tactile orientation acuity to ensure a better-fitting psychometric function (Fig. 1d). This also acted as the practice block to familiarize the participant with the task. The main test contains six blocks, with adaptor orientations (left vs. right) alternating between blocks, and the order counterbalanced between subjects. Each trial began with the tactile grating rising from inside the box to the level of the participant’s fingertip already rotated to the adaptor orientation used for that block. Participants passively felt the orientation for the duration of the adaptation phase (30 s for initial adaptation, 5 s top-ups thereafter). After the adaptation phase, and a gap of 500 ms, the test stimulus was then presented to the participant through the tactile device for 1.5 s, and the participant was cued to passively feel the orientation of the test stimuli with their fingertip, and then report whether the stimulus orientation was left (anticlockwise) or right (clockwise) compared to vertical (proximal-distal orientation) by pressing the arrow keys with the other hand (See Fig.2a).

Experiment 2: visuotactile crossmodal tilt aftereffect

Experiment 2 consisted of two consecutive sessions, each testing a different condition: tactile adaptation, visual test (TV) and visual adaptation, tactile test (VT). The order of the two conditions was counterbalanced between participants. Both conditions consisted of an adaptation phase followed by a test phase.

In Condition TV, in order for the participant to focus their attention on resolving the tactile orientations during the adaptation phase, which is thought to be critical in deploying the visual cortical process in tactile contexts12. A 2IFC orientation discrimination task was introduced as the adaptation task: two tactile orientations were presented in succession, with each presentation lasting five seconds. Participants responded on the keyboard whether the second orientation was more towards the left or right of the first. All adaptation stimuli were pseudo-randomly drawn from four orientations (\(18^\circ , 23.4^\circ , 30.6^\circ , 36^\circ\), left or right based on the block). At the beginning of each block, a sequence of six adaptation trials was performed, adding up to 60 seconds of initial adaptation. After each visual test sequence, another top-up adaptation trial was conducted, which accounted for 10 seconds of top-up adaption (see Fig. 2b) For each block, the average of the pseudo-randomised adaptation trial stimulus orientation was \(-27^\circ\) or \(27^\circ\), alternating between blocks and counterbalanced between subjects.

In the VT condition, the visual adaptation phase contained a dot location task designed to keep the participant’s attention on the adaptation stimulus. The dot appeared randomly in an upper or lower location multiple times during the adaptation period and the participant responded to the dot location with the keyboard every time they saw the dot, as shown in Fig. 2c.

In both conditions, a test trial was presented immediately after the adaptation phase to test for the TAE. However, to accommodate the different nature of visual and tactile perception and the longer time needed to determine the tactile orientation, the test stimuli were presented for different durations in each condition (visual test: 150 ms, tactile test: 1.5 s). To evaluate the potential confound introduced by different time courses of testing, additional test trials were given after the first trial to test the change in TAE after the initial test. In Condition TV, five addition test trials were conducted (six test trials in total, See Fig. 2b), and in Condition VT, one addition test trial was conducted (two test trials in total, See Fig. 2c).

Divisive normalization model

To try and explain the mixed results in the visual adaptation, tactile test condition (VT), we used a simple divisive normalization model to simulate the population response of neurons in the visual cortex. This simplified illustrative model was inspired by some published data85,86,87.

The CRF (summation field) and normalization field (suppression field) tuning of each of the orientation-selective neurons is simulated with Von Mises functions (see Fig. 8a):

$$\begin{aligned} r(\theta ) = c + a*e^{b *[cos(\theta - \theta _{pref})-1]} \end{aligned}$$
(4)

where c is an offset representing the base response, a is the peak height of the summation field/normalization field, and b is a parameter that controls the bandwidth of the summation field/normalization field. \(\theta _{pref}\) stands for the preferred orientation of the neuron.

The adaptation effect \(a(\theta _{pref})\) to a neuron with preferred orientation \(\theta _{pref}\) was simulated with the complement of the Von Mises function, with different strength (k) applied to the summation field and normalization field, and the strongest drive were observed at the adaptor orientation \(\theta _{adaptation}\) (Fig. 8b, d). The adaptation effects were then multiplied with the summation field and suppression field to obtain the tuning after adaptation (Fig. 8c, e):

$$\begin{aligned} a(\theta ) = 1 - k*e^{b_{adapt} *[cos(\theta _{pref} - \theta _{adaptation})-1]} \end{aligned}$$
(5)

The normalized response \(r_{j}\) of the neuron j was calculated by dividing the CRF response (weighted sum of drive to the summation field \(\sum _{k}\omega _{ik}I_{k}\), where \(I_{k}\) was the non-normalized response of the neuron k in the summation field), divided by the normalization signal (weighted sum of drive to the normalization field \(\sum _{k}\alpha _{jk}I_k^m\)). Constant \(\sigma\) was added to the denominator to avoid division by zero. n & p are exponents to account for non-linearity:

$$\begin{aligned} r_{j}= \gamma \frac{(\sum _{k}\omega _{ik}I_{k})^n}{\sigma ^n + (\sum _{k}\alpha _{jk}I_k^m)^p} \end{aligned}$$
(6)

The population response was obtained by fitting a Von Mises function to the normalized responses of all the simulated neurons with different preferred orientations. The centre of the Von Mises function, representing the peak response, was used as the simulated perceived orientation (see Fig. 8f).