Virtual reality (VR) provides a valuable research tool for studying what occurs when sensorimotor feedback loops are manipulated. Here we measured whether exposure to a novel temporal relationship between action and sensory reaction in VR causes recalibration of time perception. We asked 31 participants to perform time perception tasks where the interval of a moving probe was reproduced using continuous or discrete motor methods. These time perception tasks were completed pre- and post-exposure to dynamic VR content in a block-counterbalanced order. One group of participants experienced a standard VR task (“normal-time”), while another group had their real-world movements coupled to the flow of time in the virtual space (“movement contingent time-flow; MCTF”). We expected this novel action-perception relationship to affect continuous motor time perception performance, but not discrete motor time perception. The results indicated duration-dependent recalibration specific to a motor task involving continuous movement such that the probe intervals were under-estimated by approximately 15% following exposure to VR with the MCTF manipulation. Control tasks in VR and non-VR settings produced similar results to those of the normal-time VR group, confirming the specificity of the MCTF manipulation. The findings provide valuable insights into the potential impact of VR on sensorimotor recalibration. Understanding this process will be valuable for the development and implementation of rehabilitation practices.
The ability to estimate the passage of time with precision is fundamental to our ability to perceive and interact with the world. One key characteristic of time perception is that it is highly plastic, supporting the ability to adapt to changing environmental conditions1,2,3,4. Although the plasticity of time perception has been well established, several open questions remain regarding the strength, persistence, and specificity of temporal recalibration effects. For instance, do these recalibration effects generalize across different modes of duration estimation (e.g., continuous and discrete motor reproduction)? Answering such questions will improve the understanding of the computational and neural basis for time perception, which has gained significant interest of late partially due to the appeal of preventing or reversing maladaptive changes to time perception with age5,6,7,8.
Computational Perspectives of Time Perception
Time perception is a multifaceted construct, and several theoretical perspectives have been proposed in the temporal processing literature (for review, see9). The dominant model of temporal processing involves a central internal clock10,11,12,13. In this model, time perception is represented by the subjective count of pulses that accumulate within a given interval. There is evidence14 that these mechanisms, theorised to underlie the conscious perception of time, also drive the timing of motor performance (also see15). Research suggests that the neural pacing signal of the internal clock system, rather than being static, can be modulated by sensory inputs from external stimuli. A supra-modal system contrasts with the idea of dedicated mechanisms for each sense, although it is possible that a central mechanism subserves supra-second intervals, while sub-second intervals are processed by modality-specific systems12,16. Others have proposed models that do not require pulse intervals for timekeeping, suggesting instead that time is encoded in spatial17,18 or temporal19 patterns of neuronal firing. These state-dependent network models have gained recognition on the basis of support from psychophysical studies of temporal processing and learning mechanisms20,21,22. While each model has proven useful, there is a lack of compelling in vivo physiological evidence for the existence of a pacing signal or state-dependent representations of timing23.
Neurophysiology of Time Perception
Describing the processes underlying time perception at the neural level has been a major challenge in the field. Although still a matter of debate, the brain structures thought to be involved in time perception include the cerebellum, prefrontal cortex, basal ganglia, and supplementary motor area16,24,25. Recent transcranial magnetic stimulation (TMS) evidence has supported the role of the cerebellum and prefrontal cortex in time perception. TMS used to disrupt cerebellar activity impairs timing of sub-second durations26,27,28, whereas inhibitory TMS over prefrontal cortex impairs timing of supra-second durations27,29. It has been proposed that the prefrontal cortex operates in a feedback role by using the sensory information following an action to update temporal expectations30, whereas the cerebellum plays a feed-forward role in making temporal predictions prior to an action16. Basal ganglia activity is associated with the encoding of temporal processing and representation of stimulus duration, which is demonstrated by both the behavioural data of Parkinson’s patients with basal ganglia dysfunction31 and by functional magnetic resonance imaging (fMRI) studies24,32,33,34.
These neuroanatomical studies in conjunction with non-human pharmacological studies have supported the idea that timekeeping is modulated by dopamine neurotransmission, specifically at the D2 receptor16,35,36. Dopamine antagonists (e.g., neuroleptics) decrease subjective estimations of event duration37, whereas dopamine agonists (e.g., methamphetamine) lead to an increase in duration estimations38,39. Recent evidence also suggests a prominent role for GABA in time perception; magnetic resonance imaging of the rat cortex shows that elevated GABA levels correspond to underestimation in the perceived duration of sub-second intervals, possibly due to diminished awareness of visual stimuli37,40.
Evidence of Temporal Recalibration
Neurophysiology studies not only provide insight into the neural mechanism of the internal clock, but they also support the idea that time perception can be manipulated. The speed of this internal clock can be increased or decreased depending on the drug administered, which can lead to behavioural changes38,41. Another phenomenon that has been shown to speed up the internal clock is the click train effect, whereby listening to a train of clicks (e.g., 5 sec of clicks at 5 clicks/sec) induces a 10% decrease in the perceived duration of subsequent intervals2,14,42. In line with pharmacological studies, the click train is thought to speed up the internal clock by increasing arousal levels acting on the calibration unit. The stimulus preceded by the train of clicks is therefore perceived as shorter than the one preceded by silence. These studies provide evidence that temporal recalibration can occur, at least at the sub-second scale. Additionally, evidence shows that motor reproduction of interval timing is similarly affected by click trains, suggesting that a common temporal oscillator may underlie both conscious time perception and motor performance14,15.
Changes in temporal processing have been reported following repeated exposure to temporal misalignments in multimodal cues, in the form of a shift in the estimated simultaneity of post-training stimuli4,43,44,45. Temporal recalibration effects in response to novel temporal correlations between motor performance and sensory feedback have also been described. Latency between action and visual feedback leads to predictable and persistent behavioral adaptation aftereffects3,46,47,48. Rohde and colleagues3 observed perceptual learning effects when a lag was introduced between hand and cursor movement in a manual tracking task. Adaptation to visuo-motor latency revealed a decrease in motor error with time, and large aftereffects in motor timing following adaptation. As in previous studies47,48, the recalibration effect was found to generalize between motor timing and perceptual measures (e.g., simultaneity judgments).
Perceptual learning as a Framework for Temporal Recalibration
Most studies that have examined the plasticity of time perception have adopted a perceptual learning approach. Theories such as sensorimotor contingency theory49,50 highlight the role of the action-perception loop in achieving perceptual learning. This theory describes sensorimotor contingencies as the role our actions play on the sensory inputs we receive: perceptual qualia emerge from learned relationships between action and the incoming sensory data produced as a consequence of the action. These sensorimotor contingencies are implicitly learned over time and shape perception51. Bompas and O’Regan52 provide an elegant example of how sensorimotor contingencies are learned by artificially coupling eye movement and color changes. By exposing participants to specific colors upon saccades to the right or left over an extended duration, they produced a predictable and persistent change in the perceived color of a neutral patch that was contingent on saccade direction.
If the qualia of perception are determined by environmental interaction, it is conceivable that even high-level perceptual qualia such as time can be altered by inducing a novel sensorimotor contingency
Applications of Plasticity in Time Perception
Manipulations of time perception are of practical interest. Several neurological conditions are associated with deficits in the perception of timing, including ADHD53, autism54, and schizophrenia55, and these deficits can be reduced through timekeeping training56,57 (for review, see58). This decrement in time perception has also been seen in the normal aging process6,8. Due to the appeal of preventing or reversing these maladaptive changes, the utility of interventions that influence time perception has been widely sought after. Although several studies have investigated the plasticity of time perception in the framework of perceptual learning, studies have not examined whether exposure to a novel relationship between action (e.g., moving the body) and sensory feedback (e.g., the speed and duration of environmental events) can produce changes in time perception at the supra-second time-scale.
The objective of this study was to investigate if temporal recalibration is induced by exposure to a novel sensorimotor contingency between movement and the speed of events. This question can only be investigated using a naturalistic setting that affords re-learning of normal action-perception loops. In order to provide a naturalistic setting in a controlled environment, we chose to administer the movement-contingent task using virtual reality (VR) technology. VR, which uses sensory stimulation devices to simulate an interactive environment, has the unique ability to dissociate the natural link between perception and action59.
We attempted to induce a novel sensorimotor contingency by coupling the speed and duration of visual events to the bodily movement of the participant. We term this ‘movement-contingent time-flow’ (MCTF). In this manipulation, if the participant moved their hands or head, the speed of events in their surrounding virtual environment was normal. However, if the participant stopped moving, the speed of events slowed down. With exposure to this manipulation, we expected participants to adapt their perception of time such that when they were static, the probe durations were perceived as longer. To test if temporal recalibration had occurred, we conducted pre- and post-exposure time perception tasks. The psychophysical tasks required participants to observe a probe moving in a circle and then reproduce the duration, speed, and trajectory of the probe. We assessed both continuous motor and discrete motor time perception tasks. We also measured the effects of VR without the MCTF mechanic, to assess if exposure to VR alone affected time perception. As an additional control, we conducted a non-VR control task for a subset of participants in order to determine whether physical activity alone results in temporal recalibration.
The present study tests the hypothesis that adaptation to a novel relationship between action and the perceived timing of events in virtual reality results in recalibration of time perception.
We predicted time would be perceived as slower when participants were static following exposure to VR with the MCTF manipulation, due to the novel relationship between movement and event speed acquired during the task. As such, we predicted different pattern of data across the continuous motor and discrete motor tasks. For the continuous motor task, participants were static when observing the probe, and were moving when reproducing the probe. Perceived durations were predicted to be longer when the probe was observed (no movement), and shorter when the probe was reproduced (movement). This difference was not predicted for the discrete motor task, where the ‘observe’ and ‘reproduce’ phases of the trials were both static (see Table 1).
We expected to find no difference between pre- and post-adaptation estimates for participants who were exposed to VR with no MCTF manipulation, or by participants who simply performed a dynamic motor coordination task (a ball-toss) instead of the VR task.
Thirty-four students from the University of Waterloo participated in the study (the final dataset included data from 31 participants due to our exclusion criteria, as described in Results; 18 females, 13 males; age in years M = 21.1, SD = 2.4). All participants reported having normal or corrected-to-normal vision and reported no sensory, musculoskeletal, or neurological disorders. Protocols were approved by the University of Waterloo Research Ethics Committee and were carried out in accordance with the Declaration of Helsinki. All participants gave informed written consent, but all were naïve to the hypotheses of the research. Participants either volunteered their time or were remunerated $10 per hour for their participation. With respect to participants whose data comprised our final dataset, prior experience with VR was generally low – nine participants had used VR before, but all of those had only used it on one occasion. In addition, with respect to average experience with video games, the final dataset consisted of the following distribution of participants: 1 reported playing ≥15 hours of video games/week, 3 reported playing 5–14 hours per week, 9 reported playing video games for ≤5 hours/week, and 18 participants reported that they typically did not play video games.
The equipment used for the time perception tasks was a laptop (R590, Samsung, 1366 × 768 resolution, 60 Hz refresh rate). The laptop was positioned at the participant’s eye level and the approximate distance from the screen to their eyes was 60 cm (visual angle was ~18 × 32°), although a chin rest was not used. The time perception task was run using Matlab R2016a with the Psychophysics toolbox60. The mouse used in this task was a USB mouse (Apple Inc.), which was positioned on a mouse mat with a gel wrist rest.
The VR environment was presented via a head mounted display (Rift CV1, Oculus VR; 90 Hz refresh rate, 1080 × 1200 resolution per eye) with hand-held controllers (Touch, Oculus VR). The headset and controllers were motion tracked by a combination of the inertial (accelerometer/gyroscope) and optical (3 x infrared Oculus cameras) sensors that were included with the system. Movement of the head was translated into motion of the observer viewpoint in the VR task. The system ran on a custom built workstation with a high-end graphics board (GTX 1070, NVIDIA). The software packaged with the head mounted display was used to calibrate the capture space (2.41 × 2.41 m) and the inter-pupillary distance of the headset for each participant.
The equipment used for the non-VR control task included 20 foam-rubber balls, three buckets, and a pair of tight fitting goggles. The foam-rubber balls were 5.08 cm in diameter and consisted of both white and colored balls. The target buckets were 21.59 cm wide and 21.59 cm tall. The goggles, which were used in order to replicate the narrow visual field of the head-mounted display, produced a field-of-view of approximately 90 × 50° (horizontal x vertical).
Time perception tasks
In the time perception tasks, participants were first presented with a fixation cross for 500 ms. Participants were instructed to fixate on the cross throughout the task. Participants were presented with a blue circular probe located at approximately 3.5° eccentricity from the central fixation cross. The probe then rotated around the fixation cross for a given duration and velocity before disappearing. Next, participants were prompted to reproduce the timing of the probe using one of two methods. The first method required continuous motor reproduction of the probe’s spatiotemporal trajectory by moving a white circle using a mouse (the continuous motor task), and the second method required a button press to signal the start and end of the perceived probe duration (the discrete motor task). All responses were recorded at the moment of button-press (i.e., not on button-release). The two tasks are described in detail below (also see Fig. 1).
Continuous motor reproduction task: To reproduce the speed and duration of the probe movement, participants were instructed to move their hand in a circle while holding a mouse. The white circle that appeared after the probe disappeared moved in a one-to-one manner relative to the hand movement of the observer. Participants were required to press the mouse button to indicate duration onset and offset. Supplementary Movie 1 depicts four example trials from this condition.
Discrete motor reproduction task: To reproduce the duration of the probe movement, participants first pressed the mouse button indicating duration onset, then waited until the perceived duration had elapsed, and finally pressed the button again to indicate duration offset.
Virtual reality tasks
The VR content used in the experiment was an off-the-shelf consumer game (Robo Recall, Epic Games; Fig. 2). This was a first-person action game wherein the user took the role of a robot whose task was to destroy other robots that had been let loose in a realistic city environment. Participants were instructed to play the game by shooting the target robots until the time limit was reached. Points towards the participants’ scores were received upon performing one of several actions, such as destroying an enemy, avoiding enemy projectiles, and chaining together multiple enemy takedowns. All points were computed by the in-game scoring system and the experimenter recorded the total score after termination of the VR block.
Depending on their group assignment, participants experienced one of two versions of this game. The manipulated factor between these groups was whether the speed of events in the game was either normal (VR control, n = 13 following exclusions: see results) or modified (VR MCTF, n = 12 following exclusions: see results) as a function of the movement of the player’s hands (tracked by the Touch Controllers) and head (tracked by the Oculus Rift headset). In order to achieve the latter, we implemented a user-made modification of the game (MGS Studios, Berkshire, UK). In this modification, the movement of the participants was coupled to the speed of events occurring in their surroundings. If the head and hands were stationary, event speed was decreased by a factor of 8 compared to control conditions (e.g., a ball that would normally hit the ground after 1 s of falling would instead take 8 s to fall). Conversely, movements of 100 cm/s or greater were associated with normal event speed (i.e., no decrease in the speed of events). Participant movement speeds between 0 cm/s and 100 cm/s were linearly and inversely related to event speed; for example, 70 cm/s movement caused event speed to decrease by a factor of 2.4 (30% of 8), and 30 cm/s movement reduced event speed by a factor of 5.6 (70% of 8). The program did not combine the speed of multiple sensors (e.g., moving both hands at 40 cm/s resulted only in a factor of 4 decrease in event speed). In the VR control condition, participants were exposed to the same experience, but without the movement-time coupling.
Non-virtual reality control task
In addition to the two VR groups (MCTF and control), we conducted a non-VR control task with a subset of participants (n = 6) in order to decouple the effect of VR exposure from the possible effects of physical activity on time perception, as durations tend to be overestimated following physical activity61,62.
The objective of the task was to throw foam balls into the correct color of bucket in order to accrue points (Fig. 3). The experimenter began the task by throwing the balls one at a time to the participant, who was instructed to toss the white balls into the white bucket, and the colored balls into the green bucket (correctly doing so gained the participant 1 point). Participants were also informed that they could throw any of the balls into the red bucket, thus earning 4 points. Participants were instructed to accrue the highest possible points tally. Once all of the balls in the bag were tossed, participants walked around the area to collect the balls that did not reach the buckets, while the experimenter counted the ones that did. Once the balls were returned to the experimenter, the next round of throws commenced. This process continued until 10 minutes had elapsed.
Design and procedure
Participants first completed two time perception tasks (continuous motor and discrete motor) to establish pre-adaptation measures (details below). Next, participants were exposed to the VR task and a post-adaptation time perception task (e.g., continuous motor). Finally, participants were exposed to the VR task a second time and then completed another post-adaptation time perception task (e.g., discrete motor). Participants each had a predetermined group assignment and order of time perception tasks that were counterbalanced across participants.
Each time perception task contained 15 unique trials, which consisted of 3 angular velocities (25, 75, 125°/s), and 5 different durations that spanned a logarithmic space (0.5, 0.9, 1.7, 3.2, 6.0 seconds). Each trial was repeated 5 times, resulting in a total of 75 trials per task. The order of the trials was randomized. The order in which the continuous motor and discrete tasks were performed was counterbalanced across participants. Each time perception task required approximately 10 minutes to complete. Upon performing the time perception tasks for the first time (pre-adaptation), 4 practice trials were completed. The 4 practice trials were conducted at 0.5 and 6 seconds for both 25 and 125°/s.
Participants were randomly assigned to one of the two VR conditions: MCTF VR or control VR. In both conditions, the VR exposure occurred in two phases and exposure lasted for 10 minutes in each phase. An additional group of participants were assigned to complete the non-VR control task.
We assessed if participants correctly performed the time perception tasks by measuring the correlation between the ‘actual’ and ‘perceived’ durations across trials for each individual (conditions and trial blocks were pooled). The data from one participant were excluded from further analysis due to a non-significant correlation (p > 0.05), and data from two additional participants were excluded due to errors with data recording (excluded from VR MCTF n = 2; excluded from VR control n = 1; none excluded from non-VR control). For the remaining 31 participants the mean Pearson r(148) score for the correlation between actual and perceived durations was high, both in the continuous motor task (M = 0.89 ± 0.09 SD), and in the discrete motor task (M = 0.89 ± 0.08 SD; all ps < 0.001).
Analysis of participants’ performance in the VR task revealed that there was no significant difference between scores in the first and second VR blocks (paired samples t-tests; VR Control, t(12) = 1.31, p = 0.21; VR MCTF, t(11) = 0.81, p = 0.43), nor was there a difference between the VR MCTF and VR Control groups (independent samples t-tests, block 1, t(23) = 1.27, p = 0.22; block 2, t(23) = 1.85, p = 0.08).
Informal observation of participants during the task indicated that participants in the VR MCTF group used several strategies in order to optimise performance in the task. The majority of participants demonstrated a ‘freezing’ strategy during the task: the participant would adopt and maintain a specific posture for 2–3 seconds and observe the environment while time was moving slowly; following this, the participant would break out of this posture in order to act (e.g., shooting an enemy). While the majority of participants in the VR MCTF group adopted this strategy, it was used infrequently, and did not appear to result in measurable differences in performance (i.e., total point scores).
Raw scores for temporal duration estimates are plotted below for the discrete (Fig. 4) and continuous (Fig. 5) time perception tasks. These scores were separated by block and were used to compute ‘adaptation scores’: the difference in duration estimates between pre- and post-adaptation (see Fig. 6). As such, negative adaptation scores indicate that post-adaptation estimates were shorter than pre-adaptation estimates, and positive adaptation scores indicate that estimates were longer in the post-adaptation block.
We assessed differences between pre- and post-adaptation scores using a series of one-sample tests (Wilcoxon signed rank tests) on adaptation scores (post- minus pre-adaptation duration estimates) that were pooled across duration and velocity conditions. The results revealed adaptation effects that were significantly different from zero for the VR MCTF group in the continuous motor time perception task (M = −0.44 s, SEM = 0.13 s, Wilcoxon’s V = 3, p = 0.002), as shown in Fig. 6. For the VR MCTF group in the discrete time perception task (M = 0.12 s, SEM = 0.05 s, Wilcoxon’s V = 64, p = 0.052), and for the VR control group in both continuous (M = −0.12 s, SEM = 0.13 s, Wilcoxon’s V = 32, p = 0.38) and discrete (M = 0.01 s, SEM = 0.07 s, Wilcoxon’s V = 47, p = 0.95) tasks, we observed no evidence of significant adaptation.
To determine the source of the adaptation effects, we conducted a mixed design ANOVA with the within-subjects factors trial duration (0.5–6.0 sec), trial velocity (25/75/125°/s), task type (continuous/discrete motor time perception task), and the between-subjects factor MCTF (MCTF/normal time). Greenhouse-Geisser corrections were applied to account for sphericity assumption violations where appropriate.
Adaptation differed significantly as a function of both trial duration (F(1.63, 37.53) = 14.12, p < 0.001, Generalized Eta Squared (GES) = 0.10) and velocity (F(1.59, 36.52) = 10.95, p < 0.001, GES = 0.02). Trend analysis indicated that adaptation scores were more negative in the longer duration trials (linear trend t(92) = 6.93, p < 0.001, Cohen’s d = 0.72) and in the slower velocity trials (linear trend t(46) = 4.66, p < 0.001, Cohen’s d = 0.68). Task type also affected adaptation, with more negative adaptation scores in the continuous motor task than the discrete motor task (t(23) = 3.72, p = 0.001, Cohen’s d = 0.77). No main effect of VR MCTF was observed (p = 0.38).
There were several significant interactions, including interactions between MCTF and other factors that were relevant to our hypothesis. We observed a significant three-way interaction between MCTF, task type, and trial duration (F(2.04, 46.85) = 6.25, p = 0.004, GES = 0.02) and a significant two-way interaction between MCTF and task type (F(1, 23) = 5.58, p = 0.027, GES = 0.03). In addition, task type interacted significantly with both trial duration (F(2.04, 46.85) = 3.61, p = 0.015, GES = 0.01) and trial velocity (F(1.80, 41.37) = 3.75, p = 0.04, GES = 0.005). No other interactions were significant (ps ≥ 0.06).
We conducted least-squares means tests with Tukey adjustments for multiple comparisons63 to follow up significant interactions and main effects. Examining the three-way interaction, we found adaptation scores in the continuous motor task for the MCTF group were significantly more negative only at trial durations of 3.2 sec (t(127.04) = 2.51, p = 0.013, Cohen’s d = 0.22) and 6.0 sec (t(127.04) = 4.50, p < 0.001, Cohen’s d = 0.40) (other ps ≥ 0.23). For trials where the probe duration was 6.0 sec, the magnitude of adaptation was 0.87 sec, which indicates an increase in estimates of probe durations of 14.5%.
The interaction between MCTF and task type was due to adaptation scores being significantly more negative for the MCTF group in the continuous motor task (t(44.66) = 2.21, p = 0.03, Cohen’s d = 0.32) but not the discrete motor task (p = 0.41). This interaction is depicted in Fig. 7.
Next we examined the interaction between task type and duration/velocity of trials. Compared to the discrete motor task, adaptation scores in the continuous motor task were significantly more negative for all durations (ts(56.24) ≥ 2.17, ps ≤ 0.03, Cohen’s ds ≥ 0.28) except for 0.5 sec (p = 0.08). In the continuous motor task, adaptation scores tended to be more negative for low velocity trials than higher velocity trials (ts(88.62) ≥ 3.05, ps ≤ 0.06, Cohen’s ds ≥ 0.23; although the difference between 75 and 125°/s did not reach significance, p = 0.06). Velocity had no effect on adaptation scores in the discrete motor task (ps ≥ 0.15).
We also conducted a post-hoc analysis to assess if the difference in post-adaptation duration estimates between the VR MCTF and VR Control groups was attributable to group differences in spatial error when reproducing the movement of the probe. In order to assess spatial error we computed the absolute distance between the endpoint of the probe and the participant’s movement in each axis (left-right, and up-down; X and Y respectively) for each trial. We averaged these values across trials for each participant, resulting in average endpoint error estimates for X and Y. Exemplar spatiotemporal trajectories are plotted in Fig. 8.
We found no significant difference between the post-adaptation spatial error values for the VR MCTF and VR Control groups in terms of X (t(24) = 0.28, p = 0.78, Cohen’s d = 0.11), Y (t(24) = 0.30, p = 0.77, Cohen’s d = 0.12), or the average of the X and Y values (t(24) = 0.29, p = 0.77, Cohen’s d = 0.10). The absolute spatial errors demonstrated by the two groups were as follows: VR MCTF, MX = 12.46 mm, SDX = 4.76 mm, MY = 12.57 mm, SDY = 7.36 mm; VR Control, MX = 11.90 mm, SDX = 5.34 mm, MY = 11.72 mm, SDY = 7.08 mm.
Given that the difference between duration estimates in the VR MCTF and VR Control groups was largest in the 6-second trials, we also assessed spatial errors in these trials alone. Again, no difference was observed between the groups with respect to X error (t(24) = 0.24, p = 0.81, Cohen’s d = 0.09), Y error (t(24) = 0.20, p = 0.84, Cohen’s d = 0.12), or averages of X and Y errors (t(24) = 0.24, p = 0.81, Cohen’s d = 0.09) in the 6 second probe trials (absolute spatial errors: VR MCTF, MX = 15.03 mm, SDX = 6.80 mm, MY = 13.83 mm, SDY = 9.15 mm; VR Control, MX = 14.43 mm, SDX = 5.99 mm, MY = 13.17 mm, SDY = 7.30 mm).
Non-virtual reality control task
We conducted a similar analysis for data obtained in the non-VR control task. We assessed if there were adaptation effects using one-sample tests (Wilcoxon signed rank tests) on the adaptation scores. For both the continuous (M = −0.10 s, SEM = 0.14 s, Wilcoxon’s V = 8, p = 0.69) and discrete (M = 0.12 s, SEM = 0.11 s, Wilcoxon’s V = 16, p = 0.31) tasks, we found no evidence of significant adaptation. The results were overall highly similar to those obtained for the VR control group (see Fig. 6). As in the VR control group, we observed a significant effect of velocity on adaptation effects (F(2, 10) = 4.36, p = 0.043, GES = 0.05) which was driven by significantly more negative adaptation effects in the low velocity (25°/sec) trials compared to the high velocity (125°/sec) trials (t(19.87) = 3.22, p = 0.01, Cohen’s d = 0.70) only for the continuous motor task (discrete motor task ps ≥ 0.72). We observed no other significant main effects or interactions (ps ≥ 0.08).
Cybersickness levels were very low across participants. Only 5 participants reported cybersickness levels that were greater than zero, and of those 5 participants, the maximum score reported was 3 (range of 0 of 20), with an average of 2.38 (SEM = 0.26).
The plasticity of time perception at the sub-second scale is well established in previous literature2,3,4,42. Here we examined the potential for inducing a novel relationship between action and perception that would influence the perceived duration of time. In line with predictions, the results revealed significant effects of the manipulation we introduced in VR. These effects emerged only in the continuous motor time-perception task such that the probe intervals were under-estimated by approximately 15% following exposure to the VR MCTF manipulation for the longest duration trials. The lack of adaptation for VR and non-VR control groups supports the conclusion that temporal recalibration was induced by means of a novel sensorimotor contingency between movement and event speed.
The sensorimotor contingency theory of perception49,50,51,64 has been supported by evidence that novel relationships can be experimentally induced between perception and action for several perceptual qualia, including color52 and musical sound65. Consistent with these studies, our results indicate a significant reduction in the reproduced probe duration following adaptation to a movement-contingent VR game. The current results provide the first evidence for a novel contingency between movement and time perception. The findings have implications for our understanding of the embodied nature of temporal processing, and reiterate the potential for temporal recalibration that has been shown previously. This potential is highly relevant for future rehabilitation initiatives that focus on preventing or reversing maladaptive changes in temporal processing, such as those occurring with age.
The results revealed no evidence in support of temporal recalibration in participants who completed either the control VR task or control non-VR task. This supports a specific effect on the VR manipulation and suggests that neither VR nor physical activity alone were responsible for the temporal recalibration we observed. Adaptation effects were obtained in the continuous motor psychophysical task, but not the discrete motor task. We interpret this effect in terms of a sensorimotor contingency induced by coupling the speed of events in VR to the movement of the participant. In other words, participants who experienced the manipulation perceived the passage of time to be slower when they are not moving, and vice versa. This is indicated by a significant reduction in duration between the observation of the probe (static observer), and reproducing the movement (moving observer). In the discrete motor task, observing and reproducing the probe required no movement from the participant, and thus no effects of the novel sensorimotor contingency were observed.
Results revealed a significant reduction in the reproduced probe duration following exposure to the VR MCTF manipulation of approximately 400 ms on average. However, it should be noted that the majority of this effect was carried by significant adaptation for the longest duration trials, where approximately 0.9 sec adaptation was observed for 6 sec trials (~15% decrease in the reproduced probe duration). Note that while the MCTF manipulation caused event speed to slow by a factor of eight, the reproduced durations were only decreased by a factor of 1.15 in the condition that gave rise to the strongest effect here (6 sec trials). We interpret this result as evidence of a (partial) multiplicative slowdown in the perceived passage of time when the observer was stationary. For short trials (0.5 sec) a multiplicative slowdown would generate small differences between the perceived and reproduced durations, relative to the variability associated with internal/motor noise. At longer duration trials, however, a multiplicative modulation of the reproduced duration would result in larger effects that can be more easily detected among the noise sources. Since spatial error did not differ between the VR Control and VR MCTF groups (overall, and for only the 6 second trials), this leads us to rule out a specific effect of the MCTF manipulation on spatial errors during the reproduction task, and to instead conclude that temporal recalibration occurred for the VR MCTF participants.
Several previous studies have documented sub-second temporal recalibration. Vroomen and colleagues exposed observers to audio-visual stimuli that contained latencies of 100–200 ms4. Their results revealed a shift in the point of subjective simultaneity between two multisensory cues of approximately 10–15 ms that occurred in the direction of the exposure lag. Rohde and colleagues also reported sub-second temporal adaptation by introducing a 200 ms lag between hand and cursor movement in a manual tracking task3. The time course of adaptation to visuomotor latency revealed an adaptation effect in terms of a 30 ms decrease in motor error with time, and large aftereffects in motor timing. The effects we observed here were large relative to these previous studies, and this may be partially due to the exposure delays used in previous studies. For instance, Vroomen and co-workers4 did not expose participants to larger delays between multisensory cues than 200 ms due to the likelihood that greater delays would extinguish perceptual binding of the cues.
Participants were generally very reliable with respect to their responses on the time perception tasks, as we observed strong correlations between the reported and actual durations of the probes for the majority of participants. The results showed that participants tended to over-estimate durations for the shorter trials, especially at the shortest duration trials. This finding aligns with work by others showing overestimation of durations when short intervals (<1 sec) were reproduced10,66,67. However, the same studies identified trends consistent with underestimation for longer durations (5–10 sec), which we did not observe here. While the overestimation at short durations could indicate a perceptual effect (whose origin is unexplained), it might also reflect a methodological constraint on the speed at which participants could make a response. Future replications with more rapid response techniques (e.g., tactile interfaces) may shed light on this apparent overestimation.
In the current study, trial durations in the time perception tasks were limited to 6 seconds. Given that the short-duration trials typically revealed no adaptation effects, future replications of this study can benefit from discarding these conditions in favor of assessing whether similar effects emerge in longer duration trials. Another problematic issue inherent to our design was the lack of experimental control involved in the completion of the VR task. We employed an off-the-shelf, high fidelity VR game in our experiment to enhance the perceptual affordances and naturalism of the environment, but this choice meant that each participant had a slightly different experience while completing the task. This likely contributed a source of noise to the adaptation effects, given that some participants might have explored and engaged with the environment less than others. Greater experimental control is required to clarify the inter-individual variability in temporal recalibration effects. In addition, the movement of the participant was determined by the movement of the hands and the head, whereas in future experiments, a more compelling sensorimotor contingency may be induced if full body motion tracking is employed to modulate VR event speed. We also observed differences in the behaviour of participants through informal observations, but since we did not quantify this behaviour (e.g., analysis of head or hand velocity during VR), we are unable to determine if differences in performance strategies across participants in the VR MCTF group contributed to any variance in the adaptation effects. However, analyses of participant performance did not reveal any evidence of a difference in performance outcomes (i.e., total point scores). We intended the task to have a high ecological validity and strong affordances for action, and as such, the VR MCTF manipulation was expected to affect behaviour in overt and subtle ways. However, a full quantification of the behavioural changes produced by this manipulation would be a desirable outcome of future research.
According to the verbal self-report measures of comfort, participants experienced very little cybersickness during the experiment, suggesting that the VR task was a comfortable experience. These findings provide evidence that VR can be used to induce temporal recalibration, which may contribute to a clinical intervention in preventing or reversing the maladaptive changes to time perception, such as those observed in aging, Parkinson’s disease, schizophrenia, autism, and ADHD (for review, see54). However, the current findings are highly preliminary, and show an effect that is specific to a motor task involving continuous movement and may not be beneficial for rehabilitation purposes. Due to our specific time-flow manipulation, participants observed a reduction in the reproduced probe durations when stationary compared to when moving; in practice, it may be more useful to slow down the perceived passage of time when an individual is moving, thus enabling better control of the body (e.g., during a fall when establishing stable balance requires relocating the centre of gravity). It remains to be seen if adaptation effects opposite to those we obtained here can be produced. Several other extensions, refinements, and replications will be required before a similar paradigm is used in a clinical setting.
How does the brain represent the temporal recalibration effects observed here? Although our experiment was not designed to test mechanistic theories of time perception, our findings are consistent with multiple accounts, including the internal clock11. For instance, a movement-dependent increase in the number of pulses emitted by the internal oscillator per unit of time, or modulation of the gating of these pulses by the calibration unit (as in the click train effect2,14,42, would result in the pattern of data obtained. At the same time, the spatial or temporal pattern of neuronal activity may have been modulated by adaptation17,18,19. The transfer of adaptation from a VR setting to a simple psychophysics task supports adaptation of a central mechanism for time perception, in line with other evidence13. However, future neuroimaging studies are needed to provide insight into the neural basis of sensorimotor contingency acquisition for temporal processing.
In summary, here we exposed participants to a VR experience where movement and event-speed were experimentally coupled, and we observed evidence that a novel relationship between action and event-speed caused recalibration of time perception in a psychophysical task. The sensorimotor contingency between movement and the speed of events induced temporal recalibration that did not emerge in control VR and non-VR conditions. This study provides further evidence of the flexibility of time perception, and indicates that sensorimotor contingency theory offers a useful framework for studying high-level perceptual qualia such as time perception. The utility of VR for modulating time perception is also evident from these results, although future refinements are needed before the practical relevance of these findings can be established.
The data that support the findings of this study will be made freely available upon publication on the Open Science Framework: Bansal, A., Weech, S., & Barnett-Cowan, M. (2018, September 10). Movement-Contingent Time Flow in Virtual Reality Causes Temporal Recalibration. Retrieved from osf.io/nt2jh.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Faisal, A. A. & Wolpert, D. M. Near optimal combination of sensory and motor uncertainty in time during a naturalistic perception-action task. J Neurophysiol. 101, 1901–1912 (2008).
Wearden, J. H. Slowing down an internal clock: Implications for accounts of performance on four timing tasks. Q J Exp Psychol. 61, 263–274 (2008).
Rohde, M., van Dam, L. C. & Ernst, M. O. Predictability is necessary for closed-loop visual feedback delay adaptation. J Vis. 14, 4–4 (2014).
Vroomen, J., Keetels, M., Gelder, B. D. & Bertelson, P. Recalibration of temporal order perception by exposure to audio-visual asynchrony. Cogn Brain Res. 22, 32–35 (2004).
Barnett-Cowan, M. Introduction to the special issue on multisensory processing and aging. Multisens Res. 31, 145–149 (2018).
Coelho, M. et al. Assessment of time perception: The effect of aging. J Int Neuropsychol Soc. 10, 332–341 (2004).
de Dieuleveult, A. L., Siemonsma, P. C., van Erp, J. B. & Brouwer, A. M. Effects of aging in multisensory integration: a systematic review. Front Aging Neurosci. 9, 80 (2017).
Gunstad, J., Cohen, R. A., Paul, R. H., Luyster, F. S. & Gordon, E. Age effects in time estimation: relationship to frontal brain morphometry. J Integr Neurosci. 5, 75–87 (2006).
Grondin, S. Timing and time perception: A review of recent behavioural and neuroscience findings and theoretical directions. Atten Percept Psychophys. 72, 561–582 (2010).
Bobko, D. J., Thompson, J. G. & Schiffman, H. R. The perception of brief temporal intervals: Power functions for auditory and visual stimulus intervals. Perception. 6, 703–709 (1977).
Treisman, M. Temporal discrimination and the indifference interval: Implications for a model of the “internal clock”. Psychol Monogr. 77, 1 (1963).
Ivry, R. B. & Schlerf, J. E. Dedicated and intrinsic models of time perception. Trends Cogn Sci. 12, 273–280 (2008).
Heron, J., Hanson, J. V. & Whitaker, D. Effect before cause: supramodal recalibration of sensorimotor timing. PLoS ONE. 4 (2009).
Treisman, M., Faulkner, A., Naish, P. L. & Brogan, D. The internal clock: Evidence for a temporal oscillator underlying time perception with some estimates of its characteristic frequency. Perception 19, 705–742 (1990).
Keele, S. W., Pokorny, R. A., Corcos, D. M. & Ivry, R. Do perception and motor production share common timing mechanisms: A correlational analysis. Acta Psychol. 60, 173–191 (1985).
Coull, J. T., Cheng, R. K. & Meck, W. H. Neuroanatomical and neurochemical substrates of timing. Neuropsychopharmacology. 36, 3 (2011).
Karmarkar, U. R. & Buonomano, D. V. Timing in the absence of clocks: encoding time in neural network states. Neuron. 53, 427–438 (2007).
Medina, J. F. & Mauk, M. D. Computer simulation of cerebellar information processing. Nat Neurosci. 3, 1205–1211 (2000).
Durstewitz, D. Self-organized neural integrator predicts interval times through climbing activity. J Neurosci. 23, 5342–5353 (2003).
Grondin, S., Bisson, N., Gagnon, C., Gamache, P. L. & Matteau, A. A. Little to be expected from auditory training for improving visual temporal discrimination. Neuro Quantology. 7, 95–102 (2009).
Grondin, S. & McAuley, J. D. Duration discrimination in crossmodal sequences. Perception. 38, 1542–1559 (2009).
Lapid, E., Ulrich, R. & Rammsayer, T. On estimating the difference limen in duration discrimination tasks: A comparison of the 2AFC and the reminder tasks. Percept Psychophys. 70, 291–305 (2008).
Mauk, M. D. & Buonomano, D. V. The neural basis of temporal processing. Annu Rev Neurosci. 27, 307–340 (2004).
Rao, S. M., Mayer, A. R. & Harrington, D. L. The evolution of brain activation during temporal processing. Nature. 4, 317–322 (2001).
Gibbon, J., Malapani, C., Dale, C. L. & Gallistel, C. Toward a neurobiology of temporal cognition: advances and challenges. Curr Opin Neurobiol. 7, 170–184 (1997).
Fierro, B. et al. Role of the cerebellum in time perception: a TMS study in normal subjects. J Neurol Sci. 263, 107–112 (2007).
Koch, G. et al. Repetitive TMS of cerebellum interferes with millisecond time processing. Exp Brain Res. 179, 291–299 (2007).
Lee, K. H. et al. The role of the cerebellum in subsecond time perception: evidence from repetitive transcranial magnetic stimulation. J Cogn Neurosci. 19, 147–157 (2007).
Jones, C. R., Rosenkranz, K., Rothwell, J. C. & Jahanshahi, M. The right dorsolateral prefrontal cortex is essential in time reproduction: an investigation with repetitive transcranial magnetic stimulation. Exp Brain Res. 158, 366–372 (2004).
Vallesi, A., McIntosh, A. R., Shallice, T. & Stuss, D. T. When time shapes behavior: fMRI evidence of brain correlates of temporal monitoring. J Cogn Neurosci. 21, 1116–1126 (2009).
Pastor, M. A., Artieda, J., Jahanshahi, M. & Obeso, J. A. Time estimation and reproduction is abnormal in Parkinson’s Disease. Brain. 115, 211–225 (1992).
Coull, J. T., Nazarian, B. & Vidal, F. Timing, storage, and comparison of stimulus duration engage discrete anatomical components of a perceptual timing network. J Cogn Neurosci. 20, 2185–2197 (2008).
Harrington, D. L., Zimbelman, J. L., Hinton, S. C. & Rao, S. M. Neural modulation of temporal encoding, maintenance, and decision processes. Cereb Cortex. 20, 1274–1285 (2010).
Coull, J. T. & Nobre, A. Dissociating explicit timing from temporal expectation with fMRI. Curr Opin Neurobiol. 18, 137–144 (2008).
Meck, W. H. Selective adjustment of the speed of internal clock and memory processes. J Exp Psychol. 9, 171–201 (1983).
Williamson, L. L., Cheng, R. K., Etchegaray, M. & Meck, W. H. “Speed” warps time: methamphetamine’s interactive roles in drug abuse, habit formation, and the biological clocks of circadian and interval timing. Curr Drug Abuse Rev. 1, 203–212 (2008).
Terhune, D. B., Russo, S., Near, J., Stagg, C. J. & Kadosh, R. C. GABA predicts time perception. J Neurosci. 34, 4364–4370 (2014).
Maricq, A. V., Roberts, S. & Church, R. M. Methamphetamine and time estimation. J Exp Psychol. 7, 18–30 (1981).
MacDonald, C. J. & Meck, W. H. Interaction of raclopride and preparatory-interval effects on simple reaction-time performance. Behav Brain Res. 175, 62–74 (2006).
Buhusi, C. V. & Meck, W. H. Relative time sharing: new findings and an extension of the resource allocation model of temporal processing. Philos Trans R Soc Lond B Biol Sci. 364, 1875–1885 (2009).
Meck, W. H. Affinity for the dopamine D2 receptor predicts neuroleptic potency in decreasing the speed of an internal clock. Pharmacol Biochem Behav. 12, 1185–1189 (1984).
Penton-Voak, I. S., Edwards, H., Percival, A. & Wearden, J. H. Speeding up an internal clock in humans? Effects of click trains on subjective duration. J Exp Psychol. 22, 307–320 (1996).
Chan, Y. M., Pianta, M. J. & McKendric, A. M. Reduced audiovisual recalibration in the elderly. Front Aging Neurosci. 6, 226 (2014).
Fujisaki, W., Shimojo, S., Kashino, M. & Nishida, S. Recalibration of audiovisual simultaneity. Nat Neurosci. 7, 773–778 (2004).
Keetels, M. & Vroomen, J. No effect of auditory–visual spatial disparity on temporal recalibration. Exp Brain Res. 182, 559–565 (2007).
Kennedy, J. S., Buehner, M. J. & Rushton, S. K. Adaptation to sensory-motor temporal misalignment: Instrumental or perceptual learning? G J Exp Psychol. 62, 453–469 (2009).
Meegan, D. V., Aslin, R. N. & Jacobs, R. A. Motor timing learned without motor training. Nat Neurosci 3, 860–862 (2000).
Pesavento, M. J. & Schlag, J. Transfer of learned perception of sensorimotor simultaneity. Exp Brain Res. 174, 435–442 (2006).
O’Regan, J. K. Why Red Doesn’t Sound Like a Bell: Understanding the Feel of Consciousness. (New York: Oxford University Press, 2011).
O’Regan, J. K. & Noë, A. A sensorimotor account of vision and visual consciousness. Behav Brain Sci. 24, 939–973 (2001).
Little, D. Y. & Sommer, F. T. Learning and exploration in action-perception loops. Front Neural Circuits. 7, 1–19 (2013).
Bompas, A. & O’Regan, J. K. Evidence for a role of action in colour perception. Perception. 35, 65–78 (2006).
Toplak, M. E., Dockstader, C. & Tannock, R. Temporal information processing in ADHD: findings to date and new methods. J Neurosci Methods. 51, 15–29 (2006).
Allman, M. J., DeLeon, I. G. & Wearden, J. H. Psychophysical assessment of timing in individuals with autism. Am J Intellect Dev Disabil. 116, 165–178 (2011).
Carroll, C. A., Boggs, J., O’Donnell, B. F., Shekhar, A. & Hetrick, W. P. Temporal processing dysfunction in schizophrenia. Brain and Cognition 67, 150–161 (2008).
Leisman, G. & Melillo, R. Effects of motor sequence training on attentional performance in ADHD Children. Int J Disabil Hum Dev. 9 (2010).
Melillo, R. & Leisman, G. Autistic spectrum disorders as functional disconnection syndrome. Rev Neurosci. 20, 111–31 (2009).
Allman, M. J. & Meck, W. H. Pathophysiological distortions in time perception and timed performance. Brain. 135, 656–677 (2011).
Wright, W. G. Using virtual reality to augment perception, enhance sensorimotor adaptation, and change our minds. Front Syst Neurosci. 8 (2014).
Brainard, D. H. The psychophysics toolbox. Spat Vis. 10, 433–436 (1997).
Curton, E. D. & Lordahl, D. S. Effects of attentional focus and arousal on time estimation. J Exp Psychol. 103, 861 (1974).
Droit-Volet, S., Brunot, S. & Niedenthal, P. Perception of the duration of emotional events. Cogn Emot. 18, 849–858 (2004).
Lenth, R. V. Least-squares means: the R package lsmeans. J Stat Softw. 69, 1–33 (2016).
König, S. U. et al. Learning new sensorimotor contingencies: Effects of long-term use of sensory augmentation on the brain and conscious perception. PloS ONE. 11 (2016).
Pieter-Jan, M., Leman, M., Palmer, C. & Wanderley, M. M. Action-based effects on music perception. Front Psychol. 4, 1–14 (2014).
Eisler, H. Experiments on subjective duration 1868-1975: A collection of power function exponents. Psychol Bull. 83, 1154–1171 (1976).
Schab, F. R. & Crowder, R. G. Accuracy of temporal coding: Auditory-visual comparisons. Mem Cogn. 17, 384–397 (1989).
McGill, R., Tukey, J. W. & Larsen, W. A. Variations of box plots. Am Stat. 32, 12–16 (1978).
This research was supported by funding from NSERC and Oculus Research awarded to MBC.