Effects of synchronised engine sound and vibration presentation on visually induced motion sickness

Abstract

Driving simulator usage is often accompanied by motion sickness, and techniques for its prevention are not yet established. To reduce visually induced motion sickness (VIMS), we investigated the effects of synchronised presentation of engine sounds and motorcycle vibration on VIMS. A total of 80 participants experienced a driving scene with a head-mounted display for 5 minutes with or without synchronised presentation of engine sound and vibration. The results showed that VIMS scores, as measured by the Fast Motion Sickness scale, were significantly lower in participants who experienced the driving scene with sounds and vibration than in those who experienced the scene with sounds only, vibration only, or neither. Multiple regression analyses revealed that susceptibility to VIMS consistently explained the severity of VIMS to some extent but not with perceived realism of the virtual reality (VR) scene, sex, and experiences about VR devices and vehicles. This study demonstrated that simultaneous presentation of engine sounds and vibration, which were synchronous to each other and tightly coupled with the visual flow speed, effectively reduces VIMS while experiencing motorcycling simulators. The findings not only improve practical knowledge for reducing VIMS in driving simulators but also provide evidence for understanding the mechanisms of VIMS.

Introduction

Driving simulators1 let their users experience virtual driving scenes, which is useful in many applications, including the safe evaluation and improvement of driving abilities2,3,4,5. However, driving simulators can cause simulator sickness, which is one type of visually induced motion sickness (VIMS)6,7. VIMS is a broad concept, including cybersickness and virtual reality (VR) sickness, and is elicited at a probability of 30–80%8. Although the frequency and magnitude of VIMS depend on simulator types and driving tasks, VIMS can be elicited regardless of the availability of a motion system (fixed-base or moving-base) and the type of displays. The severity of the sickness increases by the length of visual scene observation7,9,10. VIMS is characterised by various symptoms such as drowsiness, dizziness, fatigue, cold sweat, headache, nausea, stomach discomfort, pallor and vomiting11,12,13. Although previous studies have tried to reveal the mechanism of VIMS using extensive types of visual stimuli, such as driving scenes3,4,14 and VR games15,16, a consensus has not been reached about what causes VIMS. Furthermore, some of the currently proposed techniques (e.g., narrowing the field of view) to reduce VIMS could deteriorate the VR experience, or the effect size was not reliable17; therefore, immediate implementation of the techniques has been hindered. In this study, we examined the effect of synchronised presentation of engine sound and vibration on VIMS to develop more comfortable driving simulators in a simple and economical way.

The most influential theory for explaining the mechanism of motion sickness is the sensory conflict theory18, which, however, has been loosely defined and has many variants in the literature11. The sensory conflict theory originally assumes that motion sickness is caused by a mismatch between sensory inputs between modalities. Reason18 introduced the idea of a neural store, which represents the memory of paired sensory input and motor command, and thus neural mismatch signals are proportional to the magnitude of a difference between the expected sensory input and actual sensory input11. Although Reason18 and Oman11 mainly focused on the sensory conflict with vestibular inputs, they also suggest that conflicts between visual and other modality inputs without vestibular conflicts can provoke motion sickness. According to this theory, motion sickness associated with a driving simulator is induced by the inconsistency between the expected sensory inputs based on a real-world experience and actual sensory inputs in terms of sensory correspondence of visual, auditory, somatosensory and vestibular sensory information.

Various factors have been demonstrated to be involved in VIMS. Draper, et al.19 showed that more than 48 ms of display delay in a head-mounted display (HMD) drastically increased the occurrence of VIMS, which has impacted the guideline of HMD developers like Oculus (Facebook technologies, LLC., USA), which requires the display delay (motion-to-photon latency) to be less than 20 ms. The motion sickness is more likely to be induced by 3-dimensional than by 2-dimensional visual stimuli20. Display resolution, field of view and angular velocity of the visual stimulus are also known factors associated with VIMS19,21. Individual characteristics such as sex15, age 5,22, stress and driving experience23, as well as sleep problems24 may affect the severity or probability of VIMS. In addition, VIMS could be reduced by the adaptation for repetitive experiences10,25,26, narrowing the field of view 17,27, showing a “nose” in the visual field as a reference28, bone-conducted vibration to the vestibular system29 and a comfortable music and smell30.

Usually, it is not easy to avoid motion sickness when one watches a moving scene on a display that has a large field of view. D’Amour, et al.31 investigated the effects of vibration and airflow on VIMS. In their experiment, participants watched a visual scene from a first-person view that one is driving a bicycle in a town. They found a significant reduction effect only of airflow, but not of vibration, on VIMS. In addition, Keshavarz and Hecht did not found significant reduction effects either of background sounds during a bicycle ride20, nor of environmental noises, footsteps or the character’s breathing during a video game from the first-person perspective32. These findings suggest that airflow, but not a simple presentation of sounds and vibration, is effective in reducing VIMS at least in experiencing a VR scene of a bicycle ride, which usually runs against the wind. Furthermore, a recent study of Keshavarz, et al.5 reported that presenting engine sounds, motion cues and their combination did not reduce VIMS in a car driving simulator.

Nevertheless, in a motorcycle driving scene, it is expected that synchronously presented engine sounds and vibration have substantial effects on VIMS reduction. The sensory conflict theory argues that VIMS will not happen if anticipated sensory integration occurs or multisensory integration is successfully resolved7; therefore, the presentation of more than one modality of sensory inputs that coincide in reality would decrease VIMS. Accordingly, it would be possible that synchronised presentation of engine sounds and vibration, which are always coupled and proportional to engine speed in reality, decreases VIMS, although the simple presentation of them cannot reduce VIMS. A simple addition of individual sensory feedback could rather elicit a “conflict” to the stored sensory integration because the corresponded feedbacks are missing. This mechanism is possibly linked to the “uncanny valley” in the humanoid literature. In the study by D’Amour, et al.31, the simultaneous presentation of airflow and vibration did not decrease VIMS compared with a simple presentation of airflow. This result could be explained by assuming that the addition of information is not coupled in reality, and the vibration is constant regardless of the driving scene or airflow. The insignificant effect of background noise20,32 can be explained by introducing the idea that the motion of the visual field and background sounds were not sufficiently coupled; that is, the sounds such as the noise of the town might not be helpful to predict changes in the first person’s view. The insignificant effects of combining speed-dependent engine sounds and motion cues on VIMS in a car driving simulator5 might be due to the lack of tight coupling among engine sounds, motion cues and the visual flow, although the motion information was not clear from the study description. The present study thus focused on the sounds and vibration both induced by an engine, which are inevitably accompanied by a motorcycle driving.

In this study, we specifically investigated whether a synchronised presentation of engine sounds and vibration decreases VIMS while watching a motorcycle-driving scene in a virtual environment. The engine sounds and vibration were modulated by motorcycle driving speed. Participants sat on a chassis of a scooter and wore an HMD, and vibration was provided to participants under the seat. An HMD has advantages in presenting an immersive visual environment without large-scaled equipment and in installing dynamic interactions easily 33,34. In addition, applying vibration has an advantage in installing to driving simulators because the system can be established with consumer devices, and the algorithm is not special. We used the Simulator Sickness Questionnaire (SSQ)35 and Fast Motion Sickness (FMS) scale9 to evaluate motion sickness based on subjective reports31. SSQ is a questionnaire to evaluate the degree of motion sickness subjectively, and FMS was invented to quickly evaluate the degree of motion sickness by verbally reporting the number. Because previous studies have suggested other individual factors such as realism and presence of virtual environment and past experiences about car, motorcycle and VR could affect VIMS 23,34,36,37, we additionally analysed these factors using multiple linear regression.

Results

FMS and SSQ scores

The participants (n = 80) were randomly assigned to the audio-vibration group (AV group), no-audio-vibration group (no-AV group), audio-only group, or vibration-only group. They experienced a 5-minute motorcycle driving scene. The average FMS scores during watching a driving scene are shown in Fig. 1a and total SSQ scores after watching the scene are presented in Fig. 1b. The average scores related to VIMS were summarised in Table 1. We used a two-factor ANOVA (4 experimental groups × 5 evaluation phases) instead of a three-way ANOVA (audio presentation × vibration presentation × evaluation phase) to focus on the effect of simultaneous audio-vibration presentation on VIMS reduction.

Figure 1
figure1

FMS and total SSQ scores. FMS scores are significantly lower in the AV group than in the other groups. SSQ scores are not significantly different among the groups. FMS: Fast Motion Sickness scale; SSQ: Simulator Sickness Questionnaire; AV: Audio-vibration; Vib: Vibration.

Table 1 Average values of the VIMS-related variables, realism and presence scores.

The two-factor ANOVA on FMS scores revealed significant main effects of the experimental group (F [3, 76] = 3.63, p = 0.02, ω2 = 0.06) and evaluation period (F [4, 304] = 58.64, p < 0.001, ω2 = 0.12), and a significant interaction effect of experimental group and evaluation period (F [12, 304] = 1.95, p = 0.03, ω2 = 0.01). Simple main effects of experimental group were significant in all periods except for the 1st period (1st: F [3, 76] = 2.24, p = 0.09, ω2 = 0.04; 2nd: F [3, 76] = 3.05, p = 0.03, ω2 = 0.07; 3rd: F [3, 76] = 3.94, p = 0.01, ω2 = 0.10; 4th: F [3, 76] = 3.02, p = 0.03, ω2 = 0.07; 5th: F [3, 76] = 3.47, p = 0.02, ω2 = 0.08), and simple main effects of evaluation period were significant in all groups (AV: F [4, 76] = 11.16, p < 0.001, ω2 = 0.10; no-AV: F [4, 76] = 13.66, p < 0.001, ω2 = 0.11; audio-only: F [4, 76] = 11.08, p < 0.001, ω2 = 0.11; vibration-only: F [4, 76] = 25.26, p < 0.001, ω2 = 0.18). Multiple comparisons revealed that from the 2nd period, there were significant differences in FMS between the AV group and other groups. Table 2 indicates the statistical results of the multiple comparisons. Note that we compared only pairs between the AV group and other groups using Holm’s method, considering the focus of the present study and to keep the statistical power in the analysis.

Table 2 The results of multiple comparisons between the AV group and other groups in each evaluation phase.

The average total SSQ scores were 27.3 (SD = 34.0), 44.5 (SD = 34.0), 46.2 (SD = 38.0) and 51.2 (SD = 38.3) in the AV, no-AV, audio-only and vibration-only groups, respectively. A one-way ANOVA found no significant effect of the experimental group (F [3, 76] = 1.83, p = 0.18, ω2 = 0.02). The correlation coefficient between the 5th FMS and total SSQ scores was 0.74 with a 95% confidence interval of 0.62–0.83.

Figure 2 shows average realism and presence scores in each group. Realism scores were 5.6 (SD = 2.4), 4.1 (SD = 1.5), 6.1 (SD = 1.4) and 6.1 (SD = 1.8) in the AV, no-AV, audio-only and vibration-only groups, respectively. Presence scores were 6.6 (SD = 2.2), 6.1 (SD = 2.2), 6.9 (SD = 1.5) and 7.1 (SD = 1.9) in the same order, respectively. A one-way ANOVA found a significant effect of experimental group in the realism score (F [3, 76] = 5.68, p = 0.001; ω2 = 0.15) but not in the presence score (F [3, 76] = 0.97, p = 0.41; ω2 = 0.00). Multiple comparisons revealed that the realism score in the no-AV group was significantly lower than in the other three groups (ts [76] = 2.72, 3.51, and 3.59, p < 0.05).

Figure 2
figure2

Realism (a) and presence (b) scores. The realism score is significantly lower in the no-AV group than in the other groups. AV: Audio-vibration; Vib: Vibration.

Correlation coefficients were calculated to confirm the relationships between VIMS scores, realism and presence scores. Figure 3 shows the scatter plots of each pair. Correlation coefficients and 95% confidence intervals of the realism score were −0.03 (−0.25 and 0.19) for the 5th FMS scores and 0.09 (−0.13 and 0.30) for the total SSQ scores, and those of presence score were 0.19 (−0.03 and 0.39) for the FMS scores and 0.16 (−0.06 and 0.37) for the SSQ scores.

Figure 3
figure3

Scatter plots of presence and realism scores against VIMS scores (5th FMS and total SSQ scores). Different markers indicate data of different experimental groups. Circle: AV group; Triangle: no-AV group; Square: audio-only group; Diamond: vibration-only group. Open markers indicate female data. There are no significant linear correlations between the variables. FMS: Fast Motion Sickness scale; SSQ: Simulator Sickness Questionnaire; AV: Audio-vibration.

To further identify the contributing factors of VIMS, we conducted multiple regression analyses for the 5th FMS and total SSQ scores. To predict the VIMS score, we used the MSSQ score, SSQ score before the experiment (pre-SSQ), realism score, presence score, sex, car experience, motorcycle experience, VR experience, audio presentation and vibration presentation. The latter six variables were treated as binary dummy variables. In addition, we assumed that an interaction effect between presentation and vibration presentation would contribute to the VIMS scores. The results of the multiple regression analyses are given in Table 3. In the analysis for FMS, the coefficient of determination (R2) was 0.33, and the factors of MSSQ and presence were significant with beta coefficients = 0.36 and 0.24, respectively. The beta coefficient for the interaction between audio and vibration was −0.33 but not statistically significant. In the analysis for SSQ, R2 was 0.38, and the pre-SSQ and MSSQ were significant with beta coefficients = 0.33 and 0.39, respectively. The amplitude of the beta coefficient for the interaction between audio and vibration (−0.12) was lower than that of the FMS analysis (−0.32).

Table 3 The results of multiple regression analyses for the FMS and SSQ scores.

Discussion

The present study investigated whether synchronised presentation of engine sound and vibration decreases VIMS while viewing a simulated motorcycle ride in a virtual environment. We showed that FMS scores were significantly lower in the AV group than in the other experimental groups that experienced only sound, vibration, or neither, which supports our hypothesis. In contrast to previous studies that investigated simple effects of sound or vibration, which did not reduce VIMS 20,31,32, this study demonstrated that simultaneous presentation of sound and vibration, where both were modulated by driving speed, can substantially reduce VIMS. Furthermore, a multiple regression analysis revealed that the MSSQ score was able to predict the severity of VIMS in our study.

Significant differences in the average FMS scores were observed even in the 2nd phase (2 minutes after the start of the visual stimulus), indicating that the effect of simultaneous sound and vibration presentation appears immediately after the start of the experiment. However, it should be noted that the FMS score increased as the time elapsed in the AV group. Although there was a significantly higher correlation between FMS in the 5th phase and SSQ scores (r = 0.74), SSQ scores did not significantly differ among the experimental groups. This discrepancy between the VIMS scales could be interpreted by considering that the SSQ evaluates a lot more aspects of motion sickness than FMS and by assuming that VIMS symptoms would decrease after one has finished a VR experience. Collectively, although generalizability of the technique to other VR scenes should be tested in future studies, the present results suggest that the presentation of engine sound and synchronised vibration suppresses the development of VIMS induced by a motorcycle driving scene.

This study, for the first time, showed that the presentation of engine sound and vibration substantially reduced VIMS while watching a driving scene. Whereas previous studies did not find a reliable effect of a simple presentation of sound or vibration on VIMS 5,20,31,32, using engine sounds and vibration that are necessarily entailed by motorcycle transportation, we showed that their combination had a significant reduction effect on VIMS. Adding environmental sounds, which have been used to investigate the role of sound in VIMS 20,32, would not be critical to decrease VIMS according to the sensory conflict theory since the environmental sounds mainly consist of unpredictable sounds, and many of them may not contain information directly linked to the visual flow. In other words, the noise of the town would not bring useful information to predict the motion of the first-person character. By contrast, engine sounds and vibration that sensitively reflect moving speed would help to predict incoming visual feedbacks, and the sensory integration would be more likely consistent with a “stored sensory integration”. Consistent with this idea, constant vibration, which is presented regardless of the visual scene, might not effectively reduce VIMS31. In addition to the reduction effect of audio-vibration presentation on VIMS, our experiments have demonstrated that adding either sound or vibration to a visual stimulus was insufficient to reduce VIMS, even though each of them is coupled with visual changes in a real world. The insignificant effect of the simple presentation of engine sounds is consistent with a previous result of a study with a car-driving simulator5. These findings may raise two possibilities; first, the effect of a simple presentation of sound or vibration is not strong enough to reduce VIMS, and the effect is accumulated and exceeds a certain threshold when they are presented at the same time. This hypothesis is not limited to sound and vibration but includes other types of sensory information, for example, tactile stimulation by airflow. Second, a simple presentation of sound or vibration does not work for reducing VIMS at all because they are tightly coupled with each other in reality and missing one could create an inconsistency in sensory integration. These possibilities are not mutually exclusive and should be further examined in future studies.

The current study also investigated the relations between perceived realism and presence and VIMS. The correlation analysis did not find any reliable correlation coefficients among the factors (r = −0.03–0.19). The findings with regard to the relation between the feelings of presence in the visual scene and VIMS are inconsistent with those in previous studies. A recent review38 has shown that there may be a negative relationship between presence and VIMS; when the perceived presence in a VR increases, VIMS decreases. Moreover, negative correlations between presence and VIMS scores have been found in Nichols, et al.39 and Milleville-Pennel and Charron23. However, D’Amour, et al.31, who investigated VIMS during watching a bicycle-ride scene, did not found a significant correlation between the presence and FMS scores and a positive correlation (r = 0.26) between the realism and FMS scores. Furthermore, in contrast with the review’s implication, our multiple regression analysis showed a positive contribution of presence score to the FMS score (beta = 0.24). The reasons for these divergent results may be caused by inconsistency in VR scenes or tasks, types of VR device38, the severity of VIMS and statistical designs (e.g., whether the correlation analysis included inter-group data or not, and presence/realism was manipulated or co-variated; statistical power related to sample size largely differs between studies). It might also be a problem to explore the relations between the subjective feelings to a VR scene and VIMS solely based on simple correlation analyses because influences by other factors cannot be excluded. The current findings at least suggest that the perceived presence may interact with the mechanism that induces VIMS, although it is not always linearly correlated with VIMS scores.

It is worth to mention two other findings. First, the presentation of sound and/or vibration increased the realism score (ω2 = 0.15) but not the presence score (ω2 = 0.00). These results suggest the presence of the VR scene, “the feeling of being there in the virtual scene”, may not be increased by just presenting engine sounds and vibrations of a motor cycle, whereas the realism, “how real the virtual scene is”, is more sensitive to addition or interaction of sensory information. Consistent with our results, D’Amour et al.31 did not find any significant differences in presence scores among experimental conditions, while one of their experimental condition using air flow and vibration successfully reduced VIMS. It is, nevertheless, possible that a ceiling effect might have occurred in the presence scores, and thus the scores did not differ among the experimental groups because our study used an HMD, which more likely provides us an immersive experience than display-based devices. Second, the MSSQ scores had significant estimated contributions to both FMS and SSQ scores (beta = 0.35 and 0.36), whereas the SSQ score before the experiment had a significant estimated contribution to the SSQ score after the experiment (beta = 0.36). Other factors investigated (sex, car experience, motorcycle experience and VR experience) did not contribute to the severity of VIMS (beta = −0.15–0.11). Consistent with Dennison and D’Zmura40, where MSSQ and SSQ scores were positively related (r = 0.67), we found a significant correlation (FMS: r = 0.41; SSQ: r = 0.40). Note, however, other VIMS studies using the MSSQ have not found a correlation between the MSSQ and VIMS measures41,42,43. Therefore, further studies are needed to confirm that the MSSQ is related to VIMS. Our findings suggest that adding sound or vibration information likely increases the perceived realism of a VR driving scene, and self-reported vulnerability to motion sickness can predict the severity of VIMS symptoms in a motorcycle driving scene, even when other factors are taken into account.

The present study has several limitations. First, this study did not include a sufficient number of female participants; second, the length of the VR experience (5 min) was relatively short; and third, we did not examine the effect of synchronization among the sensory information on VIMS. Nevertheless, the present study provided strong evidence that the combined presentation of sound and vibration significantly reduces VIMS. Future studies should include a larger number of female participants. They could further consider experimental conditions, where sound and vibration are not synchronous with one another or driving speed. Longer presentation of the VR experience would also be useful to remove a possible floor effect of VIMS symptoms and thus make statistical analyses more rigorous. In addition, variables included in the multiple regression model in our study were selected rather in a bottom-up way, and therefore should be refined based on the current results in future studies.

In conclusion, the present study demonstrated a reduction effect of the synchronous presentation of sound and vibration on VIMS. This result may be explained by the sensory conflict theory. In addition, we found that adding sound and vibration increased realism but did not affect presence, suggesting that VIMS can be suppressed without deteriorating the VR experience when using a vibration device. If vibrations from a handlebar or an HMD, instead of the seat vibration, also reduces VIMS, the application of the proposed methods would become further easier. The results of this motorcycle simulation study may also pertain to other driving simulations, where speed-dependent engine sounds and vibrations also play a role. In accordance with sensory conflict theory, we would also expect similar effects for sounds and vibrations that are produced by roads, tracks, or air resistance as well as engine motion itself, that are usually proportional to engine speed. The application to human locomotion in VR scenes may not be the same but worthwhile trying.

Materials and Methods

Participants

Eighty healthy students (16 females, 64 males) participated in the study (Mean age = 21.3 years, SD = 1.2). This study used a between-participants design to avoid the adaptation effect 25,26 on the VR environment and VIMS. The participants were randomly assigned to the audio-vibration (AV), no-audio-vibration (no-AV), audio-only, or vibration-only groups (4 females in each group). All participants were recruited via a university mailing list, and they had no signs of sickness, sleep problems, or sensory-motor problems and provided written informed consent before the study. The study was approved by the ethics committee of Shizuoka University (No. 18–14) and was conducted following the approved guidelines and regulations.

Stimuli and apparatus

The experiment was conducted in an indoor room. Participants watched a visual stimulus through an HMD when sitting on a chassis of a scooter that was removed of an actuator and grabbing the handgrips (Fig. 4a). A vibration device (Vt7, Acouve, Inc., Japan) was equipped under the seat. The scooter’s tires did not touch on the floor, and the body could not lean to the lateral sides. The HMD (Oculus Rift, Facebook Technologies, LLC., USA), which offers the visual resolution of 2160 × 1200 in the 110° field of view, was used to display a driving scene. The driving scene was controlled by Unity. The refresh rate of the display was 90 frames per second. The visual stimulus simulated a driving scene on a winding road from a first-person viewpoint (Fig. 4b). The winding road, which was demonstrated to induce VIMS more than a straight road10, was selected to elicit VIMS. The bird view of the road is shown in Fig. 4c. A driving scene, which took 54 seconds, was repeated seamlessly in a 5-minute experiment. The time of experiencing the driving scene was determined based on the results of a preliminary experiment. A head-tracking function was available during the experiment.

Figure 4
figure4

Experimental settings (a), presented visual scene (b) and winding road in the virtual reality environment (c). The individual in the 3a is one of the co-authors who provided informed consent to publish the image.

Figure 4 In reality, driving a curved road causes the inclination of the motorcycle body. To simulate this motion in the virtual environment, we used equation (1) below:

$$\theta ={\tan }^{-1}\left(\frac{{v}^{2}}{ag}\right)$$
(1)

where g = the gravity acceleration, v = the speed of the motorcycle, a = the radius of motorcycle displacement and θ = the inclination of the motorcycle body relative to the road.

Engine sounds were created using a motorcycle engine sound synthesis software (Sound Design Lab, LLC., Japan) and were presented by earphones. The speed of the motorcycle changed from 10 to 50 km/h. The pitch of the engine sound was modulated with the driving speed, but the amplitude of the sound did not change. The vibration device (Vt7) attached under the seat could present vibrations with frequencies from 16 to 150 Hz. The frequency of the vibration was modulated based on the frequency of the sound. Figure 5 shows the speed of the motorcycle (top panel), the accelerations measured on the seat (second panel) and forehead (third panel) and the frequency spectrogram of the presented engine sound (bottom panel) in the VR environment. The mean sound pressure level (Lp) of the engine sound in 30 seconds at the earphone was 91.3 dB in the AV and audio-only groups, its loudness level (LA) was 96.8 dB, which was measured by a sound level meter (6226, ACO, Co., Ltd, Japan), the root mean square (RMS) of the total accelerations (RMS of the vector sum of the three components) in 30 seconds was 4.56 (Max = 13.02) m/s2 on the seat and 0.15 (Max = 0.25) m/s2 on the forehead. The acceleration of the seat was measured without a participant sitting on it, and the acceleration on the forehead was measured with a representative participant sitting on the seat. The participants wore earphones, except in the vibration-only group, where they wore a noise cancelling headphone (QuietComfort 35, Bose, Inc., USA) to reduce possible sounds induced by vibration. The RMS of the acceleration values was similar to that in the study by D’Amour, et al.31 (Seat = 4.25 m/s2, Head = 0.10 m/s2).

Figure 5
figure5

The speed of motorcycle in the virtual reality environment, frequency spectrogram of the presented engine sound and accelerations measured on the seat and head-mounted display.

First, we evaluated the proneness of motion sickness using the MSSQ44 based on the experience of motion sickness. The possible maximum MSSQ score is 427.68. The mean MSSQ scores were 22.9 (SD = 20.1), 21.5 (SD = 27.0) and 77.2 (SD = 38.6) in the studies by Dennison and D’Zmura40, Bos, et al.42 and Bos41, respectively. In the present study, the average MSSQ scores were 77.2 (SD = 30.2), 80.5 (SD = 25.7), 70.5 (SD = 18.9) and 91.3 (SD = 32.9) in the AV, no-AV, audio-only and vibration-only groups, respectively.

Before watching the visual stimulus through the HMD, the participants answered the SSQ to confirm that they did not report symptoms prior to exposure. In the main observation phase, participants watched the presented stimulus for 5 minutes and were instructed not to move the handlebar and not to lean their trunk. When the participants wore the HMD, the virtual motorcycle waited to start with idling engine. The motorcycle was kept driving from the start to the end of the observation phase, and engine sounds and/or vibrations were provided to the participants according to the experimental group type. Sounds and vibration were not provided to the no-AV group. The participants answered the FMS by saying the number (0–20) when they found the word “Answer” in the centre of the screen. The FMS was evaluated every 1 minute, and therefore five times in total. The display of “Answer” disappeared after 3 seconds from the presentation, and all participants appropriately answered FMS without missing them. Immediately after the 5th evaluation of FMS, the visual stimulus was stopped, and the participants took off the HMD and answered SSQ. They then took a short rest and answered subjective reality and presence (11-point scale from 0 to 10) of the visual stimulus presented in the HMD, in addition to the various experience, to comprehensively identify the factors of VIMS. The questionnaire items (realism: the feeling of being there in the virtual scene that you experienced; presence: the feeling of how real the virtual scene that you experienced was) were created based on previous studies23,31.

We evaluated motion sickness using two different questionnaires. SSQ, a standardised questionnaire that covers extensive symptoms of VIMS35, consists of 16 items with 4-point scales (0 [absent] to 3 [severe]), and the possible maximum total score (SSQ-TS) is 235.6245. Although SSQ has three subscales (N: nausea; O: oculomotor; D: disorientation), we analysed only the total score to simplify the analysis. The mean SSQ scores assessed before the experiment were 11.4 (SD = 11.7), 9.7 (SD = 14.1), 13.5 (SD = 13.3) and 6.0 (SD = 10.0) in the AV, no-AV, audio-only and vibration-only groups, respectively. We did not calculate the difference between the scores at the baseline and after the observation phase. VIMS during watching the driving scene was evaluated by FMS. FMS focuses on nausea, general discomfort and stomach problems but asks participants to ignore other symptoms such as nervousness, cold sweat, boredom, fatigue and drowsiness9, which was instructed to the participants prior to the observation phase. Participants answered the degree of VIMS by verbally answering the number (0 [no sickness at all] to 20 [frank sickness]). The correlation coefficients between the SSQ total score and FMS score were r = 0.79 in the study by Keshavarz and Hecht9, r = 0.71 in the study by Keshavarz and Hecht30 and r > 0.61 in the study by D’Amour, et al.31.

The participants also rated the realism and presence of the visual stimulus (0–10). Before the rating, the participants were explained that realism means the feeling that the virtual environment is a part of real life, and presence means the feeling of being there in the virtual scene31,46. Furthermore, they answered the frequency of their daily use of a car or motorcycle, and they stated whether they had ever experienced a VR device such as Oculus Rift.

The participants could quit the experiment whenever they wanted and by whatever reasons to avoid a severe motion sickness that affects daily activities after the experiment. Six participants quit watching visual stimulus; the number of dropouts was two, one and three in the no-AV, audio-only and vibration-only groups, respectively. Their data were not excluded from the analysis, and the FMS that could not be answered were scored as the FMS rating immediately before the quitting, as in the study by D’Amour, et al.31.

Data availability

The datasets generated and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. 1.

    Yoshimoto, K. & Suetomi, T. The history of research and development of driving simulators in Japan. Journal of mechanical systems for transportation and logistics 1, 159–169 (2008).

    ADS  Article  Google Scholar 

  2. 2.

    Meuleners, L. & Fraser, M. A validation study of driving errors using a driving simulator. Transportation research part F: traffic psychology and behaviour 29, 14–21 (2015).

    Article  Google Scholar 

  3. 3.

    Dziuda, Ł., Biernacki, M. P., Baran, P. M. & Truszczyński, O. E. The effects of simulated fog and motion on simulator sickness in a driving simulator and the duration of after-effects. Appl. Ergon. 45, 406–412 (2014).

    Article  Google Scholar 

  4. 4.

    Helland, A. et al. Driving simulator sickness: impact on driving performance, influence of blood alcohol concentration, and effect of repeated simulator exposures. Accid. Anal. Prev. 94, 180–187 (2016).

    Article  Google Scholar 

  5. 5.

    Keshavarz, B., Ramkhalawansingh, R., Haycock, B., Shahab, S. & Campos, J. Comparing simulator sickness in younger and older adults during simulated driving under different multisensory conditions. Transportation research part F: traffic psychology and behaviour 54, 47–62 (2018).

    Article  Google Scholar 

  6. 6.

    Diels, C. & Howarth, P. A. Frequency characteristics of visually induced motion sickness. Hum. Factors 55, 595–604 (2013).

    Article  Google Scholar 

  7. 7.

    Keshavarz, B., Riecke, B. E., Hettinger, L. J. & Campos, J. L. Vection and visually induced motion sickness: how are they related? Front. Psychol. 6, 472 (2015).

    PubMed  PubMed Central  Google Scholar 

  8. 8.

    Rebenitsch, L. & Owen, C. Review on cybersickness in applications and visual displays. Virtual Reality 20, 101–125 (2016).

    Article  Google Scholar 

  9. 9.

    Keshavarz, B. & Hecht, H. Validating an efficient method to quantify motion sickness. Hum. Factors 53, 415–426 (2011).

    Article  Google Scholar 

  10. 10.

    Reinhard, R. et al. The best way to assess visually induced motion sickness in a fixed-base driving simulator. Transportation research part F: traffic psychology and behaviour 48, 74–88 (2017).

    Article  Google Scholar 

  11. 11.

    Oman, C. M. Motion sickness: a synthesis and evaluation of the sensory conflict theory. Can. J. Physiol. Pharmacol. 68, 294–303 (1990).

    CAS  Article  Google Scholar 

  12. 12.

    LaViola, J. J. Jr A discussion of cybersickness in virtual environments. ACM SIGCHI Bulletin 32, 47–56 (2000).

    Article  Google Scholar 

  13. 13.

    Johnson, D. M. Introduction to and review of simulator sickness research. (ARMY RESEARCH INST FIELD UNIT FORT RUCKER AL, 2005).

  14. 14.

    Mullen, N. W., Weaver, B., Riendeau, J. A., Morrison, L. E. & Bédard, M. Driving performance and susceptibility to simulator sickness: Are they related? Am. J. Occup. Ther. 64, 288–295 (2010).

    Article  Google Scholar 

  15. 15.

    Munafo, J., Diedrick, M. & Stoffregen, T. A. The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects. Exp. Brain Res. 235, 889–901 (2017).

    Article  Google Scholar 

  16. 16.

    Kennedy, R. S., Drexler, J. & Kennedy, R. C. Research in visually induced motion sickness. Appl. Ergon. 41, 494–503 (2010).

    Article  Google Scholar 

  17. 17.

    Fernandes, A. S. & Feiner, S. K. Combating VR sickness through subtle dynamic field-of-view modification. in 2016 IEEE Symposium on 3D User Interfaces (3DUI). 201-210 (IEEE) (2016).

  18. 18.

    Reason, J. T. Motion sickness adaptation: a neural mismatch model. J. R. Soc. Med. 71, 819–829 (1978).

    CAS  Article  Google Scholar 

  19. 19.

    Draper, M. H., Viirre, E. S., Furness, T. A. & Gawron, V. J. Effects of image scale and system time delay on simulator sickness within head-coupled virtual environments. Hum. Factors 43, 129–146 (2001).

    CAS  Article  Google Scholar 

  20. 20.

    Keshavarz, B. & Hecht, H. Stereoscopic viewing enhances visually induced motion sickness but sound does not. Presence: Teleoperators and Virtual Environments 21, 213–228 (2012).

    Article  Google Scholar 

  21. 21.

    Kolasinski, E. M. Simulator Sickness in Virtual Environments. (Army research Inst for the behavioral and social sciences Alexandria VA, 1995).

  22. 22.

    Brooks, J. O. et al. Simulator sickness during driving simulation studies. Accid. Anal. Prev. 42, 788–796 (2010).

    Article  Google Scholar 

  23. 23.

    Milleville-Pennel, I. & Charron, C. Do mental workload and presence experienced when driving a real car predispose drivers to simulator sickness? An exploratory study. Accid. Anal. Prev. 74, 192–202 (2015).

    Article  Google Scholar 

  24. 24.

    Altena, E. et al. How sleep problems contribute to simulator sickness: Preliminary results from a realistic driving scenario. J. Sleep Res., e12677 (2018).

  25. 25.

    Domeyer, J. E., Cassavaugh, N. D. & Backs, R. W. The use of adaptation to reduce simulator sickness in driving assessment and research. Accid. Anal. Prev. 53, 127–132 (2013).

    Article  Google Scholar 

  26. 26.

    Heutink, J., Broekman, M., Brookhuis, K. A., Melis-Dankers, B. J. & Cordes, C. The effects of habituation and adding a rest-frame on experienced simulator sickness in an advanced mobility scooter driving simulator. Ergonomics, 1-11 (2018).

  27. 27.

    Duh, H.-L., Lin, J., Kenyon, R. V., Parker, D. E. & Furness, T. A. Effects of field of view on balance in an immersive environment. in Proceedings IEEE Virtual Reality 2001. 235-240 (IEEE) (2001).

  28. 28.

    Whittinghill, D. M., Ziegler, B., Case, T. & Moore, B. Nasum virtualis: A simple technique for reducing simulator sickness. in Games Developers Conference (GDC). (2015).

  29. 29.

    Weech, S., Moon, J. & Troje, N. F. Influence of bone-conducted vibration on simulator sickness in virtual reality. PLoS One 13, e0194137 (2018).

    Article  Google Scholar 

  30. 30.

    Keshavarz, B. & Hecht, H. Pleasant music as a countermeasure against visually induced motion sickness. Appl. Ergon. 45, 521–527 (2014).

    Article  Google Scholar 

  31. 31.

    D’Amour, S., Bos, J. E. & Keshavarz, B. The efficacy of airflow and seat vibration on reducing visually induced motion sickness. Exp. Brain Res. 235, 2811–2820 (2017).

    Article  Google Scholar 

  32. 32.

    Keshavarz, B. & Hecht, H. Visually induced motion sickness and presence in videogames: The role of sound. in Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 1763-1767 (SAGE Publications Sage CA: Los Angeles, CA) (2012).

  33. 33.

    Sharples, S., Cobb, S., Moody, A. & Wilson, J. R. Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems. Displays 29, 58–69 (2008).

    Article  Google Scholar 

  34. 34.

    Llorach, G., Evans, A. & Blat, J. Simulator sickness and presence using HMDs: comparing use of a game controller and a position estimation system. in Proceedings of the 20th ACM Symposium on Virtual Reality Software and Technology. 137-140 (ACM) (2014).

  35. 35.

    Kennedy, R. S., Lane, N. E., Berbaum, K. S. & Lilienthal, M. G. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology 3, 203–220 (1993).

    Article  Google Scholar 

  36. 36.

    Tanaka, N. & Takagi, H. Virtual reality environment design of managing both presence and virtual reality sickness. J. Physiol. Anthropol. Appl. Human Sci. 23, 313–317 (2004).

    Article  Google Scholar 

  37. 37.

    Jaeger, B. K. & Mourant, R. R. Comparison of simulator sickness using static and dynamic walking simulators. in Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 1896–1900 (SAGE Publications Sage CA: Los Angeles, CA) (2001).

  38. 38.

    Weech, S., Kenny, S. & Barnett-Cowan, M. Presence and cybersickness in virtual reality are negatively related: a review. Front. Psychol. 10, 158 (2019).

    Article  Google Scholar 

  39. 39.

    Nichols, S., Haldane, C. & Wilson, J. R. Measurement of presence and its consequences in virtual environments. Int. J. Hum. Comput. Stud. 52, 471–491 (2000).

    Article  Google Scholar 

  40. 40.

    Dennison, M. S. & D’Zmura, M. Cybersickness without the wobble: experimental results speak against postural instability theory. Appl. Ergon. 58, 215–223 (2017).

    Article  Google Scholar 

  41. 41.

    Bos, J. E. Less sickness with more motion and/or mental distraction. J. Vestib. Res. 25, 23–33 (2015).

    Article  Google Scholar 

  42. 42.

    Bos, J. E., de Vries, S. C., van Emmerik, M. L. & Groen, E. L. The effect of internal and external fields of view on visually induced motion sickness. Appl. Ergon. 41, 516–521 (2010).

    Article  Google Scholar 

  43. 43.

    Golding, J. F. & Stott, J. R. Objective and subjective time courses of recovery from motion sickness assessed by repeated motion challenges. J. Vestib. Res. 7, 421–428 (1997).

    CAS  Article  Google Scholar 

  44. 44.

    Golding, J. F. Motion sickness susceptibility questionnaire revised and its relationship to other forms of sickness. Brain Res. Bull. 47, 507–516 (1998).

    CAS  Article  Google Scholar 

  45. 45.

    Biernacki, M. P. & Kennedy, R. S. & Dziuda, Ł. [Simulator sickness and its measurement with Simulator Sickness Questionnaire (SSQ)]. Med. Pr. 67, 545–555 (2016).

    Article  PubMed  Google Scholar 

  46. 46.

    Heeter, C. Being there: The subjective experience of presence. Presence: Teleoperators & Virtual Environments 1, 262–271 (1992).

    Article  Google Scholar 

Download references

Acknowledgements

This study was partially supported by JSPS KAKENHI (19H01087). The authors thank T. Ueda for their assistance with conducting the experiments.

Author information

Affiliations

Authors

Contributions

Y.I. and M.Miy. supervised the project. Y.S., Y.I., T.M., M.Mik., T.K. and M.Miy. designed the study. Y.S. and T.M. programmed the experimental stimuli. Y.S., M.H., K.A. and T.M. conducted the experiments. Y.S. and Y.I. analysed the data. All authors discussed the results. Y.S. and Y.I. wrote the manuscript.

Corresponding author

Correspondence to Yoshihiro Itaguchi.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Sawada, Y., Itaguchi, Y., Hayashi, M. et al. Effects of synchronised engine sound and vibration presentation on visually induced motion sickness. Sci Rep 10, 7553 (2020). https://doi.org/10.1038/s41598-020-64302-y

Download citation

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.