Virtual experiences have become increasingly immersive owing to advances in virtual reality (VR) and augmented reality (AR). At present, VR glasses or head-mounted displays are used to create realistic visual and auditory sensations, enabling individuals to virtually experience various environments, such as standing atop a mountain in Italy. The applications of VR and AR extend across various fields, encompassing entertainment, rehabilitation, education and communication.

Credit: Stefania Pelfini, La Waziya Photography / Getty Images

So far, VR and AR devices have mainly focused on simulating visual and auditory experiences, while the integration of touch perception, or haptics, has been overlooked due to methodological challenges. However, touch, which provides sensory feedback about temperature, texture and rigidity, has a crucial role in creating a truly immersive virtual environment. Unlike vision and hearing, touch directly involves the physical interaction between a person and the virtual elements, making it a key aspect of a lifelike virtual experience. To illustrate this, imagine encountering a snake in virtual or augmented reality. Seeing a lifelike representation of a snake close-up and hearing it hiss could already be intense, but the corporeal sensation of touching it would be a different level of experience. Depending on the context, touching a virtual snake could even be beneficial in helping one to overcome the fear of snakes.

Incorporating realistic touch perception is essential to enhancing the authenticity and depth of virtual experiences. As René Descartes wrote, “Of all our senses, touch is the one considered least deceptive and the most secure”1. Indeed, ‘hands-on’ experiments have long been considered essential in science education. As technology has advanced, haptic interfaces can be used to make the most of students’ ability to learn from tactile experiences; for instance, biology students can move and rotate 3D models of molecules to learn about complex interactions such as between proteins and ligands2.

Active touch has an important role in action. The hand, for example, is equipped both for sending sensory information to the brain and for making movements. The inclusion of touch within the technology for simulating virtual worlds promises to add an extra level of agency and psychological realism to virtual experiences. In a research Article in this issue, Zhang et al. contribute to this area by introducing an origami-based haptics device, synchronized with visual representations, that enables humans to actively engage with a wide range of tactile interactions with virtual objects. The authors emphasize the importance of enabling two types of perception that are essential for people’ s everyday interactions with the physical world: the feeling of hardness (or softness) of an object and the feeling of breaking or falling.

The authors show that such positive and negative stiffness properties, respectively, can be emulated with origami-type materials. They design materials with pre-defined folding patterns that determine the material’s tunable stiffness properties. The range, from positive to negative, can be adjusted with small motors integrated in the material that change the folding angle. As force is actively applied on the device by human touch, different sensations can be mimicked, such as those associated with touching hard or soft objects — concrete or grass, for example, or those associated with breaking or falling, like crushing ice.

Zhang et al.demonstrate the device in two VR applications involving in-hand haptic feedback and haptic foot feedback. The authors present their approach as a first-person, human-triggered and active method that allows users to engage in mechanical interactions with different perceptions of stiffness. This approach is in contrast to previous haptics research that was machine-triggered and passive, resulting in sensations that differed from real-world experiences. The authors’ haptic device thus introduces a new interactive method for VR, enhancing users' sense of immersion. Potential applications of this technology include rehabilitation robotics, in which a haptics device could help patients who have had a stroke to hold objects of varying stiffness; and treatment for fear of heights, in which combining haptics with VR exposure therapy could offer more effective therapeutic options.

The devices presented by Zhang et al. were designed with fixed shapes, but as the authors note, origami-inspired approaches lend themselves well to creating a range of morphologies. An interesting direction of research is the combination of haptic feedback with origami-based robotic devices that can adapt their morphology, for a more versatile range of touch sensations3. Such research promises to further integrate sensation and perception with agency and actions, leading to more immersive and practical virtual experiences.