Should a robotic nurse jab a person’s arm as quickly and precisely as possible? Or should it contrast the precision of a rapid needle insertion with the heavy pressure of a cotton swab on the wound to lessen our discomfort and fear of the needle, as a real nurse would?
We applaud when machines beat us at board games or perform backflips. But people do more than calculate and jump. We flirt, paint and dance. We communicate through motion in ways that have not yet been fully described by the laws of mechanics.
No robot really moves like we do. A human backflip leverages an articulated spine, heavy hips thrusting through the air whipped by floating vertebrae. By contrast, the backflipping Atlas robot built by Massachusetts firm Boston Dynamics begins its somersault at the ankles, tipping its hulking middle off a wooden block to induce a fall. The only similarities are a start, rotation and finish.
Scientists have quantified the wiggle of the nematode worm Caenorhabditis elegans using 100 numbers1, modelling it in more detail than they have for human postures. Microsoft’s discontinued Kinect controller for Xbox computer games tracked only about two dozen aspects of a player’s movements.
By comparison, human motions are much more intricate. Whereas the Atlas robot has 28 coupled moving parts, animators use tens of thousands of parameters to simulate how people breathe2. We respond differently to an outstretched hand depending on whether it has tensed or soft forearm muscles. And meeting someone who is holding their breath is less welcoming than being greeted with an exhale. But roboticists often view such complexity as unnecessary. They favour a minimal aesthetic, designing robots that outperform nature only in force, speed and precision. Although the vision for most robots is that they will blend into our lives as caregivers or guides, they struggle to interact with humans or to operate in changing environments.
Roboticists need to learn more about how humans move and the ways in which we physically outpace our artificial counterparts. This means studying body language as well as anatomy, in collaboration with choreographers and dancers, whose qualitative and embodied expertise is a wealth of overlooked knowledge. Entrenched attitudes in the robotics field — that electronic devices are inert and that robots should be optimized, for instance — also need to be overturned.
In the future, robots will be pervasive. Everything will move — visibly, audibly and in a tactile way. Chairs will roll across a room to catch us as we sit; exercise equipment will move our bodies to aid recovery from injury; intelligent prosthetics will replace lost limbs. But how exactly should these devices move and behave?
Standing in a factory full of noisy, whizzing robots is exhausting. It is a physical relief to walk outside into natural light, surrounded by leaves and grasses blowing in the wind. Robotics engineers will need to decide whether living in a mechanized landscape will feel like hurrying through a busy road junction or sitting calmly in a forest.
People have evolved to respond to their environments. When developing animals are deprived of a stimulus, such as horizontal or vertical lines, they lose the ability to see those features3. A person viewing a cityscape will note that it has different statistical properties from a forest: the hard lines of buildings are more ordered and predictable than chaotic tangles of twigs, brambles and leaves4.
Humans will adapt to a cacophony of artificial motion regardless. But technology needs to be designed carefully to protect our health and well-being. Look around at the people hunched over their smartphones, poking at tiny keys. This crouching is a by-product, not part of the smartphone’s design.
As robots become integral parts of the built environment, they will affect our lives as buildings do. Buildings are designed to do more than minimize cost and maximize durability. Architects consider the cultural context of a structure, the flow of activity within it and its light, form and colour5. Variability and expression are important to humans.
Humans are never still. When we breathe, our lungs operate in three dimensions: we can expand them up and down, left and right, and push out the chest to the front and back. Day to day, we might not need to use this full capacity. But if we get stuck under a fallen tree or need to release a hearty belly laugh, we are glad to have this flexibility.
And animals evolve to suit their environments. In 2014, biologist Deborah Gordon at Stanford University in California and her colleagues sent a species of pavement ant (Tetramorium caespitum) to the International Space Station. Despite the station’s low gravity, the ants were able to climb, finding new ways to cling to surfaces with their feet6. How could a system that is optimized to crawl end up climbing? How can a system that is honed to gravity’s acceleration at the surface of Earth adapt to its absence?
Knowing the answers to questions such as these could, for instance, help autonomous cars to adapt to different environments — for example, switching from the broad roadways of Phoenix, Arizona, to New Delhi’s narrow streets, where signage, convention and culture differ. Yet little research has been done on how the motion of the environment changes the sensory statistics and experiences of humans.
Robots should be able to move in myriad ways, but their mechanics have changed little. Atlas’s 28 moving parts are much like those of earlier humanoids — its legs comprise three rigid links connected by a handful of rotary actuators, which allow twisting of each link, but no stretching (unlike human joints). Increased computing power, more than any leap in mechanical design, makes its backflip possible.
Researchers need to understand why different forms of locomotion evolved. Long-held assumptions, such as the need for energy efficiency7, have already been overturned. For example, a mechanical ankle brace can improve the metabolic efficiency of human walking, implying that walking is inefficient8. But variation of movement is important, too: such an ankle brace holds you back if you try to skip, gallop or skitter. Similarly, legged robots struggle to deploy different gaits, just as roboticists struggle to enumerate them.
Humans need to be able to recognize what a robot will do. Roboticist Anca Dragan at the University of California, Berkeley, has argued that robots need to be able to communicate their intent to humans by taking an exaggerated path through space that expends extra energy9. But an element of surprise is sometimes desirable, such as when taking a penalty kick or evading an enemy in combat.
Algorithms to control robots should include individual, varied or unexpected responses. Adding a degree of ‘noise’ into the positions of robotic arms can make it easier for humans to follow and replicate their movements10. But few programmers want to add noise to their codes.
The latest generation of soft robots — constructed from elastic, flexible and compliant materials — will increase the range of movements and uses that are possible. Examples include a soft robot that mimics an octopus and various soft grippers that gently conform to the irregular surface of an object being held. However, these, too, will be limited if the goal remains only to rigidly control the positions, velocities and speeds of actuators.
Widen the repertoire
The diversity of robot movements needs to be expanded to include variable and complex motions. This area is receiving too little attention for two reasons.
First, engineers often fail to see the synergy between robots and computers. They tend to view a robot as moving and a computer as inanimate. Yet this is an illusion — at an atomic level, a smartphone is a lively orchestra of complex, coordinated movement, with electrons in transistors switching back and forth as information and commands stream through.
Second, researchers tend to see robots as optimal reproductions of natural systems. Yet they forget that electronic devices are imbued with human traits and values. The motion of transistor networks is not ‘optimal’, but has been engineered to get the job done. Each computer system has a different architecture, ethos and approach to trade-offs — the Windows interface allows users to customize many more settings than does the Macintosh operating system, for example.
Roboticists are also failing to explore more expressive high-level architectures and programming languages, such as Scratch, which computer scientists use to control trillions of transistors without knowing exactly what each is doing. Whereas an assembly language allows the person to specify the action of particular transistors, visual languages such as Scratch give users predetermined transistor behaviour inside an easier-to-use packaging. In robotics, by contrast, we continue to refine and obsess over the precise actions of each and every joint.
Meanwhile, insights from experts in human motion are missing. Dancers and choreographers are skilled at conveying intent and meaning through body movements, even in new ways. For example, in the late 1920s, choreographer Martha Graham rejected the flitting birds and ethereal fairies of ballet and developed a language of visceral postures — crunching her stomach to express struggle as well as ecstasy, for example.
And dancers understand the physical processes through which the human body contorts. For example, choreographer Gregory Catellier, a professor of dance at Emory University in Atlanta, Georgia, once asked me how a small humanoid robot I was operating moved. I rushed to explain the Wi-Fi connection, the servo motors and the positional encoders. He cut me off: “No, no, how it is even possible for a humanoid shape to move without a core?” He saw that the device had no spine. I recalled the dance classes I had taken with him: learning how to contract the spine to execute a spin more quickly, or stabilize it sideways to perform evenly timed cartwheels.
Robotics researchers need to investigate why the bodies of people and other animals feature so much variation and complexity. Humans have 33 vertebrae, corresponding to roughly the same number of moving parts as an Atlas robot, yet we can do so much more. None of our joints is a perfect hinge and our arms rarely make the exact same motion twice. What advantages do they offer? Natural environments change rapidly and living things can adapt, whereas artificial environments for robots are held steady and constant. What is the relationship between the environment and the agents that move in it?
New quantitative models of natural motion should be developed, along with design goals for robotic motion. With this in mind, studying the simplest animals could be fruitful. Roboticist Kirstin Petersen at Cornell University in Ithaca, New York, investigates termites and spiders to find better ways to exploit modularity, complexity and soft materials in artificial systems. Robotics researcher Zeynep Temel at Carnegie Mellon University in Pittsburgh, Pennsylvania, has found that even tiny natural mechanical structures, such as the flagella of bacteria, are challenging to replicate11.
Collaborations must encompass a much broader range of disciplines. For example, roboticist Heather Knight at Oregon State University in Corvallis works with actors to capture variations in their movement to drive expressive robotic systems. At Emory University, biomedical engineer Lena Ting and movement scientist Madeleine Hackney are working across science and arts disciplines to understand gait and rehabilitation, funded by the US National Science Foundation’s Mind, Machine and Motor Nexus programme. But these cross-cutting efforts will fail unless they embrace an equal partnership with qualitative, embodied and artistic practice. We need to honour the expertise of dancers and movement analysts, and craft canonical problems in expression together.
Roboticists need to get up out of their chairs to try out, discuss and refine the movements they are designing into machines.
Nature 565, 422-424 (2019)
Competing Financial Interests
A.L. is a co-founder and part-owner of AE Machines, Inc. and CAAII, LLC.