To be useful in a variety of daily tasks, robots must be able to interact physically with humans and infer how to be most helpful. A new theory for interactive robot control allows a robot to learn when to assist or challenge a human during reaching movements.
What if robots could physically interact with humans, work alongside us and help us perform daily tasks? Such robots could act as a helping hand by assisting with difficult tasks such as lifting heavy objects, or act as a personal coach by providing moderate physical challenges to exercise more effectively. Such adaptive robots would sense forces and movement during physical interaction, and infer our goals, motor capability and level of effort in order to generate the best interactive behaviour1. Although we are a long way off from seeing robots helping out routinely in our offices and gyms, new research from Y. Li and colleagues2, published in Nature Machine Intelligence, yields insight into how robots — through physical interaction — can understand our actions and adapt their behaviour to help us reach our goals.
Li et al. tackle human–robot interaction using a game-theoretic robot control framework. In game theory, multiple players interact in a game, competing or collaborating to complete a task. Each player has their own strategy — how they choose their next action based on the current state of the game — and all players try to optimize their performance, while assuming their opponents will also play optimally3. If you have played chess against a computer, then you have probably interacted with a machine using a competitive game-theory controller. Here, the computer evaluates the state of the game (the arrangement of chess pieces on the board) and selects the move that will maximize its chances of capturing your king, while also considering moves that you might make in the future.
In the study by Li et al., the game is a reaching task: the human holds a handle on the end effector of the robot and moves to a prespecified goal location within a plane. The state of the game is the error: how far the human hand is from the goal location, and how fast it is moving. The human can apply forces directly to the handle to move it towards the goal, and the robot can actuate its joints to move the handle. However, because the human and the robot act simultaneously and are coupled through physical contact, forces from the human can cause the robot to deviate from its desired motion, and vice versa, creating a difference between the actual motion of the handle and the motion planned by either the human or the robot.
The authors’ major innovation is in using game theory to determine how the robot responds to the effects of interacting with a human. In a typical physical robot–human interaction, robots have overpowered the human to reduce error4, which could accidentally harm the human. Or they have allowed the human to move the robot easily5, which could increase error in the task. The authors introduce a new approach: the robot uses the difference between its expected and actual motions to estimate the human’s strategy — that is, how the human uses errors in the task to generate new actions. By estimating the human’s strategy, the robot can change its own strategy in response. For example, if the human’s strategy is insufficient to complete the task, the robot can increase its effort to help them.
By changing how the robot views errors in the task, the robot of Li et al. provides assistance as needed, which has been accomplished previously6,7, and also challenges and trains humans (Fig. 1). Less-than-needed assistance can keep humans engaged and prevent them from slacking off8. In this case, the robot tolerates some error, requiring the human to increase effort to complete the task. The robot can also challenge the human by increasing errors — that is, moving the handle away from the goal, in a strategy similar to error augmentation9.
The authors tested their game-theoretic framework for physical human–robot interaction in both simulations and experiments with human subjects. In simulation, the authors tested the limits of their theory. They showed that the robot can adapt to situations when the human’s strategy changes slowly, as if the human was recovering strength, and when the human’s strategy is highly variable, as may be the case following injury, in which the human does not always make steady progress. In human experiments, they showed that the robot can aid healthy individuals in a reaching task by increasing assistance when the user is too weak to complete the task. Interestingly, the robot also automatically transitions from assistive to competitive behaviours as the human improves in the task. This behaviour allows the robot to help the human when they cannot complete the task, and then to challenge them to improve when they can.
The methods presented by Li et al. represent a significant advance in physical human–robot interaction. The game-theoretic framework allows the robot to smoothly transition among a variety of interactive behaviours by estimating a human’s strategy from errors in movement. A single theoretical framework encompassing numerous behaviours allows the robot to respond to users in a flexible way, creating control strategies and behaviours tailored to each individual that change with the user’s capabilities. The use of optimal control also allows for formal and rigorous analysis of conditions that guarantee stability, which is essential to the safety of the human–robot interaction.
This study helps to lay a foundation for both theoretical and experimental work in human–robot interaction with physical contact. Future studies may extend the framework to include more than two agents, such as a robot mediating physical interactions among multiple individuals, or teams of robots helping humans with dangerous or difficult tasks. Future generalization of the theory to systems with nonlinear or unknown dynamics would be useful for robots that interact with multiple joints of a human, such as robotic gait trainers and exoskeletons.
The results of this study demonstrate that physical interactions between robots and humans can help humans to not only achieve but also exceed goals. By interpreting our actions, these robots can adapt alongside us and be personalized to continually provide the assistance or challenge each person needs to improve. The robots of the future may lend us a helping hand and make our working lives easier, but they may also challenge us to be healthier individuals.