There has been substantial progress made in the development of autonomous vehicles (AVs) within the past two decades, but the dream of having fully driverless cars has not yet become a reality. As a matter of fact, the safety performance of AVs has not reached the same level as that of cars driven by humans, in part because safety validation in naturalistic driving environments (NDEs) is inefficient and expensive. While artificial intelligence (AI) systems can be used for training in safety-critical scenarios, looking for these events in the training data is like searching for a needle in a haystack: there is a relatively small number of safety-critical events compared to a large amount of non-safety-critical data, given that the occurrence probability of the former is rare. As such, there is not sufficient data for an AI system to learn how to react under unsafe conditions.
In a recent work, Henry X. Liu and colleagues developed a test environment that makes use of a dense deep-reinforcement-learning (D2RL) approach to improve the safety validation of AVs. In a nutshell, the D2RL approach edits the Markov decision process by removing the non-safety-critical states and reconnecting the safety-critical ones, meaning that the neural networks are trained on the critical states only. In this case, the reinforcement learning has a ‘dense reward’: the reward function gives value to most of the transitions, and thus the system receives feedback at almost every time step, becoming better at distinguishing between safe and non-safe situations. In addition to conducting simulation experiments, the authors also conducted field experiments in physical test tracks. Of particular interest, the authors developed an augmented-reality testing platform that made it possible for a real AV in the test track to interact with virtual ones for testing. The experiments demonstrated that the D2RL approach is more efficient than the NDE one, meaning that it requires a substantially smaller number of tests — sometimes on the order of 105 more efficient than NDE — both in simulated and real scenarios. Overall, the proposed framework has the potential to accelerate training and testing of AVs, hopefully making us one step closer to the dream of having fully driverless cars.
This is a preview of subscription content, access via your institution