Over 1.2 million people die in road-related incidents every year1. Human error is a major contributing factor to fatalities and the long-term ambition is that autonomous vehicles may be a means to eradicate this issue. However, to suit widespread deployment, the required sensor technology needs to overcome many hurdles, including size, cost and, most importantly, effectiveness and safety.

Credit: Luminar

Various sensing approaches to generate real-time data of the vehicle’s surroundings exist, including those based on cameras, RADAR (typically operating at around 80 GHz (~4 mm wavelength) and LiDAR. At present, most self-driving car prototypes implement multiple schemes.

Most LiDAR schemes to date rely on mechanically rotating multiple optical transmitters and receivers on the vehicle’s roof. However, this approach imposes size and cost limitations, raises questions about long-term reliability of moving parts, and can limit imaging rates. Currently, there is a huge academic and industry effort to develop next-generation LiDAR systems that are faster and more reliable (both physically and in terms of object detection). Ideas include the use of microelectromechanical systems (MEMs) devices to implement scanning, optical phased arrays where beam steering is performed on-chip, and solutions that do away with scanning entirely using dispersed laser illumination to ‘flash’ an entire scene and capture the response with an arrayed imaging detector (for example, FLASH LiDAR).

The level of interest is perhaps well indicated by a recent Yole Développement report that predicts the current ~US$0.8 billion automotive market for LiDAR is likely to expand to US$5 billion by 2023, and to US$28 billion by 2032 (ref. 2).

Waymo (a subsidiary of Google’s parent company, Alphabet Inc.) has been leading the way in autonomous vehicle development with trials taking place in California. However, incumbent automakers not wishing to be left in the rear-view mirror have been busy making sensor-related acquisitions and investments in the area. General Motors recently purchased Strobe, a photonic start-up developing chip-scale low-cost LiDAR. Ford and Baidu have invested US$150 million in Velodyne (an early pioneer of LiDAR systems), while Volvo and Toyota have struck deals with Technologies Inc. Meanwhile, Ford’s subsidiary Argo AI acquired Princeton Lightwave and its LiDAR technology. Magna International and Innoviz Technologies have an arrangement to supply BMW with solid-state LiDAR.

In news earlier this year, US-based Blackmore Sensors and Analytics, a developer of LiDAR technology for autonomous vehicles, announced US$18 million in series B funding with investors including BMW i Ventures and Toyota AI Ventures. Stephen Crouch — chief technology officer at Blackmore — agrees that the usual concerns of range, size, speed and cost are hurdles for the technology, but told Nature Photonics that interference with sunlight or other LiDAR devices is another real issue that is a bit of a ‘dirty secret’ for mainstream LiDAR technology. However, he says that their continuous-wave coherent LiDAR system overcomes this issue via frequency modulation techniques. Another concern is that when objects are moving, the Doppler effect can result in blurred images. However, Blackmore’s approach makes it possible to separate out this relative motion, and obtain a clear image as well as velocity information at each image point — a key benefit.

“We are working with partners to leverage existing technology and manufacturing expertise in the optical fibre communications industry,” Crouch told Nature Photonics. “Basically, our systems take modern Internet hardware and ‘point it at the world’. From lasers to detectors, our photonic hardware is already mass produced in high-quality processes.”

Last year, Luminar — headed by then 22-year-old founder and CEO Austin Russell — came out from 5 years of stealth mode, announcing US$36 million in seed funding and a long-range high-resolution LiDAR system. One thing that is unique about Luminar’s approach is that the components are built in-house. Russell told Nature Photonics there are many requirements to make a commercially viable product that can achieve safer than human-level performance and that it may be difficult to achieve all of them with off-the-shelf components. Specific requirements are related to, for example, range, resolution, depth precision, frame rate, field of view, rain and snow capability, limited interference (with other LiDAR sensors of its type and sunlight), production scalability, eye safety, auto-grade reliability, low cost.

Luminar uses a single laser and single receiver to rapidly scan over the field of view, in contrast to other LiDAR systems, which can require 64 lasers and 64 receivers, or more. Russell told Nature Photonics that the main hurdles to the commercialization of LiDAR and autonomous vehicles aren’t cost, size, regulatory requirements, social acceptance, or any of the oft-mentioned roadblocks; it’s safety. He explained that it’s easy to get an autonomous vehicle to work 99% of the time, but tackling that last 1% of cases is challenging. According to Russell, even the best commercially deployed LiDAR systems cannot reliably ‘see’ today and this is the reason why all ‘autonomous’ vehicles at present require a backup driver to correct for the vehicle’s perception mistakes.

Moving forwards, we will no doubt see photonics continue to play a pivotal role in the development of autonomous vehicles. While the potential safety benefits of such vehicles are a key driver, economic benefits should not be underestimated as 3% of the gross domestic product is lost in less wealthy countries due to road traffic crashes1.