On-Chip LiDAR Technology Advances for Cars, Cell Phones

Mitch Leslie

Senior Technology Writer

When Google unveiled its self-driving car prototypes in 2010,the feature that drew the most comment—and some mockery—was the spinning, 13 kg cylinder light detection and ranging(LiDAR) sensors mounted on their roofs [1,2]. Even the head of Google’s autonomous car development program joked that the devices resembled the buckets used by fast food chain Kentucky Fried Chicken [3]. At 75 000 USD apiece, the sensors were also the cars’most expensive component[1].Aesthetics aside,the units’size, cost, and other limitations made them impractical for consumer automobiles (Fig. 1), let alone products such as cell phones and security devices that could also potentially incorporate LiDAR[3].

In just over a decade, however, LiDAR devices—and their cost—have shrunk dramatically. They are now small and cheap enough to be included in some consumer electronics, such as Apple iPhones [4]. Customers in some countries can also buy cars fitted with unobtrusive LiDAR devices—production models that offer the technology are already available from a few manufacturers,including China’s XPeng and the US electric vehicle company Lucid Motors [5,6]. General Motors and Volvo are among the old-school automakers that will start selling their own LiDAR-equipped cars within the next year or so [7,8].

Many of today’s LiDAR devices can be so much smaller and cheaper because they pack all the components needed to emit and detect light onto a single computer chip—although which, if any, of these devices qualify as solid state remains a matter of fierce debate in the industry [9]. Instead of a twirling array of lasers, these LiDAR devices rely on a diversity of other strategies to map their surroundings, including miniature mirrors and jiggling liquid crystals. Competition among these strategies is ratcheting up, and which ones will prove the most useful remains uncertain.The optimal way to squeeze all LiDAR components onto a single chip ‘‘is a huge research challenge and a holy grail for industry,” said Ming Wu, professor of electrical engineering and computer sciences at the University of California, Berkeley, CA,USA.

A LiDAR device typically sends out millions of pulses of laser light per second that bounce off anything in the immediate surroundings, such as pedestrians, cyclists, other cars, the road surface, buildings, and trees [10]. The sensor then detects the reflections and uses them to create a ‘‘point cloud,” a continually refreshing, three-dimensional reconstruction of the environment(Fig.2)[11].Autonomous vehicles also typically rely on other sensors such as radar and cameras. But LiDAR is attractive because it provides high spatial resolution, can measure the distance to and often the speed of objects,and can‘‘see”at night and in most inclement weather[12].Almost every company developing self-driving vehicles has embraced LiDAR—the glaring holdout is Tesla, whose chief executive officer Elon Musk once dubbed LiDAR a ‘‘fool’s errand” [12].

Fig. 1. (a) A prototype self-driving Volkswagen from Stanford University in Palo Alto,CA,USA and(b)a robot taxi from Waymo(Mountain View,CA,USA),Google’s self-driving car spinoff company, feature large rooftop LiDAR units that help them sense and navigate their surroundings. Credit: (a) Flickr (CC BY 2.0); (b) Waymo(public domain).

Fig. 2. As a self-driving car travels along, its LiDAR system creates a continually updating,three-dimensional map—a‘‘point cloud”—of its surroundings,like the one shown here. Credit: Velodyne LiDAR (public domain).

LiDAR’s capabilities also make it valuable for a variety of other devices, including drones, warehouse and delivery robots, crowd management systems that track passenger numbers in airports,security monitoring, and police speed guns. In iPhones, LiDAR helps focus the cameras,improves the accuracy of a tool that works like a tape measure, and allows augmented reality, in which users can layer a view of their surroundings with virtual information and objects such as game characters, store prices, or furniture [4].

On-chip LiDAR devices offer several advantages over the standard spinning units [11]. One of the main concerns about rooftop LiDAR units for self-driving vehicles, for example, is whether their moving parts can endure a decade or more of rugged driving [11].Because many on-chip LiDAR devices dispense with moving parts that can break or wear out, they could be more reliable [9]. And because the on-chip devices rely on the same complementary metal-oxide-semiconductor(CMOS)technology as computer chips,semiconductor foundries could potentially produce them cheaply and in the huge quantities needed for widespread adoption, Wu said.

An ideal chip-based LiDAR automotive device must meet exacting requirements, however, needing to detect an object with only 10% reflectivity at 200 m, provide a minimum field of view of 120° by 20°, and offer a 0.1° angular resolution [9]. And to top it off,each unit should ideally cost around 100 USD.To try to achieve such performance and cost standards,the more than a dozen companies and many academic researchers working to develop and improve on-chip LiDAR devices have taken a range of approaches[13].

One problem these engineers must solve is how to scan the surroundings to identify potential hazards. The revolving, roofmounted LiDAR devices may be unattractive, but they easily meet this function [14] and can capture a 360° view of a vehicle’s environment. But for on-chip devices, scanning ‘‘is the most difficult thing,” said Wu. Designs with so-called micro-electromechanical systems (MEMS) adjust the angles of minute mirrors to move lasers across the field of view[11].Among the companies betting on this strategy are the Munich, Germany-based startup Blickfeld and Israel-based Innoviz,which has a contract with Volkswagen [15,16]. However, critics knock the MEMS approach because it includes moving parts that may not stand up to heavy use [17].

Another scanning strategy involves optical phased arrays with rows of antennas, each of which emits pulses of laser light with a specific phase [11]. The pulses from the different antennas merge to form a coherent beam. To change the direction of the beam,the device alters the phase of the light produced by certain antennas, creating interference. Whether the interference is destructive or constructive determines which way the beam points. ‘‘The beauty is that the direction is controlled by changing the phase shifts electronically. There are no mechanical moving parts,” said Hossein Hashemi,professor of electrical and computer engineering at the University of Southern California in Los Angeles, CA, USA.

Using this scheme,Hashemi and his colleagues have developed several prototype devices, including one with 1024 antennas that offered one-dimensional scanning and fit onto a 5.7 mm × 6.4 mm chip [18]. Other researchers, including Wu and his colleagues,have worked on them as well[19].But the technology faces a big obstacle, Hashemi said. The elements in the array need to be half a wavelength apart, which for LiDAR is 0.50-0.75 m. ‘‘You need to cram all of the necessary components into that tiny space,” he said, which can limit the resolution of the devices [20].

In 2022, Wu and his colleagues described another type of onchip LiDAR, a focal plane switch device that resembles the sensor of a digital camera [20,21]. Each of the 16 384 pixels on the chip houses an antenna that emits light, which then passes through a lens that directs it to the target. The same antennas also sense returning light that has bounced off objects in the surroundings.The device moves the scanning beam around by steering laser light to a specific pattern of antennas. In the team’s tests, their LiDAR device,which fit onto a 10 mm×11 mm chip,achieved a horizontal field of view of 70° and a range resolution of 1.7 cm at 10 m[20,21].So far,no such focal plane switch LiDAR devices are in production, but a spinoff company Wu and colleagues have formed intends to commercialize the chip, he said.

These are just a few of the scanning strategies that companies and researchers are pursuing. And how to scan the environment is just one of the choices that engineers designing on-chip LiDAR devices must make. Another is how to determine the distance to objects in the environment. Most LiDAR devices use time-of-flight sensors to measure how long the laser pulses take to bounce back[14]. But frequency modulated continuous wave devices make the calculation by comparing the frequencies of the original and returning beams[22].LiDAR designs also differ in a variety of other ways, including the types of lasers they use [14].

While some experts have predicted a shakeout in which some current manufacturers disappear [23], the products that have reached the market so far indicate that on-chip LiDAR ‘‘is clearly maturing,” said Alex Coney, a consultant at the Cambridge, UKbased technology and product development firm The Technology Partnership. ‘‘But it still has a way to go.”

Cost is a key area where the industry needs to improve,he said.Companies are typically coy about the price of their LiDAR units,but the San Jose, CA, USA-based LiDAR developer Luminar has revealed that the sensor it is making for a new Volvo model due out in 2022 will cost in the range of 500-1000 USD—well above the 100 USD target for vehicle systems [24]. And Apple was able to include LiDAR in its iPhones because the buyers of its highend products are willing to pay for it[25].But in 2021,Apple rival Samsung, which makes an assortment of cheaper phones,announced it would no longer include LiDAR in its models, citing cost and lack of consumer interest [25]. However, Wu said he hopes that costs will decrease as companies take advantage of the inherent efficiencies associated with large-scale semiconductor manufacturing.