Why AI Might Train Your Self-Driving Car

Self-driving cars need teachers too, and artificial intelligence (AI) can efficiently teach those vehicles to avoid accidents—probably better than people. 

One of the best ways to send cars to Driver’s Ed is by using artificial intelligence. Tesla recently unveiled its new supercomputer that will be used to train the neural nets powering Tesla’s Autopilot and upcoming self-driving AI. And as cars become more autonomous, it turns out that they need a lot of training. 

"By exposing AI to data related to cars driving, AI can start to recognize patterns," Chris Nicholson, the CEO of Pathmind, a company that applies AI to industrial operations, said in an email interview. "Show it images, and it can learn what pedestrians look like. Show it sequences of actions on the road, and it can learn what leads to accidents, and how to avoid them."

"With the right data, AI can make very accurate predictions about what it's looking at," Nicholson added. "And what are the consequences of a given action, such as turning left or accelerating in the rain, might be."

Growing Number of AI Teachers

Tesla, Audi, Toyota, GM's Cruise—almost every single major automaker is using AI in some form to increase its self-driving capabilities, Nicholson said. And some non-automakers, such as Google's Waymo, are working with carmakers like Chrysler Fiat to develop and test self-driving AI. 

Andrej Karpathy, Tesla’s head of AI, recently unveiled the company’s latest supercomputer during a presentation at the 2021 Conference on Computer Vision and Pattern Recognition.

The cluster uses 720 nodes of 8x NVIDIA A100 Tensor Core GPUs (5,760 GPUs total) to achieve 1.8 exaflops of performance. Each exaflop is equal to 1 quintillion floating-point operations per second. 

"This is a really incredible supercomputer," Karpathy said, according to a news release. "I actually believe that in terms of flops, this is roughly the No. 5 supercomputer in the world."

A deep neural network observes and makes predictions while the car is driving without actually controlling the vehicle. The predictions are recorded, and any mistakes or misidentifications are logged. Tesla engineers then use these instances to create a training dataset of difficult and diverse scenarios to refine the neural network, 

The result is a collection of roughly 1 million 10-second clips recorded at 36 frames per second, totaling about 1.5 petabytes of data. The neural network is then run through these scenarios repeatedly until it operates without a mistake. Finally, it’s sent back to the vehicle and begins the process again.

Sending Cars Back to School

Using AI can also train cars faster than any human could, Aditya Pathak, a transportation expert for the professional services firm Cognizant, said in an email interview. 

"In the development process for autonomous vehicles, one of the critical steps is data annotation," he added. "In other words, how are people, places, and things tagged so that they can be recognized by vehicles?" 

Done manually, the process of looking through the data would be time-consuming and labor-intensive. "With AI and machine learning, the process is much faster and more efficient," Pathak said. 

AI has to teach self-driving cars how to operate in any kind of condition, Anton Slesarev, the head of engineering at the self-driving car company Yandex, said in an email interview.  Weather, roadwork, accidents, and inconsistent behavior and reactions from other drivers can contribute to the unpredictability of a journey, even for drivers who commute to the same location every day, he added. 

Yandex operates Europe’s first robot taxi service and already uses automated delivery robots, the Yandex rovers, for customer order deliveries from restaurants and grocery stores. The company uses machine learning to help its robots get around.  

"For example, it helps to perform vital perception functions such as recognizing road signs, even when they’re obscured by things like rain or a tree branch," Slesarev said. "Or to provide safety functions such as noticing a pedestrian about to cross the road, even at night or when the pedestrian is partly hidden by things like parked cars."

Using artificial intelligence to train cars can boost safety, observers say. 

"AI has been shown to be more accurate than people in driving situations, and it’s very probable that it will vastly decrease the number of accidents," Nicholson said.