World’s first realistic simulated driving environment based on ‘crash-prone’ Michigan intersection
Real-time data collected at the two-lane roundabout will be used to efficiently test the safety of autonomous vehicles
Photo: Calvin Tuttle/University of Michigan, College of Engineering, U-M Transportation Research Institute (UMTRI)
ANN ARBOR—The first statistically realistic roadway simulation has been developed by researchers at the University of Michigan. While it currently represents a particularly perilous roundabout, future work will expand it to include other driving situations for testing autonomous vehicle software.
The simulation is a machine-learning model that trained on data collected at a roundabout on the south side of Ann Arbor, recognized as one of the most crash-prone intersections in the state of Michigan and conveniently just a few miles from the offices of the research team. Known as the Neural Naturalistic Driving Environment or NeuralNDE, it turned that data into a simulation of what drivers experience every day. Virtual roadways like this are needed to ensure the safety of autonomous vehicle software before other cars, cyclists, and pedestrians ever cross its path.
“The NeuralNDE reproduces the driving environment and, more importantly, realistically simulates these safety-critical situations so we can evaluate the safety performance of autonomous vehicles,” said Henry Liu, U-M professor of civil engineering, director of Mcity, a U-M-led public-private mobility research partnership. Liu is also director of the Center for Connected and Automated Transportation (CCAT) and corresponding author of the study in Nature Communications. The publication has been featured as an Editor’s Highlight.
Safety critical events, which require a driver to make split-second decisions and take action, don’t happen that often. Drivers can go many hours between events that force them to slam on the brakes or swerve to avoid a collision, and each event has its own unique circumstances. Together, these represent two bottlenecks in the effort to simulate our roadways, known as the “curse of rarity” and the “curse of dimensionality” respectively. The curse of dimensionality is caused by the complexity of the driving environment, which includes factors like pavement quality, the current weather conditions, and the different types of road users including pedestrians and bicyclists.
“The NeuralNDE reproduces the driving environment and, more importantly, realistically simulates these safety-critical situations so we can evaluate the safety performance of autonomous vehicles.”
Dr. Henry Liu, Director, Center for Connected and Automated Transportation (CCAT)
Director, Mcity
Professor of Civil and Environmental Engineering, University of Michigan
To model it all, the team tried to see it all. They installed sensor systems on light poles which continuously collect data at the State Street/Ellsworth Road roundabout. “The reason that we chose that location is that roundabouts are a very challenging, urban driving scenario for autonomous vehicles. In a roundabout, drivers are required to spontaneously negotiate and cooperate with other drivers moving through the intersection. In addition, this particular roundabout experiences high traffic volume and is two lanes, which adds to its complexity,” said Xintao Yan, a Ph.D. student in civil and environmental engineering and first author of the study, who is advised by Liu.
The NeuralNDE serves as a key component of the CCAT Safe AI Framework for Trustworthy Edge Scenario Tests, or SAFE TEST, a system developed by Liu’s team that uses artificial intelligence to reduce the testing miles required to ensure the safety of autonomous vehicles (AVs) by 99.99%. It essentially breaks the “curse of rarity,” introducing safety-critical incidents a thousand times more frequently than they occur in real driving. The NeuralNDE is also critical to a project designed to enable the Mcity Test Facility to be used for remote testing of AV software.
But unlike a fully virtual environment, these tests take place in mixed reality on closed test tracks such as the Mcity Test Facility and the American Center for Mobility in Ypsilanti, Michigan. In addition to the real conditions of the track, the autonomous vehicles also experience virtual drivers, cyclists, and pedestrians behaving in both safe and dangerous ways. By testing these scenarios in a controlled environment, AV developers can fine-tune their systems to better handle all driving situations.
“The reason that we chose that location is that roundabouts are a very challenging, urban driving scenario for autonomous vehicles. In a roundabout, drivers are required to spontaneously negotiate and cooperate with other drivers moving through the intersection.”
Xintao Yan, Ph.D. Student, University of Michigan
The NeuralNDE is not only beneficial for AV developers but also for researchers studying human driver behavior. The simulation can interpret data on how drivers respond to different scenarios, which can help develop more functional road infrastructure.
In 2021, the U-M Transportation Research Institute was awarded $9.95M in funding by the USDOT to expand the number of intersections equipped with these sensors to 21. This implementation will expand the capabilities of the NeuralNDE and provide real-time alerts to drivers with connected vehicles.
The research was funded by Mcity, CCAT, and the University of Michigan Transportation Research Institute (UMTRI). Founded in 1965, UMTRI is a global leader in multidisciplinary research and a partner of choice for industry leaders, foundations, and government agencies to advance safe, equitable, and efficient transportation and mobility. CCAT is a regional university transportation research center that was recently awarded a $15 million, five-year renewal by the U.S. Department of Transportation. As of June 21, 2023, this publication has been highlighted on Nature Communications’s webpage.
Title: Learning naturalistic driving environment with statistical realism
Authors: Xintao Yan, Zhengxia Zou, Shuo Feng, Haojie Zhu, Haowei Sun, & Henry X. Liu
DOI: 10.1038/s41467-023-37677-5
This story was written by Calvin Tuttle of the Center for Connected and Automated Transportation (CCAT).