Safe and Trustworthy Human-Centered Mobility: Leveraging Foundation Models and Digital Twins for Vulnerable Road User-Vehicle Interactions

Principal Investigator(s):

Sikai Chen, Assistant Professor of Civil and Environmental Engineering – University of Wisconsin–Madison

Project Abstract:
The HUMANE (HUman-centered Mobility Agent iNtEraction) framework introduces an innovative approach to enhancing safety and trust in autonomous vehicle interactions with vulnerable road users (VRUs) by integrating foundation models with digital twin technology. This project addresses critical challenges in human-agent interaction within complex urban environments through three integrated modules:

• A multimodal human intent recognition system leveraging foundation models
• A trustworthy interactive learning framework incorporating human feedback
• An enhanced digital twin validation platform

The framework will be validated through both high-fidelity simulation and real-world testing at a signal-controlled intersection in Madison, WI, using our lab-developed autonomous vehicle fleet and roadside infrastructure. Building upon our previous CCAT-funded research in VR-based interaction platforms and hardware system integration, HUMANE represents a significant advancement in creating safe and efficient autonomous transportation systems that can naturally interact with all road users.

The project will deliver open-source software, comprehensive datasets, and educational materials, contributing to both academic advancement and practical implementation in the field of connected and autonomous vehicles.

Institution(s): University of Wisconsin-Madison

Award Year: 2025

Research Focus: Safety, Mobility

Project Form(s):