New tool evaluates progress in reinforcement learning

Urban driving, characterized by its perpetual stop-and-go nature, poses significant challenges in terms of efficiency and environmental impact. The constant acceleration and braking contribute substantially to pollution, including harmful greenhouse gas emissions. A promising solution, known as eco-driving, can be integrated into autonomous vehicle control systems to mitigate these inefficiencies and reduce a vehicle’s environmental footprint.

However, quantifying the precise impact of such systems and determining whether the environmental benefits justify the technological investment has long been a complex optimization problem. These challenges are amplified by the multi-faceted nature of urban environments, involving numerous agents like diverse vehicle types and a multitude of influencing factors such as speed, weather conditions, road topography, and traffic light synchronization.

Cathy Wu, the Thomas D. and Virginia W. Cabot Career Development Associate Professor in the Department of Civil and Environmental Engineering and the Institute for Data, Systems, and Society (IDSS) at MIT, alongside her role as a principal investigator in the Laboratory for Information and Decision Systems, recognized this critical gap. She questioned, “Is it a drop in the bucket, or is it something to think about?” when considering the potential of automated vehicles to curb emissions.

To tackle such intricate questions, a comprehensive approach to data collection is paramount. Wu emphasizes the necessity of gathering information from various sources, including detailed network topology (city intersection maps), U.S. Geological Survey data for road elevations, and environmental metrics such as temperature and humidity. Furthermore, data on vehicle types, ages, and fuel mixes are crucial for a holistic understanding of the system.

Eco-driving operates on the principle of subtle adjustments to minimize unnecessary fuel consumption. For instance, as an automated vehicle approaches a red light, instead of rushing to the intersection, it might simply “coast,” thereby avoiding burning fuel or electricity. The cascading effect of such intelligent deceleration extends beyond the automated vehicle itself; conventional cars following suit would also be compelled to slow down, amplifying the efficiency gains across the traffic flow.

While the concept of eco-driving is straightforward, determining its widespread impact presents formidable optimization problems that involve a multitude of interacting factors and parameters. This complexity has spurred a growing interest in leveraging Artificial Intelligence, particularly multi-agent deep reinforcement learning (DRL), to develop robust control solutions.

In a significant stride forward for the field, Professor Wu and her collaborators have developed a novel benchmark system called “IntersectionZoo.” Based on real-world urban eco-driving scenarios, IntersectionZoo aims to address the pressing need for standardized evaluation tools in DRL. The system was detailed in a paper presented at the 2025 International Conference on Learning Representation in Singapore.

A critical issue identified by Wu and her team two years prior was the lack of generalizability in most existing DRL algorithms. Often, an algorithm trained for a specific situation (e.g., a single intersection) would fail to remain effective even with minor modifications, such as adding a bike lane or altering traffic light timings. This problem, Wu notes, is not confined to traffic but permeates canonical tasks used to evaluate algorithm design, making it challenging to assess progress in robustness.

IntersectionZoo uniquely positions itself to advance DRL generalizability by featuring a rich set of characteristics vital for solving real-world problems. Its comprehensive dataset of 1 million data-driven traffic scenarios distinguishes it from other benchmarks, adding significant depth to the evaluation of deep RL algorithms.

Ongoing work will involve applying this sophisticated benchmarking tool to quantify the precise impact on emissions from implementing eco-driving in automated vehicles within a city, contingent on their deployment percentage. Beyond this specific application, the primary objective of IntersectionZoo is to foster the development of highly general-purpose deep reinforcement learning algorithms. These algorithms could find broad applications across diverse domains, including autonomous driving, video games, security systems, robotics, warehousing logistics, and classical control problems.

Emphasizing its collaborative spirit, Professor Wu states that “the project’s goal is to provide this as a tool for researchers, that’s openly available.” IntersectionZoo, complete with its comprehensive documentation, is freely accessible on GitHub. The paper’s lead authors are Vindula Jayawardana, a graduate student in MIT’s Department of Electrical Engineering and Computer Science (EECS), and Baptiste Freydt, a graduate student from ETH Zurich. Co-authors include Ao Qu, Cameron Hickert, and Zhongxia Yan PhD ’24.