Traffic Signal Control Using Hybrid Action Space Deep Reinforcement Learning

Area of Science:

• Artificial Intelligence
• Transportation Engineering
• Computer Science

Background:

• Intelligent traffic signal control (TSC) heavily relies on deep reinforcement learning (DRL).
• Existing DRL frameworks for TSC are limited to either discrete (phase selection) or continuous (phase duration) control.
• A flexible approach combining both discrete and continuous DRL for TSC is lacking.

Purpose of the Study:

• To propose a novel hybrid DRL framework for traffic signal control.
• To develop an approach that simultaneously determines the optimal traffic light phase and its duration.
• To enhance the efficiency of traffic signal systems through integrated decision-making.

Main Methods:

• Adapted a hybrid Deep Reinforcement Learning (DRL) approach.
• Customized a Parameterized Deep Q-Networks (P-DQN) architecture for hierarchical decision-making.
• Evaluated the framework using Simulation of Urban MObility (SUMO).

Main Results:

• The proposed hybrid DRL framework demonstrated superior performance compared to existing benchmarks.
• Achieved a 22.20% reduction in average vehicle queue length.
• Achieved a 5.78% reduction in average travel time.

Conclusions:

• The hybrid DRL approach effectively integrates discrete and continuous decision-making for TSC.
• The P-DQN based framework offers a flexible and high-performing solution for intelligent traffic management.
• Significant improvements in traffic flow efficiency were observed, highlighting the framework's practical applicability.