Apply Reinforcement Learning to Control Traffic Signals by Action Space Optimizations

Vehicle traffic systems can be improved by optimized traffic signal control. Performance metrics including accumulated waiting time and total fuel consumption are affected by different configurations of traffic signal controllers. This thesis describes a workflow for defining effective traffic signal controllers utilizing reinforcement learning. Configuration of components for reinforcement learning are explained, with a focus on phase set and action space definitions, to result in performance metrics improvement. A combinatorial search method where traffic signals were configured with different viable phase sets is presented, to identify individual phases of interest or concern. An improved phase set, now action space for the reinforcement learning agent(s), was then constructed composed of top performers from the combinatorial search. Analysis results show that reinforcement learning agents benefit from having an action space defined from highly ranked phases and learn to produce combined positive behaviors from different phase sets. At peak hours, morning, noon, and evening, of vehicle traffic simulation, optimized phase sets and action spaces produced a reduction in accumulated waiting time and fuel consumption.