TY - JOUR
T1 - A Deep Reinforcement Learning Network for Traffic Light Cycle Control
AU - Liang, Xiaoyuan
AU - Du, Xunsheng
AU - Wang, Guiling
AU - Han, Zhu
N1 - Funding Information:
Manuscript received January 22, 2018; revised November 21, 2018; accepted November 30, 2018. Date of publication January 3, 2019; date of current version February 12, 2019. This work was supported in part by the National Science Foundation under Grants CMMI-1844238, CNS-1717454, and CNS-1731424, and in part by the Air Force Office of Scientific Research under Grant MURI-18RT0073. The review of this paper was coordinated by Dr. Z. Fadlullah. (Corresponding author: Xiaoyuan Liang.) X. Liang and G. Wang are with the Department of Computer Science, New Jersey Institute of Technology, Newark, NJ 07102 USA (e-mail:,xl367@njit.edu; gwang@njit.edu).
Funding Information:
This work was supported in part by the National Science Foundation under Grants CMMI-1844238, CNS-1717454, and CNS-1731424, and in part by the Air Force Office of Scientific Research under Grant MURI-18RT0073.
Publisher Copyright:
© 1967-2012 IEEE.
PY - 2019/2
Y1 - 2019/2
N2 - Existing inefficient traffic light cycle control causes numerous problems, such as long delay and waste of energy. To improve efficiency, taking real-time traffic information as an input and dynamically adjusting the traffic light duration accordingly is a must. Existing works either split the traffic signal into equal duration or only leverage limited traffic information. In this paper, we study how to decide the traffic signal duration based on the collected data from different sensors. We propose a deep reinforcement learning model to control the traffic light cycle. In the model, we quantify the complex traffic scenario as states by collecting traffic data and dividing the whole intersection into small grids. The duration changes of a traffic light are the actions, which are modeled as a high-dimension Markov decision process. The reward is the cumulative waiting time difference between two cycles. To solve the model, a convolutional neural network is employed to map states to rewards. The proposed model incorporates multiple optimization elements to improve the performance, such as dueling network, target network, double Q-learning network, and prioritized experience replay. We evaluate our model via simulation on a Simulation of Urban MObility simulator. Simulation results show the efficiency of our model in controlling traffic lights.
AB - Existing inefficient traffic light cycle control causes numerous problems, such as long delay and waste of energy. To improve efficiency, taking real-time traffic information as an input and dynamically adjusting the traffic light duration accordingly is a must. Existing works either split the traffic signal into equal duration or only leverage limited traffic information. In this paper, we study how to decide the traffic signal duration based on the collected data from different sensors. We propose a deep reinforcement learning model to control the traffic light cycle. In the model, we quantify the complex traffic scenario as states by collecting traffic data and dividing the whole intersection into small grids. The duration changes of a traffic light are the actions, which are modeled as a high-dimension Markov decision process. The reward is the cumulative waiting time difference between two cycles. To solve the model, a convolutional neural network is employed to map states to rewards. The proposed model incorporates multiple optimization elements to improve the performance, such as dueling network, target network, double Q-learning network, and prioritized experience replay. We evaluate our model via simulation on a Simulation of Urban MObility simulator. Simulation results show the efficiency of our model in controlling traffic lights.
KW - Reinforcement learning
KW - deep learning
KW - traffic light control
KW - vehicular network
UR - http://www.scopus.com/inward/record.url?scp=85062995903&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85062995903&partnerID=8YFLogxK
U2 - 10.1109/TVT.2018.2890726
DO - 10.1109/TVT.2018.2890726
M3 - Article
AN - SCOPUS:85062995903
SN - 0018-9545
VL - 68
SP - 1243
EP - 1253
JO - IEEE Transactions on Vehicular Technology
JF - IEEE Transactions on Vehicular Technology
IS - 2
M1 - 8600382
ER -