Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks

Guanhua Qiao, Supeng Leng, Sabita Maharjan, Yan Zhang, Nirwan Ansari

Research output: Contribution to journalArticlepeer-review

238 Scopus citations


In this article, we propose a cooperative edge caching scheme, a new paradigm to jointly optimize the content placement and content delivery in the vehicular edge computing and networks, with the aid of the flexible trilateral cooperations among a macro-cell station, roadside units, and smart vehicles. We formulate the joint optimization problem as a double time-scale Markov decision process (DTS-MDP), based on the fact that the time-scale of content timeliness changes less frequently as compared to the vehicle mobility and network states during the content delivery process. At the beginning of the large time-scale, the content placement/updating decision can be obtained according to the content popularity, vehicle driving paths, and resource availability. On the small time-scale, the joint vehicle scheduling and bandwidth allocation scheme is designed to minimize the content access cost while satisfying the constraint on content delivery latency. To solve the long-term mixed integer linear programming (LT-MILP) problem, we propose a nature-inspired method based on the deep deterministic policy gradient (DDPG) framework to obtain a suboptimal solution with a low computation complexity. The simulation results demonstrate that the proposed cooperative caching system can reduce the system cost, as well as the content delivery latency, and improve content hit ratio, as compared to the noncooperative and random edge caching schemes.

Original languageEnglish (US)
Article number8879573
Pages (from-to)247-257
Number of pages11
JournalIEEE Internet of Things Journal
Issue number1
StatePublished - Jan 2020

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications


  • Content delivery
  • content placement
  • cooperative edge caching
  • deep deterministic policy gradient (DDPG)
  • double time-scale Markov decision process (DTS-MDP)
  • vehicular edge computing and networks


Dive into the research topics of 'Deep Reinforcement Learning for Cooperative Content Caching in Vehicular Edge Computing and Networks'. Together they form a unique fingerprint.

Cite this