Caching in Dynamic IoT Networks by Deep Reinforcement Learning

Jingjing Yao, Nirwan Ansari

Research output: Contribution to journalArticlepeer-review

28 Scopus citations


The sensing service of Internet-of-Things (IoT) networks enables IoT sensors to sense the environment information (e.g., temperature and traffic conditions) and send them through the IoT gateway to the users who request those information. The explosive growth of IoT users and sensors injects massive traffic into IoT networks and easily depletes the battery of IoT sensors. Caching at the IoT gateway is hence a promising solution to mitigate this problem by storing popular IoT data at the IoT gateway and sending them directly to the users instead of activating IoT sensors to transmit the data. In our work, we investigate the content placement problem, which determines data to be cached at each time epoch in dynamic IoT networks with the objective to minimize the average data transmission delay constrained by the cache storage capacity and IoT data freshness. We formulate our problem as an integer linear programming (ILP) problem and then model it as a Markov decision process (MDP). A deep reinforcement learning algorithm is proposed to solve this problem and its performances are demonstrated via extensive simulations.

Original languageEnglish (US)
Article number9123362
Pages (from-to)3268-3275
Number of pages8
JournalIEEE Internet of Things Journal
Issue number5
StatePublished - Mar 1 2021

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Information Systems
  • Hardware and Architecture
  • Computer Science Applications
  • Computer Networks and Communications


  • Actor-critic
  • Internet of Things (IoT)
  • Quality of Service (QoS)
  • caching
  • content placement
  • data freshness
  • deep reinforcement learning (DRL)


Dive into the research topics of 'Caching in Dynamic IoT Networks by Deep Reinforcement Learning'. Together they form a unique fingerprint.

Cite this