Stochastic dynamic resource allocation for HIV prevention and treatment: An approximate dynamic programming approach

Özlem Coşgun, Büyüktahtakın Esra Büyüktahtakın

Research output: Contribution to journalArticlepeer-review

13 Scopus citations


Human immunodeficiency virus (HIV) is a key global health concern, with 33 million people living with HIV worldwide and 2.7 million new infections occurring annually. To prevent the spread of this widely prevalent epidemic disease, prevention and treatment intervention strategies urgently need to be implemented. The goal of this study is to propose stochastic dynamic programming (SDP) and approximate dynamic programming (ADP) algorithms that will optimally allocate the limited intervention budget among the HIV disease compartments and determine the best set of interventions that should be applied to each disease compartment, while minimizing the number of HIV-infected and people diagnosed with acquired immune deficiency syndrome (AIDS) as well as related deaths over a multi-year planning horizon. A compartmental model is constructed and formulated as a nonstationary Markov decision process (MDP) in order to capture the progression of the disease among the highest risk group—African American/black men who have sex with men (BMSM). In order to alleviate the computational difficulties arising from the exponentially growing state space in the SDP, we propose ADP algorithms that determine the approximately optimal budget allocation policies over six years. Our results suggest a greater allocation of the limited budget to prevention measures rather than treatment interventions, such as antiretroviral therapy (ART). As opposed to traditional policies that allocate the budget only once at the beginning of the time horizon, the ADP model suggests using a dynamic proportional budget strategy, allocating the budget dynamically over a multi-period planning period as the uncertainty in disease transmission is revealed. Results show that our ADP approach provides significant increases in health benefits and cost savings in HIV prevention and intervention.

Original languageEnglish (US)
Pages (from-to)423-439
Number of pages17
JournalComputers and Industrial Engineering
StatePublished - Apr 2018

All Science Journal Classification (ASJC) codes

  • General Computer Science
  • General Engineering


  • Approximate dynamic programming
  • Disease transmission uncertainty
  • HIV epidemic disease
  • Interventions
  • Markov decision process (MDP)
  • Resource allocation
  • Stochastic dynamic programming


Dive into the research topics of 'Stochastic dynamic resource allocation for HIV prevention and treatment: An approximate dynamic programming approach'. Together they form a unique fingerprint.

Cite this