Virtual machine placement via Q-learning with function approximation

While existing virtual machine technologies provide easy-to-use platforms for distributed computing applications, many are far from efficient and not designed to accommodate diverse objectives, which dramatically penalizes their performance. These shortcomings arise from 1) not having a formal optimization framework that readily leads to algorithmic solutions for diverse objectives; 2) not incorporating the knowledge of the underlying network topologies and the communication/interaction patterns among the virtual machines/services, and 3) not considering the time-varying aspects of real-world environments. This paper formalizes an optimization framework and develops corresponding algorithmic solutions using Markov Decision Process and Q-Learning for virtual machine/service placement and migration for distributed computing in time-varying environments. Importantly, the knowledge of the underlying topologies of the computing infrastructure, the interaction patterns between the virtual machines, and the dynamics of the supported applications will be formally characterized and incorporated into the proposed algorithms in order to improve performance. Simulation results for small-scale and large-scale networks are provided to verify our solution approach.