Optimizing Handover Parameters by Q-Learning for Heterogeneous Radio-Optical Networks

Existing literature studying the access point (AP)-user association problem of heterogeneous radio-optical networks either investigates quasi-static network selection or only considers vertical handover (VHO) dwell time from optical to radio. The quasi-static assumption can result in outdated decisions for highly mobile scenarios. Solely focusing on the optical to radio handover ignores the importance of dwell time for VHO from radio to optical. In this paper, we propose a flexible and holistic framework, that runs a self-optimizing algorithm at the centralized coordinator (CC). This CC resides in the LTE eNodeB and controls the handover parameters of all the visible light communication (VLC) APs under the coverage of the LTE eNodeB. Based on Q-learning approach, the algorithm optimizes the time-to-trigger ($TTT$) values for VHO between LTE and VLC. Case studies are performed to validate the considerable gain in terms of average throughput by optimizing $TTT$s. We evaluate the impact of learning parameters on the optimal throughput and convergence speed through trace-driven simulations. The simulation results reveal that the Q-learning based algorithm improves the average throughput of mobile device by 25% when compared to the fixed $TTT$ scheme. Furthermore, this algorithm is capable of self-optimizing handover parameters in an online manner.