Database Tuning via Distributional Reinforcement Learning for Hybrid Transactional/Analytical Processing Workloads

Reinforcement learning has become a popular approach for automating physical database design decisions such as which structures to create, retain, or remove in response to dynamic workloads. However, every existing method learns a single expected return for each candidate action and acts on it as if it were a reliable summary of the future. This is a fragile assumption in hybrid transactional/analytical processing (HTAP) environments, where the same action can yield very different long-horizon returns depending on the workload phase, and where bad decisions on write-heavy tables impose persistent maintenance penalties. We propose a different approach based on distributional reinforcement learning. Instead of learning one number per action, our agent learns the full distribution of possible long-horizon returns. This richer representation reveals which actions carry heavy downside tails. Experiments on the CH-Benchmark show that this method is strongest against the matched baselines and performs favorably against prior advisors under the HTAP objective.