Learning to shift thermostatically controlled loads

Demand response is a key mechanism for accommodating renewable power in the electric grid. Models of loads in demand response programs are typically assumed to be known a priori, leaving the load aggregator the task of choosing the best command. However, accurate load models are often hard to obtain. To address this problem, we propose an online learning algorithm that performs demand response while learning the model of an aggregation of thermostatically controlled loads. Specifically, we combine an adversarial multi-armed bandit framework with a standard formulation of load-shifting. We develop an Exp3-like algorithm to solve the learning problems. Numerical examples based on Ontario load data confirm that the algorithm achieves sub-linear regret and performs within 1% of the ideal case when the load is perfectly known.