Multivariate Resource Usage Prediction With Frequency-Enhanced and Attention-Assisted Transformer in Cloud Computing Systems

Resource usage prediction in cloud data centers is critically important. It can improve providers' service quality and avoid resource wastage and insufficiency. However, the time series of resource usage in cloud environments is characterized by multidimensional, nonlinear, and high-volatility characteristics. Achieving high-accuracy prediction for time series with such characteristics is necessary but difficult. Traditional prediction methods based on regression algorithms and recurrent neural networks cannot effectively extract nonlinear features from data sets. Besides, many deep learning models suffer from gradient explosion or gradient vanishing during the training stage. Current commonly used prediction methods fail to uncover some vital information about the frequency domain features in the time series. To resolve these challenges, we design a Forecasting method based on the Integration of a Savitzky-Golay (SG) filter, a frequency enhanced decomposed transformer (FEDformer) model, and a frequency-enhanced channel attention mechanism (FECAM), named FISFA. It adopts the SG filter to reduce noise and smooth sequences in the raw sequences of resources. Then, we develop a hybrid transformer-based model integrating FEDformer and the FECAM, effectively capturing the frequency domain patterns. Besides, a meta-heuristic optimization algorithm, i.e., genetic simulated annealing-based particle swarm optimizer, is proposed to optimize key hyperparameters of FISFA. Then, FISFA predicts the future needs for multidimensional resources in highly fluctuating traces in real-life cloud environments. Experimental results demonstrate that FISFA achieves higher accuracy and performs more efficient prediction than several benchmark forecasting methods with realistic data sets collected from Alibaba and Google cluster traces. FISFA improves the prediction accuracy on average by 32.14%, 25.49%, and 27.71% over vanilla long short-term memory, transformer, and Informer methods, respectively.