GLEE: GPR-based Latent Effect Elimination for Performance Prediction of Big Data Transfer

Wuji Liu, Daqing Yun, Chase Wu

Research output: Contribution to journalConference articlepeer-review

2 Scopus citations


A wide spectrum of large-scale applications in science, industry, and business domains generate colossal amounts of data that need to be moved across geographical locations for various purposes. Such big data transfer is increasingly carried out over connections with guaranteed bandwidth provisioned in high-performance networks, as exemplified by ESnet via advance bandwidth reservation agents such as OSCARS. Accurate performance prediction of big data transfer is vital for resource management to reserve appropriate bandwidth and optimize resource utilization. Data-driven methods using machine learning offer a promising solution to such prediction capabilities. However, most of these methods highly rely on the quality of training data and might suffer from low prediction accuracy when performance measurements are collected in environments with high dynamics or under latent effects. We propose a performance prediction method to help end users obtain a good estimate of achievable throughput, which could also be used by network managers to infer expected bandwidth usage and hence optimize bandwidth reservation and utilization. Based on Gaussian Process Regression (GPR), our approach is able to automatically detect intrinsic noise and identify latent effects to enhance prediction accuracy. We implement and evaluate the proposed method using performance measurements collected in real-life networks and the results demonstrate its effectiveness.

Original languageEnglish (US)
Pages (from-to)4595-4600
Number of pages6
JournalProceedings - IEEE Global Communications Conference, GLOBECOM
StatePublished - 2022
Event2022 IEEE Global Communications Conference, GLOBECOM 2022 - Virtual, Online, Brazil
Duration: Dec 4 2022Dec 8 2022

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing


  • Big data transfer
  • latent effect elimination
  • machine learning
  • performance prediction
  • resource management


Dive into the research topics of 'GLEE: GPR-based Latent Effect Elimination for Performance Prediction of Big Data Transfer'. Together they form a unique fingerprint.

Cite this