Multiscale Drift Detection Test to Enable Fast Learning in Nonstationary Environments

Xue Song Wang, Qi Kang, Meng Chu Zhou, Le Pan, Abdullah Abusorrah

Research output: Contribution to journalArticlepeer-review

8 Scopus citations

Abstract

A model can be easily influenced by unseen factors in nonstationary environments and fail to fit dynamic data distribution. In a classification scenario, this is known as a concept drift. For instance, the shopping preference of customers may change after they move from one city to another. Therefore, a shopping website or application should alter recommendations based on its poorer predictions of such user patterns. In this article, we propose a novel approach called the multiscale drift detection test (MDDT) that efficiently localizes abrupt drift points when feature values fluctuate, meaning that the current model needs immediate adaption. MDDT is based on a resampling scheme and a paired student t-test. It applies a detection procedure on two different scales. Initially, the detection is performed on a broad scale to check if recently gathered drift indicators remain stationary. If a drift is claimed, a narrow scale detection is performed to trace the refined change time. This multiscale structure reduces the massive time of constantly checking and filters noises in drift indicators. Experiments are performed to compare the proposed method with several algorithms via synthetic and real-world datasets. The results indicate that it outperforms others when abrupt shift datasets are handled, and achieves the highest recall score in localizing drift points.

Original languageEnglish (US)
Article number9119144
Pages (from-to)3483-3495
Number of pages13
JournalIEEE Transactions on Cybernetics
Volume51
Issue number7
DOIs
StatePublished - Jul 2021

All Science Journal Classification (ASJC) codes

  • Software
  • Control and Systems Engineering
  • Information Systems
  • Human-Computer Interaction
  • Computer Science Applications
  • Electrical and Electronic Engineering

Keywords

  • Concept drift detection
  • multiscale drift detection test
  • nonstationary environments
  • resampling
  • time series data analysis

Fingerprint

Dive into the research topics of 'Multiscale Drift Detection Test to Enable Fast Learning in Nonstationary Environments'. Together they form a unique fingerprint.

Cite this