TY - JOUR
T1 - Multiscale Drift Detection Test to Enable Fast Learning in Nonstationary Environments
AU - Wang, Xue Song
AU - Kang, Qi
AU - Zhou, Meng Chu
AU - Pan, Le
AU - Abusorrah, Abdullah
N1 - Funding Information:
Manuscript received May 2, 2019; revised September 27, 2019 and April 17, 2020; accepted April 18, 2020. Date of publication June 16, 2020; date of current version June 23, 2021. This work was supported in part by the National Natural Science Foundation of China under Grant 51775385, Grant 61703279, and Grant 71371142, in part by the Strategy Research Project of Artificial Intelligence Algorithms of Ministry of Education of China, in part by the Fundamental Research Funds for the Central Universities, and in part by the Deanship of Scientific Research (DSR) at King Abdulaziz University, Jeddah, under Grant D-687-135-1441. This article was recommended by Associate Editor P. P. Angelov. (Corresponding author: Qi Kang.) XueSong Wang was with the Department of Control Science and Engineering, Tongji University, Shanghai, China. He is now with the School of Computer Science and Engineering, University of New South Wales, Sydney, NSW 2052, Australia (e-mail: xuesong.wang1@student.unsw.edu.au).
Publisher Copyright:
© 2019 IEEE.
PY - 2021/7
Y1 - 2021/7
N2 - A model can be easily influenced by unseen factors in nonstationary environments and fail to fit dynamic data distribution. In a classification scenario, this is known as a concept drift. For instance, the shopping preference of customers may change after they move from one city to another. Therefore, a shopping website or application should alter recommendations based on its poorer predictions of such user patterns. In this article, we propose a novel approach called the multiscale drift detection test (MDDT) that efficiently localizes abrupt drift points when feature values fluctuate, meaning that the current model needs immediate adaption. MDDT is based on a resampling scheme and a paired student t-test. It applies a detection procedure on two different scales. Initially, the detection is performed on a broad scale to check if recently gathered drift indicators remain stationary. If a drift is claimed, a narrow scale detection is performed to trace the refined change time. This multiscale structure reduces the massive time of constantly checking and filters noises in drift indicators. Experiments are performed to compare the proposed method with several algorithms via synthetic and real-world datasets. The results indicate that it outperforms others when abrupt shift datasets are handled, and achieves the highest recall score in localizing drift points.
AB - A model can be easily influenced by unseen factors in nonstationary environments and fail to fit dynamic data distribution. In a classification scenario, this is known as a concept drift. For instance, the shopping preference of customers may change after they move from one city to another. Therefore, a shopping website or application should alter recommendations based on its poorer predictions of such user patterns. In this article, we propose a novel approach called the multiscale drift detection test (MDDT) that efficiently localizes abrupt drift points when feature values fluctuate, meaning that the current model needs immediate adaption. MDDT is based on a resampling scheme and a paired student t-test. It applies a detection procedure on two different scales. Initially, the detection is performed on a broad scale to check if recently gathered drift indicators remain stationary. If a drift is claimed, a narrow scale detection is performed to trace the refined change time. This multiscale structure reduces the massive time of constantly checking and filters noises in drift indicators. Experiments are performed to compare the proposed method with several algorithms via synthetic and real-world datasets. The results indicate that it outperforms others when abrupt shift datasets are handled, and achieves the highest recall score in localizing drift points.
KW - Concept drift detection
KW - multiscale drift detection test
KW - nonstationary environments
KW - resampling
KW - time series data analysis
UR - http://www.scopus.com/inward/record.url?scp=85109201594&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85109201594&partnerID=8YFLogxK
U2 - 10.1109/TCYB.2020.2989213
DO - 10.1109/TCYB.2020.2989213
M3 - Article
C2 - 32544055
AN - SCOPUS:85109201594
SN - 2168-2267
VL - 51
SP - 3483
EP - 3495
JO - IEEE Transactions on Cybernetics
JF - IEEE Transactions on Cybernetics
IS - 7
M1 - 9119144
ER -