A Novel Online and Non-Parametric Approach for Drift Detection in Big Data
Document Type
Conference Proceeding
Publication Date
8-3-2017
Publication Title
IEEE Access
Volume
5
First page number:
15883
Last page number:
15892
Abstract
A sizable amount of current literature on online drift detection tools thrive on unrealistic parametric strictures such as normality or on non-parametric methods whose power performance is questionable. Using minimal realistic assumptions such as unimodality, we have strived to proffer an alternative, through a novel application of Bernstein's inequality. Simulations from such parametric densities as Beta and Logitnormal as well as real-data analyses demonstrate this new method's superiority over similar techniques relying on bounds, such as Hoeffding's. Improvements are apparent in terms of higher power, efficient sample sizes, and sensitivity to parameter values.
Keywords
Change point detection, Non-parametric methods, Hoeffding’s inequality, Bernstein’s inequality, Big data, Anomaly detection.
Language
eng
Repository Citation
Bhaduri, M.,
Zhan, J.,
Chiu, C.,
Zhan, F.
(2017).
A Novel Online and Non-Parametric Approach for Drift Detection in Big Data.
IEEE Access, 5
15883-15892.
http://dx.doi.org/10.1109/ACCESS.2017.2735378