Stream Clustering Robust to Concept Drift

Abstract

Data streams are everywhere in modern technologies, spanning from industrial process control to network traffic analysis. Stream clustering is required to describe data streams in real time and maintain accurate knowledge of their underlying structures. However, data streams frequently exhibit non-stationarity, changes in distributions, and the emergence of new classes. These alterations—commonly referred to as "concept drift"—severely disturb algorithms, resulting in inconsistent outcomes and models. We present SDOstreamclust, an incremental algorithm for stream clustering. It inherits the distinctive features of methods founded on Sparse Data Observers, i.e., lightweight, intuitive, self-adjusting, resistant to noise, capable of identifying non-convex clusters, and constructed upon robust parameters and interpretable models. We compare SDOstreamclust with established algorithms and evaluate them with a broad collection of datasets, both real and synthetic. SDOstreamclust shows outstanding performances, a major adaptability to concept drift, and a superior parameter stability and robustness. Often ignored in the evaluation of new methods, concept drift is a major challenge for next-generation algorithms, since it is inherent to evolving data and a main cause of degradation in machine learning. Hence, SDOstreamclust emerges as a major alternative for unsupervised streaming data analysis.