On Heterogeneous Ensembles for Anomaly Detection: Empirical Insights and Guidelines for the Design

Abstract

Algorithms for anomaly detection differ in how they conceptualize anomalies. When the nature and distributions of anomalies is unknown, combining diverse approaches should therefore improve robustness. However, heterogeneous ensembles remain uncommon in both studies and practice. We investigate heterogeneous ensembles and focus on three often overlooked questions: (a) how much does combining methods improve detection? (b) how do score normalization and aggregation affect ensembles? (c) does the evaluation metric influence optimal design? Our study considers structured tabular data with semantically independent features, providing key principles for designing effective ensembles. Experiments with 14 established algorithms on real and synthetic data show that small heterogeneous ensembles outperform individual methods, homogeneous ensembles, and larger ensembles. Linear normalization and mean aggregation offer stable performance across sizes and metrics. Optimal ensemble size depends on the evaluation criterion: top-focused metrics (e.g., Average Precision) favor small ensembles, while ranking metrics (e.g., AUROC) prefer slightly larger ones. Based on these findings, we recommend a heterogeneous ensemble of three complementary methods as a robust default. Given that the best individual method is dataset-dependent, this dataset-agnostic ensemble provides practical relevance. Overall, this work addresses a key gap in anomaly detection research and provides guidance for its application.