Low-False-Alarm Evaluation Changes the Ranking of Multisensor Anomaly Detectors

Abstract

This study investigated multisensor anomaly detection under low false-alarm constraints in safety-critical systems, examining whether conventional aggregate metrics provide a reliable basis for model selection under strict false-alarm budgets. A detection framework combining temporal modeling, anomaly scoring, calibration, and temporal persistence was evaluated across forecasting, autoencoding, and conformal multisensor approaches on four public multivariate sensor datasets. Low-false-alarm evaluation changed detector rankings in seven model-dataset pairs: on the Mars Science Laboratory dataset, the Temporal Autoencoder moved from third to first place under recall at false-alarm rate up to 0.05, reaching 0.1062; on the Secure Water Treatment dataset, the Gated Recurrent Unit Forecaster reached 0.6448; on the Water Distribution dataset, subsystem-aware grouping increased the same metric from 0.0241 to 0.1838, while correlation-based grouping degraded performance. Calibration improved false-alarm control but did not compensate for poor score separability, and cross-sensor modeling was beneficial only when groupings reflected the monitored system structure.