From unsupervised to semi-supervised : Leveraging self-supervised embeddings with partial anomaly supervision for tabular anomaly detection

Sammanfattning

Self-supervised learning (SSL) has shown promise for extracting useful representations from unlabelled tabular data, but recent work has found that SSL embeddings do not improve unsupervised anomaly detection. This thesis investigates whether these limitations can be overcome by pairing SSL representations with semi-supervised detector heads, a direction identified as future work by Mai et al. (2024). Using SCARF as the SSL method, embeddings from a frozen, contrastively pre-trained encoder are passed to four downstream models: Isolation Forest and One-Class SVM in the unsupervised setting, and self-training with XGBoost and logistic regression at label budgets of 1%, 5%, 10%, and 20%. Raw and PCA-transformed features serve as baselines. Experiments are conducted on three tabular benchmark datasets from ADBench (Cardiotocography, Shuttle, SpamBase), across five random seeds. The results show that SCARF embeddings consistently underperform raw and PCA features across all datasets, detectors, and supervision levels. Semi-supervision improves detection for all feature conditions, but the improvement is proportionally similar across conditions, and the performance gap between SCARF and baselines does not close as more labels are introduced. These findings extend the negative results of Mai et al. into the semi-supervised regime, suggesting that the contrastive pre-training objective of SCARF, applied without supervised fine-tuning, does not produce representations sufficiently aligned with the anomaly detection task to benefit from semi-supervised detector heads.