Evaluating LINe in Semantic Segmentation OOD Detection and Improving Performance with Pairwise Activation Statistics

Sammanfattning

In artificial intelligence, out-of-distribution (OOD) detection refers to the task of identifying classes of objects that are not part of the training data. Failure to detect such anomalies is a significant safety concern for vision systems operating in open environments, such as autonomous vehicles. While many OOD detectors have been designed for whole-image classification, far fewer look at the pixel level that semantic-segmentation networks require. This thesis studies whether a recent and promising LINe detector—originally proposed for image classification—can be applied to segmentation and how its performance can be improved. An encoder-decoder convolutional neural network, DeepLab v3+-R101, pre-trained on Cityscapes, is used as the segmentation model. The LINe method is adapted by creating class-specific activation and weight masks for every pixel, followed by log-sum-exp scoring. I then introduce a complementary pairwise Co-Activation score that flags pixels whose channel pairs are rarely active together in in-distribution data. Temperature scaling and light Gaussian blurring are applied as post-processing steps. Experiments on the Fishyscapes Lost and Found dataset show that the plain LINe method transfer already outperforms standard post-hoc baseline methods such as: soft-max confidence, max-logit, and Euclidean feature distance. However, the observed improvement was not as much as in the original classification implementation. On its own, LINe received a false positive rate (FPR@95) of 18.7 at 95% true positive rate and a 24.6% average precision (AP) on the Lost and Found validation set. When introducing the Co-Activation score and extra fine-tuning, performance increases significantly to FPR@95 of 5.25% and AP to 52.7%, showing the great potential of the proposed method.