Investigating Foundation Models in Medical Imaging

Sammanfattning

Medical image segmentation is a critical bottleneck in modern healthcare diagnostics, where the manual analysis of complex anatomical structures demands significant time and expertise. This thesis investigates how foundation models - large-scale pre-trained neural networks that have revolutionized computer vision - can automate and improve medical image segmentation accuracy. By focusing on CT scans, particularly for challenging vascular and head-and-neck anatomy in the Aortaseg24 and HaN-Seg datasets, we aim to assess whether these models can reduce diagnostic delays while maintaining clinical precision. Through systematic evaluation of six foundation models against a baseline Vanilla U-Net, we explore the practical viability of these advanced architectures in real-world medical settings. From our experiments we observe that foundation models consistently outperform the Vanilla U-Net baseline. STU-Net emerged as the top performer, achieving Dice scores of 0.76 for Aortaseg24 and 0.70 for HaN-Seg, significantly surpassing the baseline’s 0.37 and 0.36 respectively. SwinMM followed closely, showing particularly strong performance on the more complex HaN-Seg task. Class-wise analysis revealed that foundation models excel in segmenting larger anatomical structures, with STU-Net achieving Dice scores above 0.90 in Aortaseg24, while all models showed limitations with smaller, intricate structures in the HaN-Seg task. Computational efficiency varied widely among models. DAE processed the highest number of samples per second (2.99 ± 0.55), while SuPreM demonstrated the fastest convergence, requiring only 6000 iterations for Aortaseg24 and 4500 for HaN-Seg. Conversely, SPAD-Nets required the most iterations (30120 for Aortaseg24), highlighting significant differences in learning efficiency. The study also revealed task-specific performance variability. Models like SwinMM and MIS-FM showed better relative performance on HaN- Seg compared to Aortaseg24, highlighting that pre-training strategies and architectural designs influence a model’s suitability for specific anatomical regions. Our findings highlight the potential of foundation models to progress medical image segmentation while also highlighting challenges in balancing performance with computational efficiency. The observed task-specific variability emphasizes the need for careful model selection based on the target application. This comprehensive analysis provides crucial insights for future devel- opments in medical image segmentation, emphasizing the need for robust, versatile architectures capable of handling diverse and complex anatomical regions.