Multimodal Classification of Adult-Type Diffuse Gliomas using Deep Learning on Whole-Slide Images and MRI

Sammanfattning

Adult-type diffuse gliomas are the most common malignant brain tumors and accuratemolecular classification is essential for diagnosis and treatment planning. Thisthesis investigates deep learning approaches for classifying IDH mutation status and1p/19q codeletion status using H&E-stained whole-slide images (WSIs) and magneticresonance imaging (MRI). In a first step, foundation models (FMs) were usedto extract feature vectors from the images, which were subsequently used as inputto the models that performed the final classification. Both unimodal and multimodalmodels were evaluated, where different multimodal fusion techniques wereexplored to combine histopathology and MRI features. The study was conductedon data from Sahlgrenska University Hospital, including 543 WSIs, 528 MRI scansand 525 multimodal patient pairs. Results showed that multimodal models achievedthe best overall performance, with the highest test AUC of 0.965 for IDH classificationand 0.987 for the codeletion classification task. WSI-based models consistentlyoutperformed MRI-based models, while MRI provided complementary informationthat improved certain multimodal models. Furthermore, for the WSI-based models,attention heatmaps could be generated, which may improve interpretability andstrengthen their potential clinical applicability. The findings demonstrate that deeplearning and FMs can enable reliable molecular classification of adult-type diffusegliomas, while multimodal models offer modest improvements over approaches basedonly on histopathology.