Evaluation of AI-based decision support in infectious diseases

Sammanfattning

Antibiotic resistance is a growing global health concern, driven in part by the misuseand overuse of antibiotics. Effective and rapid treatment decisions are importantboth for individual patient outcomes and for limiting resistance development. Inrecent years, the exploration of AI-based methods for diagnostics and decision supporthas increased. In Confidence-based prediction of antibiotic resistance at thepatient level, Inda-Díaz et al. (2026) developed a transformer model for diagnosticsusing European data. This thesis focuses on evaluating the model for Escherichiacoli infections and nine antibiotics using additional European data and data fromThe Public Health Agency of Sweden.A baseline evaluation applies the model to both datasets to compare performance.To address potential biases caused by differences in data distributions, threemitigation approaches are applied to adapt the model: decision threshold adjustment,fine-tuning, and retraining. Performance is assessed using standard metrics inthe field, both overall and stratified by antibiotic, patient age, and gender. Fairnessis evaluated through subgroup differences in error rates, inspired by the fairnesscriteria equalized odds and equal opportunity.The results indicate that, despite distributional differences, the baseline modelsurprisingly achieves better overall performance on Swedish data than on Europeandata. Performance differences across gender and age are observed for both thebaseline and adapted models, with the lowest performance seen for isolates fromfemale patients and from the youngest age groups, compared to male patients andthe oldest age groups. None of the adapted models reduce the error rate differencesbetween genders, whereas all decrease the maximum pairwise difference across agegroups. Stratifying by antibiotic shows only marginal differences in results for mostantibiotics, but notable discrepancies for some.These findings suggest that future work toward fair and effective AI-basedtreatment decision support should combine fairness-aware bias mitigation with approachesthat address complex, antibiotic-specific resistance mechanisms.