Predicting antibiotic resistance using fusion transformers

Sammanfattning

Antimicrobial resistance threatens recent gains in global public health by making itmore difficult to treat infections. Clinicians must administer treatments based onlimited diagnostic information and increasing resistance complicates these decisions.This thesis project explores ways to support this process by developing a frameworkfor training a transformer model using data fusion of patient and genotype datawith phenotype data to make individualized predictions of antibiotic resistance inEscherichia coli on these multimodal data. To achieve this, the model was trainedin two stages: first, the model was pre-trained on large volumes of unimodal data usingmasked language modeling to learn patterns within the modalities; and second,the model was fine-tuned on a small multimodal dataset to learn patterns acrossmodalities. To evaluate pre-training strategies, the model was fine-tuned on twoclinically relevant tasks and smaller training sets. To determine the value of introducingmultimodality and the effect of genotype data availability on performance,the model was fine-tuned on varying levels of available genotype information.The results show that the model performs well on the fine-tuning tasks, that pretrainingon unimodal data improves performance, and that the model can extrapolatewell from small training sets and incomplete data. Therefore, it can be concludedthat this work has achieved the aim of developing a model that can makeaccurate predictions based on limited diagnostic information. Importantly, largeperformance improvements were observed with increasing genotype data availability,especially on difficult antibiotics. Furthermore, the model was better able toutilize available genotype information when pre-trained. However, while no clearconclusion on the best pre-training strategy can be drawn from the results of thiswork, they indicate that using systematic class masking in pre-training yields thehighest performance. Future research should further investigate the best strategyfor pre-training the model, how the model utilizes genotype data to improve performance,and how genotype data affects performance on limited training data.