Development of a Word Embedding adapted to Swedish Medical Terms

Sammanfattning

The amount of medical data has exploded in the past decades. While this may pose a challenge to doctors and medical personnel, it also yields great opportunities due to the rise of new technology. In particular, the advent of artificial intelligence (AI) raises hopes for the future of healthcare. Within AI, the development of language models makes it possible to handle and perform operations on text-based data, such as patient journals. One important element of many language models is word embeddings, which represent a model's vocabulary numerically, using several numerical values to describe each contained term. To use language models on Swedish medical data, it could be helpful to have access to a word embedding that is adapted to Swedish medical terms. Up to this point, there has not existed any such publicly available word embedding and thus this project has aimed to develop one. The developed language model, from which a word embedding is extracted, is pre-trained on the available Swedish general-purpose language model KB-BERT using sentences from the medical-oriented Swedish journal Läkartidningen. The resulting model was evaluated on a separate test set and compared with the base model, which showed that the pre-trained model had higher scores on two versions of an accuracy test.