Improving Information Retrieval in Swedish Clinical Documentation : A Comparison of Semantic, Lexical and Hybrid Retrieval

Sammanfattning

Healthcare professionals rely on clinical documentation to support decision-making, but finding relevant information is challenging when documents are long, terminology varies, and search systems depend on exact keyword matching. This is a known limitation of DocPlus, Region Uppsala's system for clinical guidelines, where clinicians have described the search functionality as sensitive to exact wording and difficult to use for exploratory search. This thesis investigates whether dense and hybrid retrieval methods can improve search performance for Swedish clinical documentation compared to traditional lexical retrieval, using DocPlus as a case study. A retrieval pipeline was developed for approximately 9300 publicly available DocPlus documents from Region Uppsala. Three retrieval approaches were implemented and compared: lexical retrieval using BM25, dense retrieval using multilingual transformer-based embeddings, and hybrid retrieval using Reciprocal Rank Fusion. The methods were evaluated using clinician-generated information needs, search queries and manually annotated relevance judgments. Retrieval performance was measured using normalized Discounted Cumulative Gain at rank 10 (nDCG@10) and Mean Reciprocal Rank (MRR). Preliminary model and parameter experiments showed that Multilingual-E5-Large clearly outperformed the evaluated Swedish clinical BERT model and was therefore used in the final retrieval comparison. Hybrid E5 achieved the highest nDCG@10 score (0.627), while the same model also achieved the highest MRR (0.744). Dense, hybrid and lexical approaches all substantially outperformed the current DocPlus search baseline in both metrics. These findings suggest that dense and hybrid retrieval are better suited than traditional keyword-based retrieval for Swedish clinical documentation search. Hybrid approaches showed great promise for combining semantic similarity with exact lexical matching and clearly outperform dense retrieval overall in this study. The results highlight the potential of semantic retrieval methods for clinical documentation search, and point to the importance of domain-specific evaluation when developing such systems.