LLM-enabled simulated patient–pharmacist conversations to improve communication training

Sammanfattning

Effective communication is an essential competence for pharmacists that leads to better health outcomes in patients. Modern practical teaching approaches, such as the Nordic platform this work is embedded into, utilize Generative Artificial Intelligence (GenAI) to simulate a Virtual Patient (VP) and provide personalized feedback to the pharmacy students after the conversation. However, these platforms are very new and can be limited in their capabilities and robustness as well as the quality of the feedback. This work aims at improving the feedback quality by introducing a Retrieval-Augmented Generation (RAG) as well as improving the platform’s capabilities by laying the technical foundation for an audio-based communication between the user and the VP. The RAG system was able to provide relevant non-parametric knowledge during the feedback generation, leading to an increase in feedback quality when evaluated by senior experts in social and clinical pharmacy. It was further able to better handle wrong and missing information, making the platform more robust. The audio-based communication was enabled through a pipeline based approach to achieve General Data Protection Regulation (GDPR) and Artificial Intelligence Act (AI Act) compliance. The student’s voice is transcribed on a university server with an Automatic Speech Recognition (ASR) model, the transcribed text is then sent on to the Large Language Model (LLM), which generates an answer with its native audio support. The best performing ASR model for this solution was whisper-large-v3 which achieved the highest transcription quality and a low latency while being the least complex to implement and maintain in the platform.