Designing Trust: Optimizing User Experience in AI-Driven Diagnostic Predictions

Sammanfattning

Over the last few years, artificial intelligence (AI) has grown significantly and has been integrated across several areas. However, research shows a significant lack of trust among medical professionals towards AI systems. Simultaneously, research has proven AI to be a beneficial aid in diagnostics in regard to the healthcare system. In collaboration with the Johannes Gutenberg University of Mainz, this study explores the concerns and challenges that exist among healthcare professionals in relation to introducing AI as a diagnostic tool. The partnership is based on designing and evaluating a user-friendly interface that presents counterfactual explanations for CT scans, focusing on addressing the identified obstacles that may hinder the embracement of similar tools by healthcare professionals. Using Design Science Research, the study involved semi-structured interviews, thematic analyses, and user testing of the designed prototype. The study found that despite some positive attitudes towards AI among the doctors and medicine students participating in the study, a number of challenges including the potential loss of critical judgment, accountability issues, and low trust in AI diagnostic tools must be addressed to ensure a successful integration of such tools within diagnostics. While the designed prototype was deemed user-friendly, it did not significantly affect participants’ attitudes towards AI, indicating that User-Centered Design alone is insufficient to overcome trust and transparency issues. Future work should focus on the implementation of AI diagnostic tools in real healthcare settings to further evaluate their trustworthiness and reliability. Additionally, it should explore AI diagnostic tools in areas lacking specialized professionals to assess their capabilities in urgent and isolated situations and explore the role of user-friendly design in training and onboarding in healthcare settings.