Navigating the Risks of AI in Management Consulting - A Qualitative and Experimental Study of AI Hallucinations, Task Suitability, and Mitigation Strategies

Sammanfattning

Large Language Models (LLMs) have been rapidly adopted not only by individual users, butalso in professional settings, particularly within knowledge-intensive industries such asmanagement consulting. The integration of these tools into professional practice has creatednew opportunities for efficiency, while also introducing significant risks. Most notably is thephenomenon of AI hallucinations, where models generate factually incorrect or unverifiablecontent with apparent confidence. While existing research has primarily examinedhallucinations from a technical perspective, little attention has been given to how theymanifest and are managed in real-world professional settings.Therefore, the purpose of this study is to explore how AI hallucinations affect small-sizedmanagement consulting firms, examining both the strategies consultants use to manageLLM-generated errors and which consulting tasks are most vulnerable to them. Using aqualitative, abductive research design, the study combines nine semi-structured interviewswith management consultants and systematic experimental testing of Microsoft Copilotacross four consulting tasks: exploratory research, client communication, summarization andinterview synthesis.The findings reveal that error management in small consulting firms is largely informal,driven by individual professional judgement rather than formalized governance structures.Consultants rely on domain expertise, pattern recognition, cross-source verification andselective task allocation as their primary safeguards. Human oversight emerges as the centraland non-negotiable control mechanism across all firms.The experimental component suggests that the task of interview synthesis was identified asthe most vulnerable task, followed by summarization, exploratory research and clientcommunication. Hallucinations were most severe and difficult to detect in tasks requiringinterpretation, prioritization and narrative construction from ambiguous input data.