LLM-based Log Analysis for Fault Localization in the Automotive Industry

Sammanfattning

This thesis investigates the application of large language models (LLMs) to aidpractitioners of log analysis for fault localization in the automotive industry. Anexisting LLM-based log summarization tool is extended and evaluated, focusing onthe cognitive load of practitioners and how satisfied they are with the tool. The effectof LLM-based log summarization on productivity of practitioners in the automotiveindustry is investigated through a case study at a company within the automotiveindustry. Think-aloud sessions and semi-structured interviews are carried out toasses the impact of the tool on the fault localization flow of study participants.Results suggest that LLM-generated log summaries can aid practitioners by givingthem a first glance of the issue, thereby potentially reducing manual effort andimproving productivity. However, the results also suggest that the context of theissue, domain knowledge, and interactivity of the tool plays a major role for success.A lack of context and means for the practitioner to guide the tool could result ina less effective workflow with higher cognitive load. The thesis provides insights onthe integration of LLM-based log analysis tools within fault localization workflowsin the industry, highlighting both the benefits and challenges of deploying LLMs inreal-world fault analysis scenarios