INTEGRATING LARGE LANGUAGE MODELS AND VIRTUAL REALITY FOR INTERACTIVE CIRCUIT ANALYSIS

Sammanfattning

This Master thesis explores the integration of Artificial Intelligence (AI) into Virtual Reality (VR)as a tool for interactive learning in electronics education. The work was carried out in collabora-tion with ByBrick and Knightec Group, focusing on creating an AI assistant that helps users buildand troubleshoot 3D electronic circuits in a VR environment. The aim is to investigate whetherlightweight, locally executed AI models could provide feedback and guidance based on user actionsand circuit configurations. The motivation for the work lies in the growing use of VR technolo-gies in education and the increasing demand for intelligent, interactive learning tools that improveunderstanding and engagement, especially in technical subjects like electronics. The thesis invest-igated the process of capturing images from the VR environment and began by investigating smallVisual Language Model (VLM)s for image-based circuit analysis. However, their limited perform-ance led to the investigation of a different approach for an AI assistant using an Large LanguageModel (LLM) combined with You Only Look Once (YOLO) for object detection, and OpenCV forimage processing, to describe the 3D circuit in a text-based Simulation Program with IntegratedCircuit Emphasis (SPICE) netlist. The AI assistant was developed by combining these models tounderstand and analyse user interactions and identify errors to determine if the circuit is closed oropen. Results showed that while smaller VLMs struggled with circuit analysis in images, the com-bination of YOLO, image processing, and LLM, produced a reliable accuracy of identifying errorsin circuits by 86.76%. The final system demonstrated potential for enhancing learning experiencesin VR environments, but it also presents certain limitations