Leveraging LLMs and Behavior Trees for Understanding User Instructions

Sammanfattning

Large Language Models (LLMs) have created new opportunities in Human- Robot Interaction (HRI), particularly in natural language understanding and processing. However, systems using LLMs in HRI face challenges such as unpredictability due to hallucinations and often lack explainability for their actions. To address these issues, in this thesis, we use a framework that combines Behavior Trees (BTs) with LLMs to process and act upon user instructions. This BT-LLM framework aims to disambiguate user requests, improve explainability, and reduce errors like hallucinations, creating a more reliable and trustworthy system that enhances user satisfaction. For comparison, a baseline system used a single long pre-prompt with ChatGPT-4o-mini, while the BT-LLM system used the modular structure of BTs. This led to fewer errors and better consistency in processing user instructions. The systematic approach of BTs also enabled clearer and more structured explanations of the robot’s behavior, by breaking down decisions into manageable steps and explicitly addressing ambiguity. The framework was tested in a kitchen setting, where Furhat, a conversational robot head, functioned as a home kitchen assistant manager. A user study with 49 participants evaluated the framework using both objective and subjective measures. Statistical analysis showed that the BT- LLM framework significantly reduced task errors compared to the baseline. Additionally, 73.2% of participants preferred the BT-LLM framework over the baseline system. With these results, this thesis contributes to HRI by demonstrating that structured approaches to disambiguating user instructions can lead to more predictable and reliable robot behavior. The proposed framework not only reduces LLM errors like hallucinations but also provides explanations for robot decisions through its systematic handling of ambiguous requests. By showing how to balance natural language understanding with explainable behavior, this thesis creates a basis for more effective and satisfying human- robot interaction, showing promising results for applications like kitchen assistance.