LLM-Defined BlackBoxFunction in JavaScript/TypeScript : A Prompt-Driven Framework for Automated Validation and Iterative Improvement

Sammanfattning

This thesis explores the reliability of code generated by large language models (LLMs) in dynamically typed programming environments, with a focus on JavaScript and TypeScript. While LLMs are increasingly used to automate coding tasks, their outputs often lack robustness and fail to handle edge cases, especially in one-shot generation scenarios. To address this problem, we propose and implement a framework called BlackBoxFunction, which integrates prompt-driven code generation with automated validation and iterative self-healing. The framework constructs functions via GPT-4.1, validates them using schema-based tests, and automatically regenerates invalid outputs until they pass or reach a retry limit. A controlled experiment was conducted across 20 diverse programming tasks grouped into algorithms, data transformations, and text processing. The results show that self-healing significantly improves functional correctness, raising accuracy from an average of 63% to nearly 100% across all categories, while also reducing the need for manual intervention. These findings highlight the potential of lightweight feedback loops to improve LLM reliability in real-world development workflows.