Fine-Tuning AI for Code Generation in IS : A Design Science Evaluation of System Quality and Developer Support.

Sammanfattning

The increasing complexity of software development has amplified the need for automation tools that enhance efficiency and reliability without compromising quality. This study investigates the role of generative Artificial Intelligence in automating JavaScript code generation, applying a Design Science Research (DSR) approach grounded in Information Systems (IS) theory. Using the fine-tuned instruction-following model arvnoodle/hcl-codellama-7b-instruct- javascript-lotuscript-GGUF, this research develops and evaluates an artifact that transforms natural language task descriptions into executable JavaScript code. The artifact’s design and evaluation are informed by key DSR frameworks, including Gregor & Hevner’s knowledge contribution matrix and ISO/IEC 25010:2011 standard (International Organization for Standardization, 2011) system quality metrics. Empirical analysis compares the fine-tuned model to its base LLaMA 7B version, assessing accuracy, reliability, token richness, and generation speed through reproducible testing with real-world prompts and visualizations. Results show that the fine-tuned model achieves 100% task accuracy and improved syntax validity, while maintaining efficient performance. The study also examines limitations in generalizability, training bias, and infrastructure scalability. Ethical considerations—such as trust, oversight, and security risks in AI-generated code—are addressed to promote responsible use. This research contributes to the IS field by presenting a validated generative AI artifact, an evaluation framework tailored for automated code generation, and theoretical insights into the socio-technical implications of AI integration in development workflows.