ClariTest: An LLM-Based Tool for Context-Aware Test Case Readability Improvement - Utilizing Prompt Structure and RAG-Augmented Large Language Models to Improve the Readability of Generated Test Suites While Preserving Test Semantics

Sammanfattning

Automated software test generation tools are highly effective at achieving high codecoverage. However, these tests usually have poor readability, generic identifiers andunclear logic. This lack of readability hinders developers from properly understanding the code, reducing developers trust and increasing long-term maintainabilitycosts. Large Language Models (LLMs) offer a promising solution to automaticallyrefactor and document code, but their usage is often hindered by “semantic drift”.Semantic drift refers to instance where the model hallucinates and unintentionally alters the tests intended behavior. To bridge this gap between machine generated codeand human comprehension, this study introduces ClariTest, an automated refactoring tool that leverages Retrieval Augmented Generation (RAG) and promptingto enhance readability of Python test suites without altering their intended behavior. ClariTest was evaluated using a subset of Pynguin-generated test cases.This study evaluates four LLMs, specifically, GPT-5-Mini, Claude Sonnet 4.6,Deepseek-Coder-33b-Instruct and Deepseek-V3.2, and two prompting strategies, Base prompt including the Direct-Instruction prompting technique and a Combined Prompt composed of Direct Instruction, Chain-of-Thought and Self-Critique.Additionally, the study investigated the impact of model temperature (by comparing 1.0 and 0.2) to evaluate the trade off between allowing higher creativity andenforcing a more deterministic behavior. To evaluate the performance of these configurations, this study employed a systematic quantitative assessment by measuringthe change in semantic drift indicators (code coverage, test passing/failing rates andtotal tests) and code naturalness via the Dependency-Aware Naturalness (DAN)metric. Additionally, a qualitative survey of software developers was conducted toassess the perceived readability of the refactored test suites. Regarding the temperature evaluation, the results revealed a trade off, while a lower temperature (0.2)generally saw enhanced improvements to code naturalness compared to temperature 1.0, however, it also led to decreased stability across most models, illustratedby an increasing standard deviation in semantic drift metrics. In comparison tothis, a higher temperature (1.0) avoided this decrease in stability, resulting in moreconsistent semantic preservation across the test suites. The quantitative resultsidentified GPT-5-Mini with the Base Prompt (Direct-Instruction) as the most optimal configuration, as this configuration successfully maximized code naturalnesswhile minimizing semantic drift. The qualitative survey illustrated that readabilityimproved according to 96.88% of the participants. Furthermore, the data from thesurvey highlighted that the introduction of meaningful identifiers and explanatoryinline comments were the most critical enhancements. Ultimately, the study demonstrates that when properly constrained by RAG and prompting, LLMs can reliablytransform machine generated tests into maintainable, human readable code.