From Java to Python: Assessing LLM Effectiveness in Automated Test Migration

Sammanfattning

Test suite migration across programming languages is a critical component of code modernization but remains largely manual due to semantic complexity. This thesis evaluates whether large language models can automatically migrate unit tests from Java to Python while preserving semantic equivalence. The study investigates whether pass rate and code coverage serve as reliable proxies for semantic correctness. We systematically analyze 1,249 generated tests across 13 algorithm modules and 254 mapped functions, measuring executability, error patterns, coverage parity, and semantic correctness. Quantitative results show that 95.6% of generated tests are executable (run without runtime errors), with 87.4% having passing assertions. However, despite high executability and pass rates, tests frequently fail to achieve coverage parity with source tests, revealing a decoupling between test executability and semantic fidelity. A taxonomy of 157 execution errors and semantic issues reveals that implementation divergences, language-level semantic differences, and behavioral contract mismatches are the primary failure modes. These findings demonstrate that syntactic correctness alone is insufficient for cross-language test migration and that semantic verification requires explicit cross-language validation. The work establishes empirical evidence of the gap between test executability and semantic equivalence, providing practitioners with guidance on when LLM-based migration is viable and where manual verification is necessary.