Perceived Usefulness and Reliability of Generative AI, as an Aid in Learning and Software Testing Activities among Quality Assurance Engineers in the Sri Lankan IT Industry

Sammanfattning

Introduction: Few empirical studies exist on how Sri Lankan QA engineers perceive the usefulness and trustworthiness of generative AI in QA tasks. This thesis examined these perceptions among QA engineers in Sri Lankan IT companies to learn testing concepts, techniques, frameworks, automation and test validation (test design, test data generation, defect analysis). Research Question: The thesis presents one main question with two parts. How do QA engineers in Sri Lankan IT companies perceive the usefulness and reliability of generative AI for: a) learning testing concepts, frameworks, and automation techniques? b) validating test design, test data preparation, and defect analysis? Method: The study adopted a cross-sectional, mixed-method approach that provided both quantitative and qualitative insights. Quantitative data were collected via an online survey of 56 QA engineers based on the widely accepted Technology Acceptance Model (TAM). To gain deeper insights, six semi-structured interviews were conducted with senior QA engineers, providing contextual and experiential insights. The quantitative analysis included descriptive statistics and multiple regression analysis, and the qualitative data were analysed thematically to draw out the insights that were not captured in the statistical results. Results: From the questionnaire responses (N = 56), the perceived usefulness of generative AI for learning tasks (PU-L, M = 3.93) and validation (PU-V, M = 3.93) were high, while its reliability (REL, M = 3.11) was lower. Perceived usefulness during validation has the strongest impact on the intention to use (R² = 0.62). These results are confirmed by the interview findings (n = 6), which indicate that QA professionals have used generative AI for learning support, test design, and test data generation, and that they consistently cross-checked results due to errors and missing context. Discussion: This study contributes empirical insight of the perceived usefulness and reliability in Generative AI, as an aid in Learning and Software Testing Activities among Quality Assurance Engineers in the Sri Lankan IT Industry. The findings answer the research questions by showing that QA engineers perceive generative AI as highly useful for both: a) learning testing concepts, frameworks, and automation techniques; and b) validating test design, test data preparation, and for defect analysis. A significant and somewhat surprising finding is that even when perceptions of reliability are modest, the intention to use generative AI remains high. Instead of blindly relying on AI-generated content, QA engineers have adopted a human verification process in which AI-generated suggestions are carefully reviewed and tested for practical use. However, the study revealed that, while a full sense of trust in the reliability of AI systems is important, so is their perceived usefulness. Conclusion: The survey and interview participants in this study believed that Generative AI was valuable for software testing in learning and validation. a) Learning: QA engineers see generative AI as a tool to help them learn about testing concepts, frameworks, and automation techniques. It helps to enhance learning, but is not a substitute for professional skills. b) Validation: QA engineers feel that generative AI is helpful in test designing, test data preparation, and defect analysis. There are concerns about reliability, but the results indicate that the concerns do not seem to constitute a major obstacle to the continued acceptance of generative AI, as the perceived practical usefulness and the use of human verification also lead to continued acceptance. In summary, Generative AI is valuable to QA engineers, but needs human review to overcome reliability issues.