Evaluating the Use of Generative AI in Software Development Proposing a Tentative Framework

Sammanfattning

Artificial intelligence (AI), especially generative AI (GenAI), has significantlyimpacted society and industries, driven by its ability to generate original content fromdata patterns. Recent advancements, like ChatGPT, have highlighted GenAI's potentialto boost productivity in software development. However, its rapid adoption also bringsrisks, such as job displacement. This underscores the need to explore both theopportunities and challenges GenAI offers, particularly its impact on professional rolesand processes in software development.This Master's thesis aims to explore the use of generative AI within a software companyin two ways. The first question revolves around understanding employees' perceptionsof the use of generative AI related to their work within the company, by exploringopportunities and challenges it presents across various departments. The secondquestion focuses on how to evaluate the use of generative AI in software development.The study was conducted as a qualitative case study at the software company Zenseact,where data was collected through interviews which enabled a detailed andcomprehensive analysis of the opportunities and challenges GenAI entails. Findingsindicate a general tendency to embrace generative AI, with varying degrees ofenthusiasm and skepticism shaped by personal and professional views. Despiteconcerns about security and privacy the sentiment towards GenAI remains positive.Findings recognize opportunities to boost efficiency and creativity by automating tasks,improving information processing, and enhancing learning. Challenges include thereliability of GenAI outputs, ethical considerations, job market effects, and reskillingneeds. Evaluating GenAI in software development involves both quantitative metricslike time savings and productivity, and qualitative measures such as user satisfactionand perceived value, to fully assess its impact and utility. Previous literature suggeststhat software development will be revolutionized through AI-generated code, however,the findings of this study imply that GenAI could have a larger impact on other parts ofthe software development process.