Large Language Model, the New Teammate? : – A qualitative study analyzing the individual perceptions of LLM-tools and the implications of the perceptions on interactions and workflow within software development teams.

Sammanfattning

Problem Definition: The development of artificial intelligent tools has evolved quickly over thepast few years. This is especially visible in the software development industry where large languagemodels (LLM) like Copilot and ChatGPT are becoming common tools in daily work. These toolsuse natural language to communicate and help with different issues, including producing code forsoftware development. The capabilities of LLM-tools enable them to interact in a human-like waywhich can question the perception of LLM as only a tool or as a teammate. However, whether theAI is perceived as a tool or a teammate, this presents implications on the interaction between thehuman and AI-tool which might influence the software development team as well. Purpose: The purpose of this study is to investigate how individuals perceive LLM-tools and howthese perceptions emerge and continually develop. Further, our study will investigate how theseperceptions affect the interaction with LLM-tools and what impact this has on the softwaredevelopment teams’ interactions and workflow. Research Method: This study is a qualitative study with constructivist ontological assumptions andinterpretivist epistemological perspective. An inductive approach has been applied throughout theresearch with an iterative process between inductive and deductive elements. Eight respondents thatwork with software development have been interviewed with a semi-structured interview guide.The data has been analyzed using a thematic analysis approach. Findings: Our empirical findings show that the LLM-tool is perceived as a tool by the respondentsand the perceptions of the capabilities and limitations of the tool influence how they interact with it.This also affects their workflow and brings both positive and negative outcomes. Discussion and Conclusions: The perceptions of the LLM-tools influenced the developers’interaction with it in the way that the limitations of the tool created a low degree of trust andtherefore they used the tool for what they perceive the tool to be good at, namely simple andrepetitive coding tasks. This saved time in their daily tasks which they could use to do other tasksinstead. However, the tool could not be trusted fully and the output always had to be verified toavoid over reliance. The tool was also not perceived as a teammate but in some situations it couldbe perceived as being more than just a tool due to their way of interacting in a human-like way