Satisfactory Performance of Text-Generative AI Compared to Human-Written Content for Websites in Digital Marketing

Sammanfattning

This thesis explores the impact of the usage of text-generative artificial intelligence (AI) in digital marketing on user satisfaction. Recently, concerns regarding job displacement and human expertise arose due to the efficiency and improved workflow provided by AI-powered tools. This study addresses these concerns by evaluating whether ChatGPT 3.5 is able to generate website texts with minimal human supervision while maintaining user satisfaction. Our investigation employs a mixed approach of qualitative and quantitative research, utilising controlled experiments with 14 participants aged between 20 and 31 to compare AI-generated texts with human-written texts. The controlled experiment included two identically looking prototypes, one containing human-written texts and the other containing texts generated by ChatGPT 3.5. Both prototypes had three different pages: Home, Joining and About. Additionally, two types of surveys were created, a Satisfactory Survey for each prototype and a Final Survey. The Satisfactory Survey contained Likert scales from one (1) to five (5) which enabled participants to rank the texts together with open-ended questions. The Final Survey included questions about demographics and an overall prototype preference. Having tested texts on the three different pages in each prototype on satisfaction, informativeness and appeal, the biggest difference was found in the satisfaction of the individual pages. While participants preferred human-written texts on the Home and the About page, they favoured AI-generated texts on the Joining page. Findings suggest that ChatGPT 3.5 can, with minimal human supervision, produce texts of nearly equally good satisfaction from a user perspective compared to texts written by humans. The study underscores the importance of human oversight and expertise in optimising AI-generated outputs and contributes to the ongoing discourse on integrating AI into marketing practices.