AI-Based Synthetic Users for Early-Stage UI/UX Validation

Sammanfattning

User Experience (UX) validation is an important step in digital product design, yet traditional human-led testing is often too resource-heavy to conduct during the early conceptual design phases. While "Shift-Left" economics heavily prefer early validation, existing automated solutions either rely on coded environments or utilize fully autonomous AI agents that generate machine-readable, unexplainable feedback, leading to a lack of designer trust. This thesis addressed these limitations by exploring how Generative Artificial Intelligence (GenAI) could be utilized to conduct early-stage usability testing. The aim was to design and evaluate an AI-based synthetic user framework capable of evaluating static UI/UX prototypes while prioritizing behavioral realism and designer trust over total autonomous scale. Following a Design Science Research (DSR) methodology, the artifact was iteratively developed over three prototype phases. Essentially, this prototype was developed not as a means initself but as a means to an end; i.e., it served as an exploratory framework to determine the extent to which GenAI can reliably execute UX validation. To prevent the Large Language Model (LLM) from defaulting to a generic technical assistant, the framework utilized parameter-free optimization. Specifically, advanced prompt engineering architectures (Intent, Domain, and Demand constraints combined with Chain-of-Thought) were combined with the 15-facet BFI-2 psychometric framework to force the AI to simulate subjective human traits and output actionable cognitive friction. Evaluation with a single domain expert, who also collaborated in the framework’s design, suggested that while synthetic users can successfully identify valid usability issues and build designer trust through transparent reasoning, they still show an inherent "rationality bias" and cannot fully replicate human unpredictability. The findings concluded that GenAI shows high potential as an early-stage, human-in-the-loop decision-support tool, but must complement rather than replace real human testing.