Exploring Cognitive Processes in AI-Assisted Academic L2 Writing

Sammanfattning

This exploratory study investigates the role of academic writing in a second language (L2) with automated writing evaluation (AWE) tools utilizing generative artificial intelligence (AI), as well as the role of such AI-assistance in cognitive writing processes. The research questions asked if there was a relationship between AWE suggestions and keystroke dynamics, how L2 writers interpreted AWE suggestions and made their decisions, and how AI-assisted writing functions within a cognitive model of writing. Six academics wrote a text in English as L2 using a keystroke logging program. The text was later revised using the AWE program InstaText while their interactions were screen recorded. The recording was then promptly followed by stimulated recall interviews. The keystroke logging data that indicates cognitive demand (e.g., pause length, pause frequency, and deletion frequency) were compared to the AWE suggestion frequency. A thematic analysis was conducted on the interviews to investigate the participants’ metacognitive reflections about their decision-making process. A cognitive model of the writing process was used to map the cognitive processes with the keystroke logging data, AWE interactions, and metacognitive reflections. The findings indicate that longer pauses and fewer typos led to fewer AWE suggestions. Six types of metacognitive reflections were identified in the qualitative data, which was then interpreted together with the quantitative data through a cognitive model of the writing process.