Customizing Conversation: Participatory Design of an LMM-Powered Digital Food List with Older Adults

Sammanfattning

Voice assistants hold promise for supporting independent living among older adults, but traditional voice assistants rely on rigid turn-taking that fails to accommodate natural speech patterns. While recent research employs large multimodal models to enable fluid conversation, most solutions rely on automatic adaptation which risks perpetuating benevolent paternalism, inadvertently infringing upon user agency by making choices for them. Granting older adults’ manual control also introduces a "double-edged sword": satisfying their psychological need for autonomy while introducing extraneous cognitive load. The present study investigates how older adults with varying sensory and physical abilities experience the tension between autonomy and cognitive burden when customizing the conversational behaviors of a voice assistant powered by a large multimodal model. Adopting Participatory Design, this study shifts older adults from passive subjects to active co-creators. Seven older adults participated in interviews and co-design sessions, using Google Stitch and the ChatGPT application as technology probes to ground abstract concepts of AI customization. Findings recast manual customization as a fundamental accessibility requirement, indicating a narrow window for demonstrating basic usability before technology abandonment. This highlights the necessity of multimodal customization enabled through both voice and a graphical user interface to grant older adults a verifiable degree of control.