Using Voice and Gesture to Interact : Exploring Multimodal Interaction for Video Editing: A User-Centered Approach to Al Integration

Sammanfattning

Artificial Intelligence (AI) has been developed and utilized to assist in video editing. When people interact with AI, it is common for them to send requests by typing. However, interaction is characteristic of being multi-modal, and multimodal interaction has benefits in the field of human-computer interaction (HCI) [17]. This study aims to research how to design multimodal interaction with voice and gesture to enhance user experiences when editing video with AI on mobile devices. Using a user-centered design methodology, a pre-study involving a survey, and an interview was conducted to explore users’ needs and expectations. Based on the pre-study results, interactive prototypes were designed and tested to collect data with a user experience questionnaire and an interview. The results showed that participants had a positive evaluation of using voice and gesture to interact in the context of video editing with AI. To enhance user experience, the design needs to ensure the familiarity and accessibility of multimodal interaction to use familiar interaction method and accessible for everyone. The design also needs to provide a feeling of control and flexibility to create a seamless multimodal interaction that enables users to select different interaction modes based on their preferences [44]. The design should provide features that enable users to recover from inaccurate recognition and modify the recognition results. This research also suggests using local large language model and making AI transparent in data processing to build trust in AI and improve the user experience of using voice and gesture to edit video through AI.