Establishing a CNN-Based Real-Time Engagement Evaluation Model for Actionable User Feedback

Sammanfattning

In recent years, Human-Computer Interaction (HCI) techniques, notably through gamification, is widely used in educational settings. With latest technological development, Fictive Reality is using Artifical Intellegence (AI), especially Large Language Models (LLMs), combined with computer animation, to create highly interactive avatars that enhances educational and training experiences. Engagement level, which reflects the attention and focus of the learner during educational activities, is used to measure the effectiveness of the learning process. Fictive Reality is hence seeking to develop an actionable feedback model that can provide precise and timely feedback on engagement level during user sessions from the webcam video input. It is a common approach in image analysis to develop a machine learning model based in Convolutional Neural Network (CNN). In this thesis, we propose the development and training of a CNN based model and apply it to actual problems. The e-drishti WACV 2026 dataset is used, and data processing techniques including data augmentation are applied to it. Three models based on CNN architectures are trained on both augmented and unaugmented datasets, including AlexNet, MobileNetV3 and ResNet, using adam optimizer and categorical cross entropy loss function, which altogether six models are obtained. The highest accuracy is 62.9%, and is obtained by the model bosed on AlexNet trained on the unaugmented dataset. The models are then tested on the dataset provided by Fictive Reality. The highest accuracy is 49%, and is achieved by the model based on MobileNetV3 trained on the unaumented dataset. Future work would be to improve the current models, for instance, deepening the neural networks, and apply better data processing techniques.