Yoga Pose Recognition under Occlusion using Region Wise Keypoint Analysis and PoseGCN

Sammanfattning

Accurate yoga pose understanding is essential for applications inhealthcare, rehabilitation, fitness monitoring, and injury prevention.However, reliable pose recognition in real-world scenarios remains challenging due to occlusion and self-overlapping limbs. These chal-lenges often lead to incomplete visual information, reduced keypoint localization accuracy, and degraded pose classification performance.This thesis presents a robust framework for yoga pose estimation and classification under occlusion, with a particular focus on iden-tifying the most critical body keypoints required for reliable pose understanding. The proposed framework consists of two main components: pose estimation and pose classification. In the pose estimation stage, Me-diaPipe Pose is used to extract landmarks from yoga images. To evaluate robustness, region based (fixed) occlusion strategies are ap-plied across six body regions. A region-wise robustness analysis is performed using PCK, JPE, and detection failure rate, along with akeypoint importance analysis to identify the most critical joints forreliable pose understanding. For pose classification, both image-based and skeleton-based ap-proaches are explored. A MobileNetV2-based CNN serves as the base-line image classifier, while structured pose features including land-mark coordinates, joint distances, and angular relationships are used with MLP, Random Forest, and SVM classifiers. To evaluate classifica-tion robustness under occlusion, random occlusions are generated by overlaying rectangular black regions of varying sizes on the raw im-ages. To better capture spatial dependencies between connected joints, this thesis proposes PoseGCN, a Graph Convolutional Network that models the human skeleton as a graph and learns robust joint em-beddings. The extracted PoseGCN embeddings are further combined with geometric features for classification. Experimental results on eight yoga poses demonstrate that the pro-posed PoseGCN-based framework significantly improves robustness under occlusion compared with conventional CNN and classical ma-chine learning methods. The region-wise and keypoint-level analysis further provide interpretable insights into which body regions and joints are most critical for accurate yoga pose recognition. These find-ings contribute toward the development of reliable AI-assisted yoga systems for real-world fitness and healthcare applications.