Explainable ergonomic risk assessment from video : Bridging pose estimation and large language models

Sammanfattning

This thesis presents a pipeline that converts industrial workplace video into explainable ergonomic risk assessments using large language models. MediaPipe pose estimation extracts two-dimensional joint angles from video, rule-based logic computes Rapid Entire Body Assessment scores, and the results are passed to a large language model as structured natural language descriptions. Two models — GPT-4o and Claude Sonnet 4 — were evaluated across three prompt conditions on 49 posture segments from 14 industrial videos, including 18 edge cases where rule-based scoring has limited applicability. The study identifies a fundamental distinction between risk classification — assigning a risk category from kinematic data — and risk identification — independently naming the specific risk factors present. In Condition C1, with kinematics alone, Claude achieved 75.5% agreement with the automated REBA score and identified 59.6% of observer-noted risk factors; GPT-4o reached 30.6% agreement with a systematic upward bias of 2.13 score points and identified 29.6% of the same factors. Both models reached 100% agreement in Condition C3. The classification versus identification gap is the principal cross-model finding and holds across conditions and video sources. Additional findings include an anchoring effect: providing REBA scores without scoring rules caused GPT-4o to adopt the pipeline score directly, reducing data grounding and actionability rather than improving reasoning. Neither model used available exposure duration to qualify extreme angle values. The simulated-to-real alignment gap shows that input quality under real industrial conditions is the binding constraint on deployment, not model capability. The work demonstrates a practical path to automated ergonomic assessment without specialist hardware or domain-specific model training.