Gesture-Based Control for Human-Centric Manufacturing in Industry 5.0

Sammanfattning

Industry 5.0 emphasizes human-centric manufacturing, where digital technologiesshould support operators in industrial work. In this context, interaction methods areneeded that allow operators to communicate with automated assets in a simple andpractical way. This thesis investigates a gesture-based control layer for connectedmanufacturing systems, where camera-based 3D pose estimates are translated intodiscrete command actions and routed to industrial assets.The aim of the work was to design, implement, and evaluate an integration blueprintfor gesture-based human-system interaction in a laboratory manufacturing environment.The system uses a small vocabulary of static upper-body gestures with holdtimeconfirmation to reduce accidental triggering. Recognized gestures are processedby a Python-based gesture module and routed either through ThingWorx and Kepwareor, for selected AMR actions, through a direct ARCL command path. The implementationwas evaluated in a use case involving an OMRON LD-250 AutonomousMobile Robot (AMR), a conveyor, and an Emulate3D-based digital shadow.The final implementation included AMR mission commands, direct AMR stop handling,an optional robot-facing activation condition, and a stop-and-replace workflowthat clears current and queued AMR mission requests after a stop gesture. For theconveyor, the implemented gesture commands included toggle, speed increase, speeddecrease, and quick-stop actions. The digital shadow was used for monitoring andrepeatable observation of AMR behavior, but it was not treated as a full digitaltwin.The system was evaluated through controlled practical trials and a preliminaryuser survey. Under the tested laboratory conditions, the gesture vocabulary wasrecognized as intended, accepted gestures were mapped to the correct commandactions, and the connected assets responded according to the implemented commandlogic. The main limitations were related to the simplified virtual AMR model inEmulate3D, the two-camera setup, pose-estimation jitter, occasional unexplainedresponse-time delays, privacy and data-governance considerations, and the limitednumber of user participants. Overall, the thesis shows that gesture-based control canbe used as a complementary interaction layer for selected manufacturing commands,provided that the sensing conditions, command logic, and safety limitations areclearly defined.