A Human-in-the-Loop Digital Twin Architecture for Real-Time Safety-Control Simulations in Manufacturing Systems

Sammanfattning

Digital twins are increasingly used to simulate and optimize manufacturing systems;however, current implementations are largely machine-centric, excluding human operatorsfrom virtual representation. This results in safety risks, reduced situationalawareness, and limited support for human-machine collaboration central to Industry5.0. This paper presents a human-in-the-loop digital twin architecture thatintegrates real-time human motion data into a simulated manufacturing environmentfor safety monitoring and human-centric production. The architecture spansthree modular layers: a vision-based physical layer, a robust communication layer,and a simulation layer. Human motion is captured via the Occurrence multi-camera3D pose fusion system, detecting 17 joints at 20 Hz, and streamed via MQTT tothe Emulate3D simulation platform. A custom JSON parser and coordinate calibrationpipeline transform incoming pose data into the simulation’s coordinate space,with a dual-protocol fallback strategy ensuring connection reliability across variednetwork configurations. The live pose data is mapped into a hierarchical 14-jointhuman proxy model using a hybrid forward- and inverse-kinematics approach, enablingstable real-time replication of full-body motion. A spatially aware, dual-zonesafety monitoring system computes real-time pelvis-to-conveyor distances using anAxis-Aligned Bounding Box (AABB) model, triggering warning alerts and latchedautomated emergency stops on zone violations. The architecture is demonstratedand validated at the drone assembly workstation in the SII lab at Chalmers Universityof Technology. Results from experiments confirm that all performance targetswere met or exceeded, like the pose update rate of 20 Hz, end-to-end latency of100 ms, etc. This demonstrates the potential of real-time human proxy integrationwith digital twin environments for adaptive safety monitoring and lays a scalablefoundation for human-centric manufacturing