Learning Whole-body Control for Contact Force and Pose Tracking with a Legged Manipulator

Sammanfattning

This thesis investigates whether deep reinforcement learning can enable a legged manipulator to track end-effector contact forces and poses during rigid-body interaction using only proprioceptive observations. Three whole-body control policies for the 18-degree-of-freedom ALMA-D robot were trained in IsaacSim using a common asymmetric actor-critic architecture with privileged state estimation, differing in how contact forces were represented: generalized external-force emulation, reaction-force-based contact emulation, and explicit rigid-body contact. The policies were evaluated on trapezoidal, sinusoidal, step, and CPR-inspired force profiles with peak forces of 60N across 2,000 end-effector poses, as well as through transfer to a different physics engine, unseen contact geometries and angles, and physical hardware. Compared with generalized force emulation, contact-force emulation reduced force-tracking root-mean-square error by 20.9–69.4%, while training with actual rigid bodies achieved reductions of 59.6–68.2% and performed best on three of the four profiles. The learned behaviors transferred successfully between simulators, although performance generally deteriorated as contact conditions moved further from the training distribution. Hardware experiments demonstrated successful force application but revealed a substantial sim-to-real gap, primarily associated with contact establishment and repeated loss of contact. These results show that physically structured contact-force emulation can substantially improve learned force control, while broader contact distributions and improved contact handling are needed for robust real-world deployment.