Object Manipulation with Robust Visual Servoing under Human Supervision : Control approaches to automate the construction assembly process

Sammanfattning

In this work, we present a manipulation framework to reliably complete pick and place tasks using mobile bases with robotic arms aiming to enable a multi-robot system to build complex structures using different construction blocks. For picking, an eye-in-hand architecture is used and a visual servoing controller is implemented to ensure successful object tracking. The visual servoing controller is filtered by Control Barrier Functions which ensure fiducial markers in the manipulated object remain visible during approach. For placing, an extra robot with an eye-to-hand architecture is used to ensure the blocks are placed precisely, which is crucial for structural stability. Provisions are also made to allow a human in the control loop to add flexibility and fault correction capabilities to the system. A mathematical analysis of the robustness of the system with respect to errors in the pose of the camera is performed, and changes to the barrier functions are proposed to deal with them. Lastly, experiments to validate the developed framework were carried out in 6-DoF mobile manipulators.