Disparity Visual Servoing

In this work, we introduce a scheme to regulate a mobile robot towards a goal pose by using disparity images. Our proposed scheme contains two stage: a stereo matching pipeline and a controller.

The stereo matching pipeline uses a left stereo image, a right stereo image, a raw disparity image, which is a low-quality disparity image generated by a stereo camera/traditional stereo matching algorithm, as inputs. Then it utilizes CRD-Fusion to generate an occlusion mask and a refined disparity image.

To regulate the mobile robot towards its goal pose, the occlusion mask and refined disparity image at the goal pose are first recorded as reference images. By using the reference images and current occlusion mask and refined disparity, the controller performs occlusion-aware visual servoing by following the direct image-based visual servoing approach (IBVS).

This proposed approch was tested through both simulation and experiments. In the simulation environment, the controller was able to regulate the robot towards its goal pose.

The results obtained on a physical robot and a demo video of this experiment are also shown below.

More details can be found in our book chapter published in Measurements and Instrumentation for Machine Vision and our code.