3D Object Detection With A Low-cost Depth Camera

Keyword: Computer Vision, Machine Learning, Navigation
In autonomous driving, object recognition primarily relies on infrared cameras. 3D object detection frameworks such as MV3D and BtcDet applied LiDAR-and-camera-based 3D object detection technology to autonomous driving. However, autonomous vehicles need to be equipped with very expensive LiDAR and camera sensors to collect high quality point cloud data and RGB images of the surrounding environment. Under HCI scenarios, we would like to build a low-cost hardware system and try using the existing network to accomplish the target detection task. We believe this will help to extend this target detection technology to low-cost small robots and smart devices for tasks such as cycling navigation, indoor navigation, and navigation for the blind.

We first replicated BtcDet with reference to the paper and conducted model training and preliminary testing on KITTI dataset. Thereafter, we built a low-cost bicycle-loaded data collection prototype to collect road condition data in Tsinghua campus. We have migrated BtcDet to our task and built a functional prototype system. Although the difference in point cloud datasets obtained by LiDAR and our own depth cameras has led to a certain decrease in performance, we have gained a better understanding of camera calibration and gained more experience in model migration tasks through the debugging process.