Roadway anchor hole recognition and positioning method based on image and point cloud fusion

The accurate identification and positioning of the anchor position of the coal mine tunneling roadway is the key technology that the drilling anchor robot urgently needs to break through to achieve intelligent permanent support. An intelligent identification and positioning method for roadway anchor hole position based on visual image and laser point cloud fusion is proposed, which includes three steps: image target recognition, point cloud image feature fusion, and positioning coordinate extraction: ① In order to solve the problem of blurred image of anchor hole contour caused by environmental factors such as low illumination, water mist and dust in coal mines, the IA (Image-Adaptive)-SimAM-YOLOv7-tiny network was used to visually identify the position of the anchor hole to be anchored in the roadway, which can adaptively enhance the image brightness and contrast, recover the high-frequency information at the edge of the anchor hole, and make the model focus on the characteristics of the anchor hole, so as to improve the success rate of anchor hole detection; ② The Region of Interest (ROI) of image detection is generated by perspective projection relationship to generate a cone-shaped area of interest to obtain the corresponding target point cloud cluster; ③ The point cloud processing algorithm is used to extract the anchor hole boundary point cloud, obtain the central coordinates of the hole position and its normal vector, and judge the correctness of the anchor hole recognition by comparing the coordinate depth difference. In this paper, a drilling and positioning system for bolting trolley manipulator is built to verify the accuracy and accuracy of the algorithm's autonomous positioning, and the experimental results show that the mean average precision (mAP) of the IA-SimAM-YOLOv7-tiny model is 87.3%, which is 4.6% higher than that of the YOLOv7-tiny model. Compared with the single vision method, the fusion algorithm proposed in this paper has a positioning error of 3 mm, and the average recognition time of the system in the case of a single anchor hole is 0.77 s, compared with the single vision method, the fusion of laser and visual multi-source can not only reduce the influence of environment and small sample training on positioning performance, but also obtain the normal vector of the anchor hole position, which provides a basis for the manipulator to adjust the drilling posture to achieve accurate anchoring.