Underground personnel detection and tracking based on improved YOLOv5s and DeepSORT

The real-time monitoring and tracking system of mine moving targets is an essential part of the construction of smart mines. The appearance of downhole inspection robots can realize the real-time monitoring of operators, but the existence of uneven lighting, coal dust interference and other factors lead to the traditional image detection algorithm can not accurately detect operators. Based on this, this paper proposes an improved YOLOv5s and DeepSORT algorithm for downhole personnel detection and tracking that can be deployed in downhole inspection robots. Firstly, the data set was made by using the video recorded by the surveillance camera and inspection robot, and then the improved YOLOv5s network was used to identify the underground personnel: Considering that the detection and tracking algorithm for downhole personnel contains complex network structure and huge parameter volume, which limits the response speed of the detection model, this paper uses an improved lightweight network ShuffleNetV2 to replace the original YOLOv5s backbone network CSP-Darknet53. Meanwhile, in order to reduce the interference of complex image background and improve the attention of operators, Transformer self-attention module is integrated into the ShuffleNetV2. Secondly, the FPN+PAN structure in Neck is replaced by BiFPN structure in order to effectively fuse multi-scale features and effectively transmit inference information. Then, improved DeepSORT was used to encode and track personnel: considering that the underground environment was dark, with low illumination and no texture, it was difficult for DeepSORT to effectively extract personnel's appearance information, so DeepSORT's small and medium residual network was replaced by deeper convolution to enhance DeepSORT's appearance information extraction ability. Finally, the improved algorithm is verified by open pedestrian data set and self-built underground personnel detection and tracking data set. The results show that compared with the original YOLOv5s model, the average detection accuracy of the improved detection model is increased by 5.2%, the number of parameters is reduced by 41%, and the speed is increased by 21%. The improved YOLOv5s-DeepSORT downhole personnel tracking method has a precision of 89.17% and a speed of 67FPS, which can be effectively deployed in downhole inspection robots to realize real-time detection and tracking of operators.