| Citation: |
ZHU Yantong，LIU Peng，WANG Yongbo，et al. Multi-sensor fusion-based intelligent supervision system for coal storage yard operation safety[J]. Coal Science and Technology，2024，52（S2）：331−342. DOI: 10.12438/cst.2023-1675
|
Workers and vehicles are the key objects of coal yard safety management. It is of great significance to obtain high-precision positioning information in real time to avoid safety accidents. Studying the detection and positioning method of personnel and vehicle based on ultra-WB and visual integration, according to the scene characteristics of the closed coal yard, the method based on flight time-time arrival difference is proposed to realize the high-precision positioning of label targets in the coal yard; In order to solve the interruption of UWB location caused by occlusion, UWB detection and identification method based on Transformer and Yolov7 is proposed to estimate the position in the image, and the Kalman filter method is used to realize the fusion positioning. This method was tested in the coal yard of Ningxia Lingwu Power Plant of National Energy. By collecting the data images of operation scenes in the coal field, the data set of operators and vehicles was made. The average accuracy index of the target detection model mAP@0.5 is 0.925, and the image detection speed of the model deployed on the site is 30 frames per second. The experimental results of the detection model show that the model has high detection accuracy and fast detection speed for the main motion targets in the coal storage field, and meets the actual requirements of the coal field application in real time. The comparative experimental results show that the proposed target detection model improves the mAP@0.5 by 1.4% over the original Yolov7 without sacrificing the processing frame rate. The positioning accuracy range in the coal yard is 0.5～0.9 m, and the positioning refresh frequency is 10 frames per second, which effectively solves the problem of UWB positioning interruption caused by occlusion.
| [1] |
武文忠,蒋华,陈良,等. 大型火力发电厂超大跨度煤场封闭施工的工艺实践[J]. 电力学报,2023,38(2):127−135.
WU Wenzhong,JIANG Hua,CHEN Liang,et al. Construction method of large span closed coal yard structure in thermal power plant[J]. Journal of Electric Power,2023,38(2):127−135.
|
| [2] |
杨凯. 露天煤矿机动车辆事故分析及预防措施[J]. 露天采矿技术,2017,32(12):85−87,90.
YANG Kai. Accident analysis and preventive measures of motor vehicle in open-pit coal mine[J]. Opencast Mining Technology,2017,32(12):85−87,90.
|
| [3] |
李航天. 2021年全国生产安全事故十大典型案例[N]. 中国应急管理报,2022−01−21(004).
LI Hangtian. Top ten typical cases of production safety accidents in China in 2021[N]. China Emergency Management News,2022−01−21(004).
|
| [4] |
苏素燕,陈金旺,王林芳,等. 室内外定位技术综述[J]. 智能计算机与应用,2023,13(10):179−183. doi: 10.3969/j.issn.2095-2163.2023.10.035
SU Suyan,CHEN Jinwang,WANG Linfang,et al. A review of indoor and outdoor positioning technology[J]. Intelligent Computer and Applications,2023,13(10):179−183. doi: 10.3969/j.issn.2095-2163.2023.10.035
|
| [5] |
曹波,王世博,葛世荣,等. 基于校准补偿和VBUKF平滑的超宽带采煤机定位技术[J]. 煤炭科学技术,2022,50(8):210−219.
CAO Bo,WANG Shibo,GE Shirong,et al. Ultra-wideband shearer positioning technology based on calibration compensation and VBUKF smoothing[J]. Coal Science and Technology,2022,50(8):210−219.
|
| [6] |
ZHAO Z Q,ZHENG P,XU S T,et al. Object detection with deep learning:A review[J]. IEEE Transactions on Neural Networks and Learning Systems,2019,30(11):3212−3232. doi: 10.1109/TNNLS.2018.2876865
|
| [7] |
ZOU Z X,CHEN K Y,SHI Z W,et al. Object detection in 20 years:A survey[J]. Proceedings of the IEEE,2023,111(3):257−276. doi: 10.1109/JPROC.2023.3238524
|
| [8] |
GIRSHICK R,DONAHUE J,DARRELL T,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway,NJ:IEEE,2014:580−587.
|
| [9] |
HE K M,ZHANG X Y,REN S Q,et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[M]//Computer vision–ECCV 2014. Cham:Springer International Publishing,2014:346−361.
|
| [10] |
REN S Q,HE K M,GIRSHICK R,et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137−1149. doi: 10.1109/TPAMI.2016.2577031
|
| [11] |
LIN T Y,DOLLÁR P,GIRSHICK R,et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway,NJ:IEEE,2017:936−944.
|
| [12] |
WANG C Y, BOCHKOVSKIY A, LIAO H M. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[EB/OL]. （2022−07−06） [2023−12−14]. https://arxiv.org/abs/2207.02696v1.
WANG C Y,BOCHKOVSKIY A,LIAO H M. YOLOv7:Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors[EB/OL]. (2022−07−06) [2023−12−14]. https://arxiv.org/abs/2207.02696v1.
|
| [13] |
LAW H,DENG J. CornerNet:Detecting objects as paired keypoints[C]// Computer Vision – ECCV 2018. Cham:Springer International Publishing,2018:765−781.
|
| [14] |
ZHOU X Y, WANG D Q, KRÄHENBÜHL P. Objects as points[EB/OL]. （2019−04−25） [2023−12−14]. https://arxiv.org/abs/1904.07850v2.
ZHOU X Y,WANG D Q,KRÄHENBÜHL P. Objects as points[EB/OL]. (2019−04−25) [2023−12−14]. https://arxiv.org/abs/1904.07850v2.
|
| [15] |
DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[EB/OL]. （2019−05−24） [2023−12−14]. https://arxiv.org/abs/1810.04805v2.
DEVLIN J,CHANG M W,LEE K,et al. BERT:Pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2019−05−24) [2023−12−14]. https://arxiv.org/abs/1810.04805v2.
|
| [16] |
CARION N,MASSA F,SYNNAEVE G,et al. End-to-end object detection with transformers[C]// Computer Vision – ECCV 2020. Cham:Springer International Publishing,2020:213−229.
|
| [17] |
ZHU X Z, SU W J, LU L W, et al. Deformable DETR: Deformable transformers for end-to-end object detection[EB/OL]. （2021−03−18） [2023−12−14]. https://arxiv.org/abs/2010.04159v4.
ZHU X Z,SU W J,LU L W,et al. Deformable DETR:Deformable transformers for end-to-end object detection[EB/OL]. (2021−03−18) [2023−12−14]. https://arxiv.org/abs/2010.04159v4.
|
| [18] |
LI F,ZHANG H,LIU S L,et al. DN-DETR:Accelerate DETR training by introducing query DeNoising[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway,NJ:IEEE,2022:13609−13617.
|
| [19] |
XIAO J S,GUO H W,ZHOU J,et al. Tiny object detection with context enhancement and feature purification[J]. Expert Systems with Applications,2023,211:118665. doi: 10.1016/j.eswa.2022.118665
|
| [20] |
JUAN T, DIANA M C. A comprehensive review of YOLO: From YOLOv1 to YOLOv8 and beyond[EB/OL]. （2023−04−02） [2023−12−14]. https://arxiv.org/abs/2304.00501.
JUAN T,DIANA M C. A comprehensive review of YOLO:From YOLOv1 to YOLOv8 and beyond[EB/OL]. (2023−04−02) [2023−12−14]. https://arxiv.org/abs/2304.00501.
|
| [21] |
LIU Z,LIN Y T,CAO Y,et al. Swin transformer:Hierarchical vision transformer using shifted windows[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway,NJ:IEEE,2021:9992−10002.
|
| [22] |
CANO J,PAGÈS G,CHAUMETTE É,et al. Clock and power-induced bias correction for UWB time-of-flight measurements[J]. IEEE Robotics and Automation Letters,2022,7(2):2431−2438. doi: 10.1109/LRA.2022.3143202
|
| [23] |
VAN HERBRUGGEN B,FONTAINE J,DE POORTER E. Anchor pair selection for error correction in time difference of arrival (TDoA) ultra wideband (UWB) positioning systems[C]//2021 International Conference on Indoor Positioning and Indoor Navigation (IPIN). Piscataway,NJ:IEEE,2021:1−8.
|
| [24] |
谢宗武,马博宇,蒋再男,等. 面向空间机械臂遥操作的虚拟相机成像技术[J]. 哈尔滨工程大学学报,2023,44(4):631−638.
XIE Zongwu,MA Boyu,JIANG Zainan,et al. Imaging of virtual camera for space manipulator teleoperation[J]. Journal of Harbin Engineering University,2023,44(4):631−638.
|
| [25] |
王谭,王磊磊,张卫国,等. 基于张正友标定法的红外靶标系统[J]. 光学 精密工程,2019,27(8):1828−1835. doi: 10.3788/OPE.20192708.1828
WANG Tan,WANG Leilei,ZHANG Weiguo,et al. Design of infrared target system with Zhang Zhengyou calibration method[J]. Optics and Precision Engineering,2019,27(8):1828−1835. doi: 10.3788/OPE.20192708.1828
|
| [26] |
刘燕,张健,肖庆高,等. UKF与EKF在导航定位中的对比研究[J]. 微处理机,2023,44(4):30−33. doi: 10.3969/j.issn.1002-2279.2023.04.009
LIU Yan,ZHANG Jian,XIAO Qinggao,et al. A comparative study of UKF and EKF in navigation and positioning[J]. Microprocessors,2023,44(4):30−33. doi: 10.3969/j.issn.1002-2279.2023.04.009
|
| [27] |
EVENSEN G. Sequential data assimilation with a nonlinear quasi-geostrophic model using Monte Carlo methods to forecast error statistics[J]. Journal of Geophysical Research:Oceans,1994,99(C5):10143−10162. doi: 10.1029/94JC00572
|
| [28] |
GE Z, LIU S T, WANG F, et al. YOLOX: Exceeding YOLO series in 2021[EB/OL]. （2021−07−18） [2023−12−14]. https://arxiv.org/abs/2107.08430v2.
GE Z,LIU S T,WANG F,et al. YOLOX:Exceeding YOLO series in 2021[EB/OL]. (2021−07−18) [2023−12−14]. https://arxiv.org/abs/2107.08430v2.
|
| [29] |
JOCHER G. YOLOv5 release v6.1[EB/OL]. （2022−02−22） [2023−12−14]. https://github.com/ultralytics/yolov5/releases/tag/v6.1.
JOCHER G. YOLOv5 release v6.1[EB/OL]. (2022−02−22) [2023−12−14]. https://github.com/ultralytics/yolov5/releases/tag/v6.1.
|
| [30] |
WANG C Y, YEH I H, LIAO H M. You only learn one representation: Unified network for multiple tasks[EB/OL]. （2021−05−10） [2023−12−14]. https://arxiv.org/abs/2105.04206v1.
WANG C Y,YEH I H,LIAO H M. You only learn one representation:Unified network for multiple tasks[EB/OL]. (2021−05−10) [2023−12−14]. https://arxiv.org/abs/2105.04206v1.
|
| [31] |
XU S L, WANG X X, LV W Y, et al. PP-YOLOE: An evolved version of YOLO[EB/OL]. （2022−05−30） [2023−12−14]. https://arxiv.org/abs/2203.16250v3.
XU S L,WANG X X,LV W Y,et al. PP-YOLOE:An evolved version of YOLO[EB/OL]. (2022−05−30) [2023−12−14]. https://arxiv.org/abs/2203.16250v3.
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: