LMIENet enhanced object detection method for low light environment inunderground mines
-
摘要:
煤矿井下工作环境复杂，存在人造光源亮度低、粉尘多和水气密度大等不利因素，导致现有的目标检测算法在应用到煤矿井下时，存在提取特征困难、目标识别和定位精度低等问题。提出一种煤矿井下低照度环境目标检测算法，由矿井低光图像增强模块LMIENet和目标检测模块组成，使用图像增强模块对原始图像进行画质提升，恢复各类图像信息，再使用目标检测网络对增强图像进行特定目标检测，有效提高检测的精确度。在图像增强模块中，改进Zero-DCE算法设计轻量级增强参数预测网络，计算像素级增强参数矩阵，用于低光照图像的亮度调整和画质增强，该网络通过设计的非参考损失函数隐性衡量图像的增强效果，引导网络进行无监督学习，使网络能够不依赖配对数据集对原始图像进行自适应的画质增强。目标检测模块中，采用YOLO v8n目标检测模型，其轻量化的模型尺寸和高灵活性可避免模型整体复杂度过高；采用Focal-EIoU Loss改进回归损失函数，有效加速模型收敛并提升模型检测精度。实验结果显示：与经典目标检测算法Faster R–CNN,SSD, RetinaNet, FCOS等相比，提出算法在自建矿井人员数据集上表现出色，低光照环境下目标检测的mAP@0.5达到98.0%，mAP@0.5∶0.95达64.8%，在实验环境中单帧图像推理时间仅11 ms，优于其他对比方法，证明提出算法能够有效实现在煤矿井下低照度复杂环境下的目标检测，且耗时短、计算效率高。
Abstract:The underground working environment of coal mines is complex, with unfavorable factors such as low brightness of artificial light sources, high dust content, and high water vapor density. This leads to difficulties in extracting features and low accuracy of object recognition and positioning when existing object detection algorithms are applied to coal mines. An object detection algorithm for low illumination environments in coal mines is proposed, which consists of an low-light mine image enhancement module LMIENet and a object detection module. The image enhancement module is used to improve the image quality of the original image, restore various image information, and then use a target detection network to perform specific target detection on the enhanced image, effectively improving the accuracy of detection. In the image enhancement module, a lightweight enhancement parameter prediction network is designed with reference to the zero reference depth curve estimation algorithm, and the pixel level enhancement parameter matrix is calculated for image quality enhancement and brightness adjustment of low light images. The network implicitly measures the image enhancement effect through the designed non-reference loss function, and guides the network to conduct unsupervised learning, Enable the network to adaptively enhance the image quality of the original image without relying on paired datasets. In the object detection module, the YOLO v8n object detection model is adopted, which has a lightweight model size and high flexibility to avoid excessive overall model complexity; Using Focal EIoU Loss to improve regression loss, accelerate model convergence, and improve model detection accuracy. The experimental results show that compared with classic object detection algorithms such as Faster R–CNN, SSD, RetinaNet, etc., the proposed algorithm performs well on the self-made coal mine object detection dataset, and is effective in object detection in low light environments mAP@0.5 reaches 98.0%, mAP@0.5∶0.95 reaches 64.8%, and the running time of a single frame image in the experimental environment is only 11 ms, which is superior to other comparison methods. It is proven that the proposed algorithm can effectively achieve object detection in low illumination and complex environments in coal mines, with short time consumption and high computational efficiency.
-
-
![]()
图 1 井下低照度环境目标检测模型整体结构
Figure 1. Overall structure of underground low illumination environment object detection model
![]()
图 10 目标检测对比实验部分可视化结果
Figure 10. Visualization results of object detection comparison experiment
![]()
图 11 图像增强与目标检测部分实验结果
Figure 11. Experimental results of image enhancement and object detection section
表 1 自建矿井人员数据集统计
Table 1 Statistics of UMPDDC dataset
类别 标签数量 场景 图片数量 作业人员 11523 运输巷设备列车 1289 托辊(左/右) 7 489/ 10504 传送带区、转载区 2391 安全头盔 11545 副井、工作活动区 3020 总计 41061 总计 6700 
下载: 导出CSV
表 2 目标检测性能指标评价
Table 2 Evaluation of object detection performance indicators
方法 Backbone mAP@0.5/% mAP@0.75/% mAP@0.5∶0.95/% FPS/(帧·s−1) Faster R–CNN[26] ResNet-50 95.3 58.5 56.0 16.4 SSD300[6] VGG16 94.0 53.3 52.9 52.2 RetinaNet[7] ResNet-50 95.8 59.6 56.9 17.7 FCOS[27] ResNet-50 95.2 56.2 54.9 19.0 CenterNet[28] ResNet-18 94.9 54.6 54.2 63.1 YOLOv3–Tiny[29] DarkNet53 95.4 60.4 58.7 188.7 YOLO v5n CSP–DarkNet53 96.8 67.0 61.1 117.7 YOLO v8n New CSP–DarkNet53 97.1 67.7 61.9 126.6 本文方法 — 98.0 72.9 64.8 90.9 注：mAP@0.5为指在IoU阈值为50%时的平均精度；mAP@0.75为在IoU阈值为75%时计算的平均精度；mAP@0.5∶0.95为依据指定的步长(5%)，在IoU阈值从50%～95%变化范围内所有取值的mAP的均值。下同。
下载: 导出CSV
表 3 LMIENet与主流图像增强算法在目标检测任务中的性能对比
Table 3 Performance comparison between LMIENet and mainstream image enhancement algorithms in object detection tasks
检测网络 增强网络 mAP@0.5/% mAP@0.75/% mAP@0.5∶0.95/% RT/s 改进YOLO v8n
(Focal-EIoU Loss)LIME 97.3 70.0 62.9 0.499 6 MBLLEN 96.2 58.2 56.7 14.002 8 RetinexNet 94.4 55.3 55.1 0.128 5 Zero-DCE 97.3 69.1 62.2 0.011 3 SCI 97.9 72.0 64.4 0.010 5 LMIENet 98.0 72.9 64.8 0.011 0
下载: 导出CSV
表 4 LMIENet在其他目标检测网络上的验证
Table 4 Verification of LMIENet on other target detection networks
方法 mAP@0.5/% mAP@0.75/% mAP@0.5∶0.95/% Faster-RCNN 95.3 58.5 56.0 Faster-RCNN+LMIENet 95.5 59.1 56.6 RetinaNet 95.8 59.6 56.9 RetinaNet+LMIENet 96.1 60.4 57.2 FCOS 95.2 56.2 54.9 FCOS+LMIENet 95.7 57.8 55.9
下载: 导出CSV
表 5 消融实验结果
Table 5 Results of ablation experiment
YOLO v8n LMIENet Focal-EIoU mAP@0.5/% mAP@0.75/% mAP@0.5:0.95/% √ 97.1 67.7 61.9 √ √ 98.0 72.7 64.4 √ √ 97.0 69.0 62.6 √ √ √ 98.0 72.9 64.8 注：“√”代表使用了该方法。
下载: 导出CSV
-
[1] 智 宁,毛善君,李 梅. 基于照度调整的矿井非均匀照度视频图像增强算法[J]. 煤炭学报,2017,42(8):2190−2197. ZHI Ning,MAO Shanjun,LI Mei. Video image enhancement al-gorithm of mine non-uniform illumination based on illumination adjustment[J]. Journal of China Coal Society,2017,42(8):2190−2197.
[2] 陈德强,钱建生,郭星歌,等. 煤矿安全生产视频AI识别关键技术研究综述[J]. 煤炭科学技术,2023,51(2):349−365. CHENG Deqiang,QIAN Jiansheng,GUO Xingge,et al. Review on key technologies of AI recognition for videos in coal mine[J]. Coal Science and Technology,2023,51(2):349−365.
[3] KRIZHEVSKY A,SUTSKEVER I,GEOFFREY E H. Imagenet classification with deep convolutional neural networks[J]. Advances in Neural Information Processing Systems,2012,25:1097−1105.
[4] GIRSHICK R,DONAHUE J,DARRELL T,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2014:580–587.
[5] REDMON J,DIVVALA S,GIRSHICK R,et al. You only look once:Unified,real-time object detection[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:779–788.
[6] LIU W,ANGUELOV D,ERHAN D,et al. Ssd:Single shot multibox detector[C]. European Conference on Computer Vision,Springer,2016:21–37.
[7] LIN T Y,GOYAL P,GIRSHICK R,et al. Focal loss for dense object detection[C]. Proceedings of the IEEE International Conference on Computer Vision,2017:2980–2988.
[8] IGOR M,CHEN Y,LIN Y,et al. NOD:Taking a closer look at detection under extreme low-light conditions with night object detection dataset[C]. BMVC,2021.
[9] 龚 云,颉昕宇. 基于同态滤波方法的煤矿井下图像增强技术研究[J]. 煤炭科学技术,2023,51(3):241−250. GONG Yun,XIE Xinyu. Research on coal mine underground image recognition technology based on homomorphic filtering method[J]. Coal Science and Technology,2023,51(3):241−250.
[10] 苏 波,李 超,王 莉. 基于多权重融合策略的Retinex矿井图像增强算法[J]. 煤炭学报,2023,48,(S2):813−822. SU Bo,LI Chao,WANG Li. Mine image enhancement algorithm based on retinex using multi-weight fusion strategy[J]. Journal of China Coal Society,2023,48,(S2):813−822.
[11] 李 曼,杨茂林,刘长岳,等. 基于图像的煤矸分选中图像照度调节方法[J]. 煤炭学报,2021,46(S2):1149−1158. LI Man,YANG Maolin,LIU Changyue,et al. Illuminance adjustment method for image-based coal and gangue separation[J]. Journal of China Coal Society,2021,46(S2):1149−1158.
[12] 乔佳伟,贾运红. Retinex算法在煤矿井下图像增强的应用研究[J]. 煤炭技术,2022,41(3):193−195. QIAO Jiawei,JIA Yunhong. Research on Application of Retinex Algorithm in Image Enhancement in Coal Mine[J]. Coal Technology,2022,41(3):193−195.
[13] WANG R,ZHANG Q,FU C,et al. Underexposed photo enhancement using deep illumination estimation[C]. IEEE Conference on Computer Vision and Pattern Recognition,2019:6849–6857.
[14] KIM H,CHOI S,KIM C,KOH Y J. Representative Color Transform for Image Enhancement[C]. The IEEE/CVF International Conference on Computer Vision,2021.
[15] ZHAO L,LU S,CHEN T,et al. Deep symmetric network for underexposed image enhancement with recurrent attentional learning[C]. The IEEE/CVF International Conference on Computer Vision,2021.
[16] 田子建,吴佳奇,张文琪,等. 基于Transformer和自适应特征融合的矿井低照度图像亮度提升和细节增强方法[J]. 煤炭科学技术,2024,52(1):29−310. TIAN Zijian,WU Jiaqi,ZHANG Wenqi,et al. An illuminance improvement and details enhancement method on coal mine low-light images based on Transformer and adaptive feature fusion[J]. Coal Science and Technology,2024,52(1):297−310.
[17] KIM I S,JEONG Y,KIM S H,et al. Deep Learning based Effective surveillance system for low-illumination environments[C]. International Conference on Ubiquitous and Future Networks,2019.
[18] SASAGAWA Y,NAGAHARA H. YOLO in the Dark – Domain adaptation method for merging multiple models[A]// European Conference on Computer Vision,2020.
[19] QU Y,OU Y,XIONG R. Low illumination enhancement for object detection in self-driving[A]// IEEE International Conference on Robotics and Biomimetics,2019.
[20] WANG W,PENG Y,CAO G,et al. Low-illumination image enhancement for night-time UAV pedestrian detection[J]. IEEE Transactions on Industrial Informatics,2021,17(8):5208−5217. doi: 10.1109/TII.2020.3026036
[21] 南柄飞,郭志杰,王 凯,等. 基于视觉显著性的煤矿井下关键目标对象实时感知研究[J]. 煤炭科学技术,2022,50(8):247–258. NAN Bingfei,GUO Zhijie,WANG Kai,et al. Study on real-time perception of target ROI in underground coal mines based on visual saliency[J]. Coal Science and Technology,2022,50(8):247–258.
[22] 杨 艺,付泽峰,高有进,等. 基于深度神经网络的综采工作面视频目标检测[J]. 工矿自动化,2022,48(8):33−42. YANG Yi,FU Zefeng,GAO Youjin,et al. Video object detection of the fully mechanized working face based on deep neural network[J]. Journal of Mine Automation,2022,48(8):33−42.
[23] LI C,GUO C,GUO J,et al. Learning to enhance low-light image via zero-reference deep curve estimation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,44(8):4225−4238.
[24] ZHANG Y,REN W,ZHANG Z,et al. Focal and efficient IOU loss for accurate bounding box regression[J]. Neurocomputing,2022,506:146−157.
[25] LORE K G,AKINTAYO A,SARKAR S. L lnet:A deep autoencoderapproach to natural low-light image enhancement[J]. Pattern Recognition,2017,61:650−662. doi: 10.1016/j.patcog.2016.06.008
[26] REN S,HE K,GIRSHICK R,et al. Faster R–CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137−1149. doi: 10.1109/TPAMI.2016.2577031
[27] TIAN Z,SHEN C,CHEN H,et al. FCOS:Fully convolutional one-stage object detection[A]// 2019 IEEE/CVF International Conference on Computer Vision(ICCV). IEEE,2020.
[28] ZHOU X,WANG D,P Krhenbühl. Objects as points[EB/OL]. 2019,arXiv preprint arXiv:1904.07850.
[29] REDMON J,FARHADI A. YOLOv3:an incremental improvement[EB/OL]. 2018,arXiv preprint arXiv:1804.02767.
[30] LYU F,LU F,WU J,et al. MBLLEN:Low-light Image/Video Enhancement Using CNNs[C]. British Machine Vision Conference,2018.
[31] GUO X,LI Y,LING H. Lime:Low-light image enhancement via illumination map estimation[J]. IEEE Transactions on Image Processing,2016,26(2):982−993.
[32] WEI C,WANG W,YANG W,et al. Deep retinex decomposition for low-light enhancement[EB/OL]. 2018,arXiv preprint arXiv:1808.04560.
[33] MA L,MA T,LIU R,et al. TOWARD Fast,Flexible,and Robust Low-Light Image Enhancement[A]// IEEE Conference on Computer Vision and Pattern Recognition,2022.
-
期刊类型引用(4)
1. 问永忠，贾澎涛，夏敏高，张龙刚，王伟峰. 基于改进YOLOv8n的井下人员多目标检测. 工矿自动化. 2025(01): 31-37+77 . 
百度学术
2. 董芳凯，赵美卿，黄伟龙. 煤矿井下暗光环境人员行为检测研究. 工矿自动化. 2025(01): 21-30+144 . 百度学术
3. 王轶玮，李晓宇，翁智，白凤山. 基于改进Zero-DCE模型的矿井低照度图像增强方法. 工矿自动化. 2025(02): 57-64+99 . 百度学术
4. 樊红卫，张超，刘金鹏，闫昕山，任跃武，王荣泉，李琳. 一种基于改进DeblurGAN-v2的煤矿带式输送机图像去运动模糊方法. 煤炭学报. 2024(S2): 1234-1247 . 百度学术
其他类型引用(1)
计量
- 文章访问数: 202
- HTML全文浏览量: 65
- PDF下载量: 92
- 被引次数: 5
