| Citation: |
TIAN Zijian，WU Jiaqi，ZHANG Wenqi，et al. An illuminance improvement and details enhancement method on coal mine low-light images based on Transformer and adaptive feature fusion[J]. Coal Science and Technology，2024，52（1）：297−310. DOI: 10.13199/j.cnki.cst.2023-0112
|
High quality mine images can provide guarantee for mine safety production, and improve the performance of subsequent image analysis technologies. Affected by low illuminance environment, mine images suffer low brightness, uneven brightness, color distortion, and serious loss of details. Aiming at the above problems, an illuminance improvement and details enhancement method on coal mine low-light images based on Transformer and adaptive feature fusion was proposed to enhance the brightness and detail of mine low illuminance images. Based on the idea of generative confrontation, a framework of generative adversary agent model was built, and the target image domain was used instead of a single reference image to drive discriminator to supervise the training of generator, so as to achieve adaptive enhancement of low illuminance images; The feature encoder was built based on the feature representation learning theory to decouple the image into illuminance component and reflection component, the method can avoid the interaction between illuminance and color features during image enhancement to solve the color distortion; the CEM-Transformer Encoder was designed to enhance the brightness component, the method can improve the overall image brightness and eliminate the local area brightness unevenness, by capturing the global context and extracting the local area features; In the process of reflection component enhancement, the skip connection combined with CEM-Cross-Transformer Encoder was used to adaptively fuse low-level features with features at the deep CNN layers, which can effectively avoid the loss of detailed features, and ECA-Net was added to the encode network to improve the feature extraction efficiency of the shallow CNN layers. The low illuminance mine image dataset was produced to provide data resources for the low illuminance mine image enhancement task. The experiments show that, compared with five advanced low illuminance image enhancement algorithms, the quality indicators PSNR, SSIM and VIF of the images enhanced by the algorithm are improved by 16.564%, 10.998%, 16.226% and 14.438%, 10.888% and 14.948% on average on the low illuminance mine image dataset and the public dataset. And the algorithm also perform well in subjective visual evaluation. The above results prove that the algorithm can effectively improve the overall image brightness and eliminate the uneven brightness, thus to achieve mine low illuminace image enhancement.
| [1] |
王国法,王 虹,任怀伟,等. 智慧煤矿2025情景目标和发展路径[J]. 煤炭学报,2018,43(2):295−305.
WANG Guofa,WANG Hong,REN Huaiwei,et al. 2025 scenarios and development path of intelligent coal mine[J]. Journal of China Coal Society,2018,43(2):295−305.
|
| [2] |
陈 伟,任 鹏,田子建,等. 基于注意力机制的无监督矿井人员跟踪[J]. 煤炭学报,2021,46(S1):601−608.
CHEN Wei,REN Peng,TIAN Zijian,et al. Unsupervised mine personnel tracking based on attention mechanism[J]. Journal of China Coal Society,2021,46(S1):601−608.
|
| [3] |
韩江洪,卫 星,陆 阳,等. 煤矿井下机车无人驾驶系统关键技术[J]. 煤炭学报,2020,45(6):2104−2115.
HAN Jianghong,WEI Xing,LU Yang,et al. Driverless technology of underground locomotive in coal mine[J]. Journal of China Coal Society,2020,45(6):2104−2115.
|
| [4] |
杨 潇,陈 伟,任 鹏,等. 基于域适应的煤矿环境监控图像语义分割[J]. 煤炭学报,2021,46(10):3386−3396.
YANG Xiao,CHEN Wei,REN Peng,et al. Coal mine monitoring image semantic segmentation based on domain adaptation[J]. Journal of China Coal Society,2021,46(10):3386−3396.
|
| [5] |
郝 帅,张 旭,马 旭,等. 基于CBAM-YOLOv5的煤矿输送带异物检测[J/OL]. 煤炭学报:1−11[2-04-04]. DOI:10.13225/j.cnki. jccs.2021.1644.
HAO Shuai,ZHANG Xu,MA Xu, et al. Foreign object detection in coal mine conveyor belt based on CBAM-YOLOv5[J/OL]. Journal of China Coal Society:1−11[2022-04-04]. DOI:10.13225/j.cnki. jccs.2021.1644.
|
| [6] |
单鹏飞,孙浩强,来兴平,等. 基于改进 Faster R-CNN 的综放煤矸混合放出状态识别方法[J]. 煤炭学报,2022,47(3):1382−1394.
SHAN Pengfei,SUN Haoqiang,LAI Xingping,et al. Identification method on mixed and release state of coal-gangue masses of fully mechanized caving based on improved Faster R-CNN[J]. Journal of China Coal Society,2022,47(3):1382−1394.
|
| [7] |
程德强,钱建生,郭星歌,等. 煤矿安全生产视频 AI 识别关键技术研究综述[J]. 煤炭科学技术,2023,51(2):349−365.
CHENG Deqiang,QIAN Jiansheng,GUO Xingge,et al. Review on key technologies of AI recognition for videos in coal mine[J]. Coal Science and Technology,2023,51(2):349−365.
|
| [8] |
KIM Yeong-Taeg. Contrast enhancement using brightness preserving bi-histogram equalization[J]. IEEE transactions on Consumer Electronics,1997,43(1):1−8. doi: 10.1109/30.580378
|
| [9] |
WANG Yu,CHEN Qian,ZHANG Baeomin. Image enhancement based on equal area dualistic sub-image histogram equalization method[J]. IEEE transactions on Consumer Electronics,1999,45(1):68−75. doi: 10.1109/30.754419
|
| [10] |
WANG Jianwei. An enhancement algorithm for low-illumination color image with preserving edge[J]. Computer Technology Development,2018,28(1):116−120.
|
| [11] |
LIU Lizhu,WANG Haiying. Image enhancement using a nonlinear method with an improved Single-Scale Retinex algorithm[C]//2011 International Conference on Electronics,Communications and Control (ICECC),2011:2086−2089.
|
| [12] |
RAHMAN Ziaur,JOBSON Daniel J,WOODELL Glenn A. Multi-scale retinex for color image enhancement[C]//Proceedings of 3rd IEEE International Conference on Image Processing,1996,3:1003−1006.
|
| [13] |
JIANG Xingfang,WANG Ge,SHEN Weimin. A method of color image enhancement using color advanced retinex[J]. Journal of Optoelectronics. Laser,2008,19(10):1402−1404.
|
| [14] |
LORE Kin Gwn,AKINTAYO Adedotun,SARKAR Soumik. LLNet:a deep autoencoder approach to natural low-light image enhancement[J]. Pattern Recognition,2017,61:650−662. doi: 10.1016/j.patcog.2016.06.008
|
| [15] |
WEI Chen,WANG Wenjing,YANG Wenhan, et al. Deep Retinex Decomposition for Low-Light Enhancement[J]. arXiv preprint arXiv:1808.04560,2018.
|
| [16] |
GOODFELLOW Ian,POUGET-ABADIE Jean,MIRZA Mehdi, et al. Generative adversarial nets[J]. Advances in Neural Information Processing Systems,2014,27.
|
| [17] |
CHOI Yunjey,CHOI Minje,KIM Munyoung, et al. Stargan:Unified generative adversarial networks for multi-domain image-to-image translation [C]//In Proceedings of the IEEE conference on computer vision and pattern recognition,2018:8789−8797.
|
| [18] |
CHEN Xi,DUAN Yan,HOUTHOOFT Rein, et al. InfoGAN:interpretable representation learning by info-rmation maximizing generative adversarial nets[C]//Conference and Workshop on Neural Information Processing Sys-tems. Barcelona:MIT Press,2016,2180−2188.
|
| [19] |
ZHU Junyan,PARK Taesung,ISOLA Phillip, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the IEEE international conference on computer vision,2017:2223−2232.
|
| [20] |
JIANG Yifan,GONG Xinyu,LIU Ding,et al. Enlightengan:Deep light enhancement without paired supervision[J]. IEEE transactions on image processing,2021,30:2340−2349. doi: 10.1109/TIP.2021.3051462
|
| [21] |
LIU Siyuan,THUNG Kim-Han,QU Liangqiong,et al. Learning MRI artefact removal with unpaired data[J]. Nature Machine Intelligence,2021,3(1):60−67. doi: 10.1038/s42256-020-00270-2
|
| [22] |
WANG Qilong,WU Banggu,ZHU Pengfei, et al. ECA-Net:Efficient channel attention for deep convolutional neural networks[C]. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition,2020:11534−11542.
|
| [23] |
VASWANI Ashish,SHAZEER Noam,PARMAR Niki, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems,2017,30.
|
| [24] |
FRAN C. Deep learning with depth wise separable convolutions[C]//IEEE conference on computer vision and pattern recognition (CVPR),2017:1251-1258.
|
| [25] |
MIYATO Takeru,KATAOKA Toshiki,KOYAMA Masanori, et al. Spectral normalization for generative adversarial networks[J]. arXiv preprint arXiv:1802.05957,2018.
|
| [26] |
CHEN Xi,DUAN Yan,HOUTHOOFT Rein, et al. Infogan:Interpretable representation learning by information maximizing generative adversarial nets[J]. Advances in Neural Information Processing Systems,2016,29.
|
| [27] |
HIGGINS Irina,MATTHEY Loic,PAL Arka, et al. beta-vae:Learning basic visual concepts with a constrained variational framework[C]//International Conference on Learning Representations,2016.
|
| [28] |
RADFORD Alec,METZ Luke,CHINTALA Soumith. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint arXiv:1511.06434,2015.
|
| [29] |
张雅荔,李文元,李昌禄,等. 融合注意力引导的多尺度低照度图像增强方法[J/OL]. 西安电子科技大学学报:1−9[2022-11-30]. http://kns.cnki.net/kcms/detail/61.1076.TN.20221129.0832.004.html.
ZHANG Yali,LI Wenyuan,LI Changlu, et al. Method for enhancement of the multi-scale low-light image by combining an attention guidance[J/OL]. Journal Of XiDian University:1−9[2022-11-30]. http://kns.cnki.net/kcms/detail/61.1076.TN.20221129.0832.004.html.
|
| [30] |
陈 勇,陈 东,刘焕淋,等. 基于深度卷积神经网络的无参考低照度图像增强[J]. 电子与信息学报,2022,44(6):2166−2174.
CHEN Yong,CHEN Dong,LIU Huanlin,et al. Unreferenced Low-lighting Image Enhancement Based on[J]. Journal of Electronics & Information Technology Deep Convolutional Neural Network,2022,44(6):2166−2174.
|
| [31] |
李 烁,王 慧,耿则勋,等. 双范数混合约束的遥感影像亮度不均变分校正[J]. 测绘学报,2018,47(12):1621−1629. doi: 10.11947/j.AGCS.2018.20170625
LI Shuo,WANG HUI,GENG Zexun,et al. Variational uneven illunination correction with double-norm hybrid constriants for remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica,2018,47(12):1621−1629. doi: 10.11947/j.AGCS.2018.20170625
|
| [32] |
王冬云,唐 楚,鄂世举,等. 基于导向滤波Retinex和自适应Canny的图像边缘检测[J]. 光学精密工程,2021,29(2):443−451. doi: 10.37188/OPE.20212902.0443
WANG Dongyun,TANG Chu,E Shiju,et al. Image edge detection based on guided filter Retinex and adaptive Canny[J]. Optics and Precision Engineering,2021,29(2):443−451. doi: 10.37188/OPE.20212902.0443
|
| [33] |
KHAN Salman,NASEER Muzammal,HAYAT Munawar,et al. Transformers in vision:a survey[J]. ACM Computing Surveys (CSUR),2022,54(10):1−41.
|
| [34] |
DOSOVITSKIY Alexey,BEYER Lucas,KOLESNIKOV Alexander, et al. An image is worth 16x16 words:Transformers for image recognition at scale[J]. arXiv preprint arXiv:2010.11929,2020.
|
| [35] |
JOLICOEUR-MARTINEAU Alexia. The relativistic discriminator:a key element missing from standard GAN[J]. arXiv preprint arXiv:1807.00734,2018.
|
| [36] |
JOHNSON Justin,ALAHI Alexandre,FEI-FEI Li. Perceptual losses for real-time style transfer and super-resolution[C]//European conference on computer vision. Springer,Cham,2016:694-711.
|
| [37] |
CHEN Chen,CHEN Qifeng,XU Jia, et al. Learning to see in the dark[C]//In IEEE Conference on Computer Vision and Pattern Recognition,2018:3291−3300.
|
| [38] |
WANG Zhou,SIMONCELLI Eero P,BOVIK Alan C. Multiscale structural similarity for image quality assessment[C]//The Thirty Seventh Asilomar Conference on Signals,Systems & Computers,Pacific Grove,CA,USA,2003:1398−1402.
|
| [39] |
ZHANG Lin,ZHANG Lijun,LIU Xiao, et al. Zero-shot restoration of back-lit images using deep internal learning[C]//Proceedings of the 27th ACM international conference on multimedia,2019:1623−1631.
|
| [40] |
LYU Feifan,LU Feng,WU Jianhua, et al. MBLLEN:low-light image/video enhancement using CNNs[J]. BMVC,2018:4.
|
| 1. |
袁亮，徐良骥. 高潜水位采煤沉陷区资源化、能源化、功能化利用构想与实践. 煤炭学报. 2024(01): 65-74 .
|
Supported by: Beijing Renhe Information Technology Co., Ltd. 
DownLoad: