高级检索

基于低光校正的双编码器井下异源图像融合

Underground heteroge neous image fusion based on low-light correction and dual encoders

  • 摘要: 井下环境的复杂性和多变性给安全生产带来严峻挑战,促使视频监控成为保障作业安全的关键技术手段。其中可见光与热红外图像凭借各自优势在井下监控中发挥着重要作用,然而单一模态图像固有的局限性难以满足井下智能监控对信息完备性的要求。为此,如何将异源图像进行融合实现优势互补,是解决上述问题的有效途径。针对传统融合算法在井下特殊环境中存在的光照适应性差、局部特征丢失及伪影干扰问题,提出一种基于低光校正的双编码器井下异源图像融合算法。首先,为避免单一编码器在提取异源图像特征时出现混淆,造成融合图像无法充分保留原始图像信息的问题,分别设计基于卷积神经网络的可见光编码器和基于Transformer架构的热红外编码器,从而实现对原始异源图像特征的有效提取;然后,为缓解井下光照不均导致的融合图像局部特征丢失问题,设计选择性光照特征增强模块以增强低照度区域的视觉质量;其次,设计并行式全局与局部特征提取模块以提取图像宏观语义信息的同时关注微观细节信息,增强融合图像的特征丰富性;最后,为缓解融合图像中存在的伪影干扰问题,提出由光照信息引导的低光校正损失函数,以辅助解码器动态调整融合权重,从而增强融合图像对异源图像互补信息的保留能力。为了验证所提出算法的优势,利用自建数据集与9种融合检测算法进行比较。结果表明:所提出的融合算法可以有效缓解由于光照不均造成的局部信息丢失,减少融合结果中存在的伪影干扰,增强融合图像的信息完整性。相较于对比算法,所提的算法在空间频率、平均梯度、谱相关差异、视觉信息保真度及相关系数五大核心指标上均呈现显著优势,同时融合图像在视觉效果上更贴近人类视觉感知。

     

    Abstract: The complexity and variability of underground environments pose severe challenges to safe production, making video surveillance a key technological means to ensure operational safety. Visible light and thermal infrared images play important roles in underground monitoring due to their respective advantages. However, the inherent limitations of single-modal images fail to meet the requirements for information completeness in intelligent underground monitoring. Therefore, fusing heterogeneous images to achieve complementary advantages is an effective solution to the aforementioned problems.Aiming at the issues of poor lighting adaptability, local feature loss, and artifact interference in traditional fusion algorithms in the special underground environment, a dual-encoder fusion algorithm for heterogeneous underground images based on low-light correction is proposed. First, to avoid the confusion caused by a single encoder when extracting features from heterogeneous images, leading to insufficient retention of original image information in the fused image, separate visible light and thermal infrared encoders based on convolutional neural networks and Transformer architectures are designed, respectively, to effectively extract the features of the original heterogeneous images. Then, to mitigate the local feature loss in fused images caused by uneven lighting in underground environments, a selective lighting feature enhancement module is designed to improve the visual quality of low-illumination areas. Next, a parallel global and local feature extraction module is designed to capture both macro semantic information and micro detail information of images, thereby enhancing the feature richness of the fused images. Finally, to alleviate the artifact interference in fused images, a low-light correction loss function guided by lighting information is proposed to assist the decoder in dynamically adjusting fusion weights, thereby enhancing the fused image’s ability to retain complementary information from heterogeneous images.To verify the advantages of the proposed algorithm, it was compared with nine fusion detection algorithms using a self-built dataset. The experimental results show that the proposed fusion algorithm can effectively mitigate local information loss caused by uneven lighting, reduce artifact interference in the fused results, and enhance the information completeness of the fused images. Compared with the contrast algorithms, the proposed algorithm shows significant advantages in five core indicators: spatial frequency, average gradient, spectral correlation difference, visual information fidelity, and correlation coefficient. Moreover, the fused images are more consistent with human visual perception in terms of visual effects.

     

/

返回文章
返回