2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于内容特征和风格特征融合的单幅图像去雾网络

杨爱萍 刘瑾 邢金娜 李晓晓 何宇清

杨爱萍, 刘瑾, 邢金娜, 李晓晓, 何宇清. 基于内容特征和风格特征融合的单幅图像去雾网络. 自动化学报, 2023, 49(4): 769−777 doi: 10.16383/j.aas.c200217
引用本文: 杨爱萍, 刘瑾, 邢金娜, 李晓晓, 何宇清. 基于内容特征和风格特征融合的单幅图像去雾网络. 自动化学报, 2023, 49(4): 769−777 doi: 10.16383/j.aas.c200217
Yang Ai-Ping, Liu Jin, Xing Jin-Na, Li Xiao-Xiao, He Yu-Qing. Content feature and style feature fusion network for single image dehazing. Acta Automatica Sinica, 2023, 49(4): 769−777 doi: 10.16383/j.aas.c200217
Citation: Yang Ai-Ping, Liu Jin, Xing Jin-Na, Li Xiao-Xiao, He Yu-Qing. Content feature and style feature fusion network for single image dehazing. Acta Automatica Sinica, 2023, 49(4): 769−777 doi: 10.16383/j.aas.c200217

基于内容特征和风格特征融合的单幅图像去雾网络

doi: 10.16383/j.aas.c200217
基金项目: 国家自然科学基金(61632018)资助
详细信息
    作者简介:

    杨爱萍:天津大学电气自动化与信息工程学院副教授. 主要研究方向为深度学习, 图像处理和计算机视觉. 本文通信作者. E-mail: yangaiping@tju.edu.cn

    刘瑾:天津大学电气自动化与信息工程学院硕士研究生. 主要研究方向为图像去雾和深度学习. E-mail: liujin23@tju.edu.cn

    邢金娜:天津大学电气自动化与信息工程学院硕士研究生. 主要研究方向为图像去雾和深度学习. E-mail: xingjinna15@sina.com

    李晓晓:天津大学电气自动化与信息工程学院硕士研究生. 主要研究方向为图像去雾和深度学习. E-mail: 17862715985@163.com

    何宇清:天津大学电气自动化与信息工程学院讲师. 主要研究方向为信号处理, 图像处理和图像超分辨率重建. E-mail: heyuqing@tju.edu.cn

Content Feature and Style Feature Fusion Network for Single Image Dehazing

Funds: Supported by National Natural Science Foundation of China (61632018)
More Information
    Author Bio:

    YANG Ai-Ping Associate professor at the School of Electrical and Information Engineering, Tianjin University. Her research interest covers deep learning, image processing, and computer vision. Corresponding author of this paper

    LIU Jin Master student at the School of Electrical and Information Engineering, Tianjin University. Her research interest covers image dehazing and deep learning

    XING Jin-Na Master student at the School of Electrical and Information Engineering, Tianjin University. Her research interest covers image dehazing and deep learning

    LI Xiao-Xiao Master student at the School of Electrical and Information Engineering, Tianjin University. Her research interest covers image dehazing and deep learning

    HE Yu-Qing Lecturer at the School of Electrical and Information Engineering, Tianjin University. His research interest covers signal processing, image processing, and image super-resolution reconstruction

  • 摘要: 基于深度学习的方法在去雾领域已经取得了很大进展, 但仍然存在去雾不彻底和颜色失真等问题. 针对这些问题, 本文提出一种基于内容特征和风格特征相融合的单幅图像去雾网络. 所提网络包括特征提取、特征融合和图像复原三个子网络, 其中特征提取网络包括内容特征提取模块和风格特征提取模块, 分别用于学习图像内容和图像风格以实现去雾的同时可较好地保持原始图像的色彩特征. 在特征融合子网络中, 引入注意力机制对内容特征提取模块输出的特征图进行通道加权实现对图像主要特征的学习, 并将加权后的内容特征图与风格特征图通过卷积操作相融合. 最后, 图像复原模块对融合后的特征图进行非线性映射得到去雾图像. 与已有方法相比, 所提网络对合成图像和真实图像均可取得理想的去雾结果, 同时可有效避免去雾后的颜色失真问题.
  • 图  1  整体网络结构

    Fig.  1  Architecture of the network

    图  2  残差块结构

    Fig.  2  Architecture of the residual block

    图  3  残差密集块结构

    Fig.  3  Architecture of the residual dense block

    图  5  合成有雾图的实验结果(MSD) ((a) 有雾图; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j)本文方法; (k) 清晰图像)

    Fig.  5  Experimental results of the synthetic hazy images (MSD) ((a) Hazy images; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j) Proposed; (k) Clear images)

    图  7  真实场景有雾图的实验结果 ((a) 有雾图; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j)本文方法)

    Fig.  7  Experimental results of real outdoor hazy images ((a) Hazy images; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j) Proposed)

    图  4  合成有雾图的实验结果(SOTS) ((a) 有雾图; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j)本文方法; (k) 清晰图像)

    Fig.  4  Experimental results of the synthetic hazy images (SOTS) ((a) Hazy images; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j) Proposed; (k) Clear images)

    图  6  去雾结果图及其对应的特征图 ((a) 有雾图; (b) 去雾图像; (c) 内容特征图(RB1_index 59); (d)内容特征图(RB7_index 13); (e) 风格特征图(RDB3_index 10); (f) 融合后的特征图(index 53))

    Fig.  6  Dehazed results and corresponding feature maps ((a) Hazy image; (b) Dehazed image; (c) Content feature map (RB1_index 59); (d) Content feature map (RB7_index 13); (e) Style feature map (RDB3_index 10); (f) Fused feature map (index 53))

    图  8  消融实验结果比较 ((a) 有雾图; (b) CF; (c) WCF; (d) WC-SF; (e) SF-WCF; (f) 清晰图像)

    Fig.  8  Comparison of ablation experiments ((a) Hazy image; (b) CF; (c) WCF; (d) WC-SF; (e) SF-WCF; (f) Clear image)

    表  1  在合成数据集上PSNR和SSIM结果

    Table  1  Comparison of PSNR and SSIM tested on synthetic hazy images

    方法室内图像室外图像
    PSNR (dB)SSIMPSNR (dB)SSIM
    DCP[5]16.620.817919.130.8148
    DehazeNet[9]21.140.847222.460.8514
    MSCNN[10]17.570.810222.060.9078
    AOD-Net[12]19.060.850420.290.8765
    DCPDN[11]15.850.817519.930.8449
    EPDN[15]25.060.923222.570.8630
    FFA-Net[16]36.390.988633.570.9840
    FS-Net[17]26.610.956124.070.8741
    Y-Net[18]19.040.846525.020.9012
    本文方法31.100.977630.740.9774
    下载: 导出CSV

    表  2  在SOTS室内数据集上PSNR和SSIM结果比较

    Table  2  Comparison of PSNR and SSIM tested on SOTS (indoor dataset)

    实验项目PSNR (dB)SSIM
    CF28.570.9703
    WCF29.760.9730
    WC-SF29.850.9774
    SF-WCF31.100.9776
    下载: 导出CSV
  • [1] 吴迪, 朱青松. 图像去雾的最新研究进展. 自动化学报, 2015, 41(2): 221-239.

    Wu Di, Zhu Qing-Song. The latest research progress of image dehazing. Acta Automatica Sinica, 2015, 41(2): 221-239.
    [2] Xu H T, Zhai G T, Wu X L, Yang X K. Generalized equalization model for image enhancement. IEEE Transactions on Multimedia, 2014, 16(1): 68-82. doi: 10.1109/TMM.2013.2283453
    [3] Jiang B, Woodell G A, Jobson D J. Novel multi-scale retinex with color restoration on graphics processing unit. Journal of Real-Time Image Processing, 2015, 10(2): 239-253. doi: 10.1007/s11554-014-0399-9
    [4] Narasimhan S G, Nayar S K. Vision and the atmosphere. International Journal of Computer Vision, 2002, 48(3): 233-254. doi: 10.1023/A:1016328200723
    [5] He K M, Sun J, Tang X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12): 2341-2353.
    [6] Berman D, Treibitz T, Avidan S. Non-local image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016. 1674−1682
    [7] Zhu Q S, Mai J M, Shao L. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 2015, 24(11): 3522-3533. doi: 10.1109/TIP.2015.2446191
    [8] 杨燕, 陈高科, 周杰. 基于高斯权重衰减的迭代优化去雾算法. 自动化学报, 2019, 45(4): 819-828.

    Yang Yan, Chen Gao-Ke, Zhou Jie. Iterative optimization defogging algorithm using gaussian weight decay. Acta Automatica Sinica, 2019, 45(4): 819-828.
    [9] Cai B L, Xu X M, Jia K, Qing C M, Tao D C. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198. doi: 10.1109/TIP.2016.2598681
    [10] Ren W Q, Liu S, Zhang H, Pan J S, Cao X C, Yang M H. Single image dehazing via multi-scale convolutional neural networks. In: Proceedings of the European Conference on Computer Vision. Amsterdam, Netherlands: Springer, 2016. 154−169
    [11] Zhang H, Patel V M. Densely connected pyramid dehazing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018. 3194−3203
    [12] Li B Y, Peng X L, Wang Z Y, Xu J Z, Feng D. AOD-Net: All-in-one dehazing network. In: Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 4780−4788
    [13] Ren W Q, Ma L, Zhang J W, Pan J S, Cao X C, Liu W, et al. Gated fusion network for single image dehazing. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018. 3253−3261
    [14] Li R D, Pan J S, Li Z C, Tang J H. Single image dehazing via conditional generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018. 8202−8211
    [15] Qu Y Y, Chen Y Z, Huang J Y, Xie Y. Enhanced pix2pix dehazing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019. 8160−8168
    [16] Qin X, Wang Z L, Bai Y C, Xie X D, Jia H Z. FFA-Net: Feature fusion attention network for single image dehazing. In: Proceedings of the Association for the Advance of Artificial Intelligence. Hilton Midtown, New York, USA: AAAI Press, 2020. 11908−11915
    [17] Guo F, Zhao X, Tang J, Peng H, Liu L J, Zou B J, et al. Single image dehazing based on fusion strategy. Neurocomputing, 2020, 378: 9-23. doi: 10.1016/j.neucom.2019.09.094
    [18] Yang H H, Yang C H H, Tsai Y C J. Y-net: Multi-scale feature aggregation network with wavelet structure similarity loss function for single image dehazing. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona, Spain: IEEE, 2020. 2628−2632
    [19] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016. 770−778
    [20] Zhang Y L, Tian Y P, Kong Y, Zhong B N, Fu Y. Residual dense network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018. 2472−2481
    [21] Chen L, Zhang H W, Xiao J, Nie L Q, Shao J, Liu W, et al. Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017. 6298−6306
    [22] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211-252. doi: 10.1007/s11263-015-0816-y
    [23] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations. San Diego, CA, USA: ICLR, 2015. 1−14
    [24] Li B Y, Ren W Q, Fu D P, Tao D C, Feng D, Zeng W J, et al. Benchmarking single image dehazing and beyond. IEEE Transactions on Image Processing, 2019, 28(1): 492-505. doi: 10.1109/TIP.2018.2867951
    [25] Scharstein D, Pal C. Learning conditional random fields for stereo. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN, USA: IEEE, 2007. 18−23
  • 加载中
图(8) / 表(2)
计量
  • 文章访问数:  1295
  • HTML全文浏览量:  432
  • PDF下载量:  264
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-04-14
  • 录用日期:  2020-06-28
  • 网络出版日期:  2022-12-22
  • 刊出日期:  2023-04-20

目录

    /

    返回文章
    返回