2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于内容特征和风格特征融合的单幅图像去雾网络

杨爱萍 刘瑾 邢金娜 李晓晓 何宇清

杨爱萍, 刘瑾, 邢金娜, 李晓晓, 何宇清. 基于内容特征和风格特征融合的单幅图像去雾网络. 自动化学报, 2020, 46(x): 1−9 doi: 10.16383/j.aas.c200217
引用本文: 杨爱萍, 刘瑾, 邢金娜, 李晓晓, 何宇清. 基于内容特征和风格特征融合的单幅图像去雾网络. 自动化学报, 2020, 46(x): 1−9 doi: 10.16383/j.aas.c200217
Yang Ai-Ping, Liu Jin, Xing Jin-Na, Li Xiao-Xiao, He Yu-Qing. Content feature and style feature fusion network for single image dehazing. Acta Automatica Sinica, 2020, 46(x): 1−9 doi: 10.16383/j.aas.c200217
Citation: Yang Ai-Ping, Liu Jin, Xing Jin-Na, Li Xiao-Xiao, He Yu-Qing. Content feature and style feature fusion network for single image dehazing. Acta Automatica Sinica, 2020, 46(x): 1−9 doi: 10.16383/j.aas.c200217

基于内容特征和风格特征融合的单幅图像去雾网络

doi: 10.16383/j.aas.c200217
基金项目: 国家自然科学基金(61632018)资助
详细信息
    作者简介:

    杨爱萍:天津大学电气自动化与信息工程学院副教授. 主要研究方向为深度学习、图像处理以及计算机视觉. 本文通信作者. E-mail: yangaiping@tju.edu.cn

    刘瑾:天津大学电气自动化与信息工程学院硕士. 主要研究方向为图像去雾和深度学习. E-mail: liujin23@tju.edu.cn

    邢金娜:天津大学电气自动化与信息工程学院硕士. 主要研究方向为图像去雾和深度学习. E-mail: xingjinna15@sina.com

    李晓晓:天津大学电气自动化与信息工程学院硕士. 主要研究方向为图像去雾和深度学习. E-mail: 17862715985@163.com

    何宇清:天津大学自动化与信息工程学院讲师, 主要研究方向为信号处理、图像处理和图像超分辨率重建. E-mail: heyuqing@tju.edu.cn

    通讯作者:

    天津大学电气自动化与信息工程学院副教授. 主要研究方向为深度学习、图像处理以及计算机视觉. 本文通信作者. E-mail: yangaiping@tju.edu.cn

Content Feature and Style Feature Fusion Network for Single Image Dehazing

Funds: Supported by National Natural Science Foundation of China (61632018)
More Information
    Corresponding author: YANG Ai-Ping Associate Professor at the College of Electrical and Information Engineering, Tianjin University. Her research interest mainly focuses on deep learning, image processing, computer vision. Corresponding author of this paper
  • 摘要: 基于深度学习的方法在去雾领域已经取得了很大进展, 但仍然存在去雾不彻底和颜色失真等问题. 针对这些问题, 本文提出一种基于内容特征和风格特征相融合的单幅图像去雾网络. 所提网络包括特征提取、特征融合和图像复原三个子网络, 其中特征提取网络包括内容特征提取模块和风格特征提取模块, 分别用于学习图像内容和图像风格以实现去雾的同时可较好地保持原始图像的色彩特征. 在特征融合子网络中, 引入注意力机制对内容特征提取模块输出的特征图进行通道加权实现对图像主要特征的学习, 并将加权后的内容特征图与风格特征图通过卷积操作相融合. 最后, 图像复原模块对融合后的特征图进行非线性映射得到去雾图像. 与已有方法相比, 所提网络对合成图像和真实图像均可取得理想的去雾结果, 同时可有效避免去雾后的颜色失真问题.
  • 图  1  整体网络结构

    Fig.  1  Architecture of the Network

    图  2  残差块结构

    Fig.  2  Architecture of the Residual Block

    图  3  残差密集块结构

    Fig.  3  Architecture of the Residual Dense Block

    图  5  合成有雾图的实验结果(MSD). (a) 有雾图; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i)Y-Net; (j)本文方法; (k) 清晰图像.

    Fig.  5  Experimental results of the synthetic hazy images (MSD). (a) hazy image; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j) Proposed; (k) clear image.

    图  7  真实场景有雾图的实验结果. (a) 有雾图; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j)本文方法.

    Fig.  7  Experimental results of real outdoor hazy images. (a) hazy image; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j) Proposed.

    图  4  合成有雾图的实验结果(SOTS). (a) 有雾图; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j)本文方法; (k) 清晰图像.

    Fig.  4  Experimental results of the synthetic hazy images (SOTS). (a) hazy image; (b) DCP; (c) DehazeNet; (d) MSCNN; (e) AOD-Net; (f) DCPDN; (g) EPDN; (h) FFA-Net; (i) Y-Net; (j) Proposed; (k) clear image.

    图  6  去雾结果图及其对应的特征图. (a) 有雾图; (b) 去雾图像; (c) 内容特征图(RB1_index 59); (d)内容特征图(RB7_index 13); (e) 风格特征图(RDB3_index 10); (f) 融合后的特征图(index 53).

    Fig.  6  Dehazed results and corresponding feature maps. (a) hazy image;(b) dehazed image;(c) content feature map(RB1_index59);(d) content feature map(RB7_index 13); (e) style feature map(RDB3_index 10);(f) fused feature map(index 53).

    图  8  消融实验结果比较. (a) 有雾图; (b) CF; (c) WCF; (d) WC-SF; (e) SF-WCF; (f) 清晰图像.

    Fig.  8  Comparison of ablation experiments. (a) hazy image; (b) CF; (c) WCF; (d) WC-SF; (e) SF-WCF; (f) clear image.

    表  1  在合成数据集上PSNR和SSIM结果

    Table  1  Comparison of PSNR and SSIM tested on synthetic hazy images

    MethodIndoorOutdoor
    PSNR/dBSSIMPSNR/dBSSIM
    DCP [5]16.620.817919.130.8148
    DehazeNet [9]21.140.847222.460.8514
    MSCNN [10]17.570.810222.060.9078
    AOD-Net [12]19.060.850420.290.8765
    DCPDN [11]15.850.817519.930.8449
    EPDN [15]25.060.923222.570.8630
    FFA-Net[16]36.390.988633.570.9840
    FS-Net[17]26.610.956124.070.8741
    Y-Net[18]19.040.846525.020.9012
    Proposed31.100.977630.740.9774
    下载: 导出CSV

    表  2  在SOTS室内数据集上PSNR和SSIM结果比较

    Table  2  Comparison of PSNR and SSIM tested on SOTS(indoor)

    CFWCFWC-SFSF-WCF
    PSNR/dB28.5729.7629.8531.10
    SSIM0.97030.97300.97740.9776
    下载: 导出CSV
  • [1] 吴迪, 朱青松. 图像去雾的最新研究进展. 自动化学报, 2015, 41(2): 221−239

    Wu Di, Zhu Qing-Song. The latest research progress of image dehazing. Acta Automatica Sinica, 2015, 41(2): 221−239
    [2] Xu H T, Zhai G T, Wu X L, Yang X K. Generalized equalization model for image enhancement. IEEE Transactions on Multimedia, 2014, 16(1): 68−82 doi: 10.1109/TMM.2013.2283453
    [3] Jiang B, Woodell G A, Jobson D J. Novel multi-scale retinex with color restoration on graphics processing unit. Journal of Real-Time Image Processing, 2015, 10(2): 239−253 doi: 10.1007/s11554-014-0399-9
    [4] Narasimhan S G, Nayar S K. Vision and the atmosphere. International Journal of Computer Vision, 2002, 48(3): 233−254 doi: 10.1023/A:1016328200723
    [5] He K M, Sun J, Tang X O. Single image haze removal using dark channel prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(12): 2341−2353
    [6] Berman D, Treibitz T, Avidan S. Non-local image dehazing. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016. 1674−1682
    [7] Zhu Q S, Mai J M, Shao L. A fast single image haze removal algorithm using color attenuation prior. IEEE Transactions on Image Processing, 2015, 24(11): 3522−3533 doi: 10.1109/TIP.2015.2446191
    [8] 杨燕, 陈高科, 周杰. 基于高斯权重衰减的迭代优化去雾算法. 自动化学报, 2019, 45(4): 819−828

    Yang Yan, Chen Gao-Ke, Zhou Jie. Iterative optimization defogging algorithm using gaussian weight decay. Acta Automatica Sinica, 2019, 45(4): 819−828
    [9] Cai B L, Xu X M, Jia K, Qing C M, Tao D C. DehazeNet: an end-to-end system for single image haze removal. IEEE Transactions on Image Processing, 2016, 25(11): 5187−5198 doi: 10.1109/TIP.2016.2598681
    [10] Ren W Q, Liu S, Zhang H, Pan J S, Cao X C, Yang M H. Single image dehazing via multi-scale convolutional neural networks. In: Proceedings of 2016 European Conference on Computer Vision. Amsterdam, NETHERLANDS: Springer, 2016. 154−169
    [11] Zhang H, Patel V M. Densely connected pyramid dehazing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018. 3194−3203
    [12] Li B Y, Peng X L, Wang Z Y, Xu J Z, Feng D. AOD-Net: all-in-one dehazing network. In: Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 4780−4788
    [13] Ren W Q, Ma L, Zhang J W, Pan J S, Cao X C, Liu W, et al. Gated fusion network for single image dehazing. In: Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018. 3253−3261
    [14] Li R D, Pan J S, Li Z C, Tang J H. Single image dehazing via conditional generative adversarial network. In: Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018. 8202−8211
    [15] Qu Y Y, Chen Y Z, Huang J Y, Xie Y. Enhanced pix2pix dehazing network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019. 8160−8168
    [16] Qin X, Wang Z L, Bai Y C, Xie X D, Jia H Z. FFA-Net: Feature fusion attention network for single image dehazing. In: Proceedings of the Association for the Advance of Artificial Intelligence. Hilton Midtown, New York: AAAI Press, 2020. 11908−11915
    [17] Guo F, Zhao X, Tang J, Peng H, Liu L J, Zou B J, et al. Single image dehazing based on fusion strategy. Neurocomputing, 2020, 378: 9−23 doi: 10.1016/j.neucom.2019.09.094
    [18] Yang H H, Yang C H H, Tsai Y C J. Y-net: Multi-scale feature aggregation network with wavelet structure similarity loss function for single image dehazing. In: Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Barcelona, Spain: IEEE, 2020. 2628−2632
    [19] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016. 770−778
    [20] Zhang Y L, Tian Y P, Kong Y, Zhong B N, Fu Y. Residual dense network for image super-resolution. In: Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018. 2472−2481
    [21] Chen L, Zhang H W, Xiao J, Nie L Q, Shao J, Liu W, et al. Sca-cnn: Spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017. 6298−6306
    [22] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3): 211−252 doi: 10.1007/s11263-015-0816-y
    [23] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of 2015 3rd International Conference on Learning Representations. San Diego, CA, USA: ICLR, 2015. 1−14
    [24] Li B Y, Ren W Q, Fu D P, Tao D C, Feng D, Zeng W J, et al. Benchmarking single image dehazing and beyond. IEEE Transactions on Image Processing, 2019, 28(1): 492−505 doi: 10.1109/TIP.2018.2867951
    [25] Scharstein D, Pal C. Learning conditional random fields for Stereo. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN, USA: IEEE, 2007. 18−23
  • 加载中
计量
  • 文章访问数:  10
  • HTML全文浏览量:  5
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-04-14
  • 录用日期:  2020-06-28

目录

    /

    返回文章
    返回