2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于递归残差网络的图像超分辨率重建

周登文 赵丽娟 段然 柴晓亮

周登文, 赵丽娟, 段然, 柴晓亮. 基于递归残差网络的图像超分辨率重建. 自动化学报, 2019, 45(6): 1157-1165. doi: 10.16383/j.aas.c180334
引用本文: 周登文, 赵丽娟, 段然, 柴晓亮. 基于递归残差网络的图像超分辨率重建. 自动化学报, 2019, 45(6): 1157-1165. doi: 10.16383/j.aas.c180334
ZHOU Deng-Wen, ZHAO Li-Juan, DUAN Ran, CHAI Xiao-Liang. Image Super-resolution Based on Recursive Residual Networks. ACTA AUTOMATICA SINICA, 2019, 45(6): 1157-1165. doi: 10.16383/j.aas.c180334
Citation: ZHOU Deng-Wen, ZHAO Li-Juan, DUAN Ran, CHAI Xiao-Liang. Image Super-resolution Based on Recursive Residual Networks. ACTA AUTOMATICA SINICA, 2019, 45(6): 1157-1165. doi: 10.16383/j.aas.c180334

基于递归残差网络的图像超分辨率重建

doi: 10.16383/j.aas.c180334
基金项目: 

中央高校基本科研业务费专项资金 2018ZD06

北京市自然科学基金 4162056

详细信息
    作者简介:

    赵丽娟   华北电力大学控制与计算机工程学院硕士研究生.2015年获得邢台学院数学与信息技术学院学士学位.主要研究方向为计算机视觉和深度学习.E-mail:zhaolj97@163.com

    段然  华北电力大学控制与计算机工程学院硕士研究生.2016年获得北京理工大学计算机学院学士学位.主要研究方向为计算机视觉和深度学习.E-mail:1162227075@ncepu.edu.cn

    柴晓亮  华北电力大学控制与计算机工程学院硕士研究生.2016年获得平顶山学院信息工程学院学士学位.主要研究方向为图像处理和图像超分辨率.E-mail:13051603700@163.com

    通讯作者:

    周登文  华北电力大学控制与计算机工程学院教授.主要研究方向为图像去噪, 图像去马赛克, 图像插值和图像超分辨率.本文通信作者.E-mail:zdw@ncepu.edu.cn

Image Super-resolution Based on Recursive Residual Networks

Funds: 

the Fundamental Research Funds for the Central Universities 2018ZD06

Beijing Natural Science Foundation 4162056

More Information
    Author Bio:

      Master student at the School of Control and Computer Engineering, North China Electric Power University. She received her bachelor degree from the School of Mathematics and Information Technology, Xingtai University in 2015. Her research interest covers computer vision and deep learning

      Master student at the School of Control and Computer Engineering, North China Electric Power University. She received her bachelor degree from the School of Computing, Beijing Institute of Technology in 2016. Her research interest covers computer vision and deep learning

     Master student at the School of Control and Computer Engineering, North China Electric Power University. He received his bachelor degree from the School of Information Engineering, Pingdingshan University in 2016. His research interest covers image processing and image super-resolution

    Corresponding author: ZHOU Deng-Wen    Professor at the School of Control and Computer Engineering, North China Electric Power University. His research interest covers image denoising, image demosaicing, image interpolation and image super-resolution. Corresponding author of this paper
  • 摘要: 深度卷积神经网络在单图像超分辨率重建方面取得了卓越成就,但其良好表现通常以巨大的参数数量为代价.本文提出一种简洁紧凑型递归残差网络结构,该网络通过局部残差学习减轻训练深层网络的困难,引入递归结构保证增加深度的同时控制模型参数数量,采用可调梯度裁剪方法防止产生梯度消失/梯度爆炸,使用反卷积层在网络末端直接上采样图像到超分辨率输出图像.基准测试表明,本文在重建出同等质量超分辨率图像的前提下,参数数量及计算复杂度分别仅为VDSR方法的1/10和1/(2n2).
    1)  本文责任编委 王亮
  • 图  1  残差块结构[18]

    Fig.  1  Residual block structure[18]

    图  2  网络结构示意图

    Fig.  2  Network structure diagram

    图  3  RRSR具体网络结构图

    Fig.  3  The specific network structure of RRSR

    图  4  递归块结构

    Fig.  4  Recursive block structure

    图  5  各种UC组合所构成网络的性能对比图

    Fig.  5  The performance of various networks at U and C combinations

    图  6  各种SISR方法的×3模型在Set5测试集上的平均PSNR值及参数数量

    Fig.  6  Average PSNR and number of parameters on the testset Set5 for scale factor ×3 of various SISR methods

    图  7  测试集BSD100中的"img_092"重建结果对比图

    Fig.  7  A comparison of the reconstruction results of "img_092" in the testset BSD100

    图  8  测试集Set5中的"butterfly"重建对比图

    Fig.  8  A comparison of the reconstruction results of "butterfly" in the testset Set5

    图  9  各种SISR方法的×4模型在测试集Set14上的平均运行时间及平均PSNR值[10]

    Fig.  9  Speed and average PSNR of various SISR methods on the Set14 with scale factor × 4[10]

    表  1  不同RRSR组件构成的$\times3$模型在Set5测试集上的平均PSNR值及参数量

    Table  1  Average PSNR and number of parameters when different RRSR components are turned on or off, for scale factor $\times3$ on testset Set5

    局部残差 递归结构 PSNR (dB) 参数数量($\times10^3$)
    × × 33.27 371
    × 33.58 371
    33.70 39
    下载: 导出CSV

    表  2  各种SISR方法的$\times2$, $\times3$和$\times4$模型在测试集Set5、Set14和BSD100上的平均PSNR值与SSIM值

    Table  2  Average PSNR/SSIMs of various SISR methords for scale factor $\times2$, $\times3$ and $\times4$ on Set5, Set14 and BSD100

    测试集 放大倍数 Bicubic PSNR/SSIM SelfEx PSNR/SSIM SRCNN PSNR/SSIM FSRCNN PSNR/SSIM VDSR PSNR/SSIM RRSR PSNR/SSIM
    Set5 ×2 33.66/0.9299 36.49/0.9537 36.66/0.9542 37.00/0.9558 37.53/0.9587 37.55/0.9588
    Set5 ×3 30.39/0.8682 32.58/0.9093 32.75/0.9090 33.16/0.9140 33.66/0.9213 33.70/0.9208
    Set5 ×4 28.42/0.8104 30.31/0.8619 30.48/0.8628 30.71/0.8657 31.35/0.8838 31.32/0.8836
    Set14 ×2 30.24/0.8688 32.22/0.9034 32.42/0.9063 32.64/0.9088 33.03/0.9124 33.04/0.9125
    Set14 ×3 27.55/0.7742 29.16/0.8196 29.28/0.8209 29.43/0.8242 29.77/0.8314 29.75/0.8307
    Set14 ×4 26.00/0.7027 27.40/0.7518 27.49/0.7503 27.60/0.7535 28.01/0.7674 28.00/0.7675
    BSD100 ×2 29.56/0.8431 31.18/0.8855 31.36/0.8879 31.51/0.8906 31.90/0.8960 31.91/0.8961
    BSD100 ×3 27.21/0.7385 28.29/0.7840 28.41/0.7863 28.52/0.7897 28.82/0.7976 28.78/0.7969
    BSD100 ×4 25.96/0.6675 26.84/0.7106 26.90/0.7101 26.97/0.7128 27.29/0.7251 27.25/0.7249
    下载: 导出CSV
  • [1] Oktay O, Bai W, Lee M, Guerrero R, Kamnitsas K, Caballero J, et al. Multi-input cardiac image super-resolution using convolutional neural networks. In: Proceedings of the 2016 International Conference on Medical Image Computing and Computer-assisted Intervention. Athens, Greece: Springer, Cham, 2016. 246-254
    [2] Luo Y, Zhou L, Wang S, Wang Z Y. Video satellite imagery super resolution via convolutional neural networks. IEEE Geoscience and Remote Sensing Letters, 2017, 14(12):2398-2402 doi: 10.1109/LGRS.2017.2766204
    [3] Rasti P, Uiboupin T, Escalera S, Anbarjafari G. Convolutional neural network super resolution for face recognition in surveillance monitoring. In: Proceedings of the 2016 International Conference on Articulated Motion and Deformable Objects. Palma de Mallorca, Spain: Springer, Cham, 2016. 175-184
    [4] 陆志芳, 钟宝江.基于预测梯度的图像插值算法.自动化学报, 2018, 44(6):1072-1085 http://www.aas.net.cn/CN/abstract/abstract19297.shtml

    Lu Zhi-Fang, Zhong Bao-Jiang. Image interpolation with predicted gradients. Acta Automatica Sinica, 2018, 44(6):1072-1085 http://www.aas.net.cn/CN/abstract/abstract19297.shtml
    [5] 熊娇娇, 卢红阳, 张明辉, 刘且根.基于梯度域的卷积稀疏编码磁共振成像重建.自动化学报, 2017, 43(10):1841-1849 http://www.aas.net.cn/CN/abstract/abstract19159.shtml

    Xiong Jiao-Jiao, Lu Hong-Yang, Zhang Ming-Hui, Liu Qie-Gen. Convolutional sparse coding in gradient domain for MRI reconstruction. Acta Automatica Sinica, 2017, 43(10):1841-1849 http://www.aas.net.cn/CN/abstract/abstract19159.shtml
    [6] Dong C, Chen C L, He K M, Tang X O. Learning a deep convolutional network for image super-resolution. In: Proceedings of the 13th European Conference on Computer Vision, Zurich Switzerland: Springer, Cham, 2014. 184-199
    [7] Dong C, Chen C L, Tang X O. Accelerating the super-sesolution convolutional neural network. In: Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, Cham, 2016. 391-407
    [8] Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 1646-1654
    [9] Lim B, Son S, Kim H, Nah S, Lee K M. Enhanced deep residual networks for single image super-resolution. In: Proceedings of the 2017 IEEE Computer Vision and Pattern Recognition Workshops. Honolulu, USA: IEEE, 2017. 1132-1140
    [10] Lai W S, Huang J B, Ahuja N, Yang M H. Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 5835-5843
    [11] Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, et al. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 105-114
    [12] Dong C, Chen C L, He K M, Tang X O. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(2):295-307 http://d.old.wanfangdata.com.cn/Periodical/jsjfzsjytxxxb201709007
    [13] Timofte R, De Smet V, Van Gool L. A+: Adjusted anchored neighborhood regression for fast super-resolution. In: Proceedings of the 2015 Asian Conference on Computer Vision. Singapore, Singapore: Springer, Cham. 2015. 111-126
    [14] 胡长胜, 詹曙, 吴从中.基于深度特征学习的图像超分辨率重建.自动化学报, 2017, 43(5):814-821 http://www.aas.net.cn/CN/abstract/abstract19059.shtml

    Hu Chang-Sheng, Zhan Shu, Wu Cong-Zhong. Image super-resolution based on deep learning features. Acta Automatica Sinica, 2017, 43(5):814-821 http://www.aas.net.cn/CN/abstract/abstract19059.shtml
    [15] Kim J, Lee J K, Lee K M. Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, Nevada, USA: IEEE, 2016. 1637-1645
    [16] Yang W H, Feng J S, Xie G S, Liu J Y, Guo Z M, Yan S C. Video super-resolution based on spatial-temporal recurrent residual networks. Computer Vision and Image Understanding, 2018, 168:79-92 doi: 10.1016/j.cviu.2017.09.002
    [17] LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11):2278-2324 doi: 10.1109/5.726791
    [18] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, Nevada, USA: IEEE, 2016. 770-778
    [19] Mao X J, Shen C H, Yang Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: NIPS, 2016. 2810-2818
    [20] Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning. Haifa, Israel: ICML, 2010. 807-814
    [21] Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks. In: Proceedings of the 30th International Conference on Machine Learning. Atlanta, USA: ICML, 2013, 28(3): 1310-1318
    [22] Perez-Pellitero E, Salvador J, Ruiz-Hidalgo J, Rosenhahn B. PSyCo: Manifold span reduction for super resolution. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 1837-1845
    [23] Huang Y, Wang W, Wang L. Video super-resolution via bidirectional recurrent convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):1015-1028 doi: 10.1109/TPAMI.2017.2701380
    [24] Yang J C, Wright J, Huang T S, Ma Y. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 2010, 19(11):2861-2873 doi: 10.1109/TIP.2010.2050625
    [25] Schulter S, Leistner C, Bischof H. Fast and accurate image upscaling with super-resolution forests. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015. 3791-3799
    [26] Martin D, Fowlkes C, Tal D, Malik J. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the 2002 International Conference on Computer Vision. Vancouver, BC, Canada: IEEE, 2002. 416-423
    [27] Bevilacqua M, Roumy A, Guillemot C, Alberi Morel M L. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the 23rd British Machine Vision Conference. Surrey UK: BMVA Press, 2012. 1-10
    [28] Zeyde R, Elad M, Protter M. On single image scaleup using sparse-representations. In: Proceedings of the 2012 International Conference on Curves and Surfaces. Avignon, France: Springer, Berlin, Heidelberg, 2012. 711-730
    [29] Jia Y Q, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, et al. Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia. New York, USA: ACM, 2014. 675-678
    [30] He K M, Zhang X Y, Ren S Q, Sun J. Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the 2015 International Conference on Computer Vision. Santiago, Chile: IEEE, 2015. 1026-1034
    [31] Huang J B, Singh A, Ahuja N. Single image super-resolution from transformed self-exemplars. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015. 5197-5206
    [32] Shi W Z, Caballero J, Huszar F, Totz J, P Aitken A, Bishop R, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 1874-1883
    [33] Wang Z W, Liu D, Yang J C, Han W, Huang T. Deep networks for image super-resolution with sparse prior. In: Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago, Chile: OALib Journal, 2015. 370-378
    [34] Vedaldi A, Lenc K. MatConvNet: convolutional neural networks for MATLAB. In: Proceedings of the 23rd ACM International Conference on Multimedia. New York, USA: ACM, 2015. 689-692
  • 加载中
图(9) / 表(2)
计量
  • 文章访问数:  3239
  • HTML全文浏览量:  618
  • PDF下载量:  628
  • 被引次数: 0
出版历程
  • 收稿日期:  2018-05-22
  • 录用日期:  2018-11-05
  • 刊出日期:  2019-06-20

目录

    /

    返回文章
    返回