单幅图像超分辨率重建技术研究进展

张芳; 赵东旭; 肖志涛; 耿磊; 吴骏; 刘彦北

doi:10.16383/j.aas.c200777

单幅图像超分辨率重建技术研究进展

doi: 10.16383/j.aas.c200777

张芳^{1, 2,},
赵东旭^{1, 2,},
肖志涛^{1, 2,},
耿磊^{1, 2,},
吴骏^{2, 3,},
刘彦北^{1, 2,}

1.
天津工业大学生命科学学院天津 300387
2.
天津市光电检测技术与系统重点实验室天津 300387
3.
天津工业大学电子与信息工程学院天津 300387

基金项目: 天津市高等学校创新团队培养计划(TD13-5034)资助

详细信息

作者简介:
张芳：天津工业大学生命科学学院教授. 2009年获天津大学大学光学工程专业博士学位. 主要研究方向为图像处理与模式识别. E-mail: hhzhangfang@126.com

赵东旭：天津工业大学生命科学学院硕士研究生. 2018年获天津工业大学电子信息工程专业学士学位. 主要研究方向为图像处理. E-mail: zhaodongxu1028@163.com

肖志涛：天津工业大学生命科学学院教授. 2003 年获天津大学电子信息工程学院博士学位. 主要研究方向为智能信号处理和图像处理与模式识别. 本文通信作者. E-mail: xiaozhitao@tiangong.edu.cn

耿磊：天津工业大学生命科学学院教授. 2012年获天津大学精密仪器与光电子工程学院博士学位. 主要研究方向为图像处理与模式识别,智能信号处理技术与系统和DSP系统研发. E-mail: genglei@tiangong.edu.cn

吴骏：天津工业大学电子与信息工程学院副教授. 2007 年获天津大学电子信息工程学院博士学位. 主要研究方向为图像处理与模式识别和人工神经网络. E-mail: wujun@tiangong.edu.cn

刘彦北：天津工业大学生命科学学院副教授. 2017年获天津大学大学电路与系统专业博士学位. 主要研究方向为机器学习与医学数据分析. E-mail: liuyanbei@tiangong.edu.cn

计量
- 文章访问数: 2099
- HTML全文浏览量: 1059
- PDF下载量: 560
- 被引次数: 0
出版历程
- 收稿日期: 2020-09-28
- 网络出版日期: 2021-08-11
- 刊出日期: 2022-11-22

Research Progress of Single Image Super-resolution Reconstruction Technology

ZHANG Fang^{1, 2
,},
ZHAO Dong-Xu^{1, 2
,},
XIAO Zhi-Tao^{1, 2
,},
GENG Lei^{1, 2
,},
WU Jun^{2, 3
,},
LIU Yan-Bei^{1, 2
,}

1.
School of Life Science, Tiangong University, Tianjin 300387
2.
Tianjin Key Laboratory of Optoelectronic Detection Technology and System, Tianjin 300387
3.
School of Electronics and Information Engineering, Tiangong University, Tianjin 300387

Funds: Supported by Innovative Research Team Program of Tianjin Universities (TD13-5034)

More Information

Author Bio:
ZHANG Fang　Professor at the School of Life Science, Tiangong University. She received her Ph.D. degree in optical engineering from Tianjin University in 2009. Her research interest covers image processing and pattern recognition

ZHAO Dong-Xu　Master student at the School of Life Sciences, Tiangong University. She received her bachelor degree from the School of Electronic Information Engineering, Tiangong University in 2018. Her main research interest is image processing

XIAO Zhi-Tao　Professor at the School of Life Science, Tiangong University. He received his Ph.D. degree from the School of Electronics and Information Engineering, Tianjin University in 2003. His research interest covers intelligent signal processing, image processing and pattern recognition. Corresponding author of this paper

GENG Lei　Professor at the Sch-ool of Life Science, Tiangong University. He received his Ph.D. degree from the School of Precision Instrument and Opto-electronics Engineering, Tianjin University in 2012. His research interest covers image processing and pattern recognition, intelligent signal processing technology and system, DSP system research and development

WU Jun　Associate professor at the School of Electronics and Information Engineering, Tiangong University. He received his Ph.D. degree from the School of Electronics and Information Engineering, Tianjin University in 2007. His research interest covers image processing and pattern recognition, and artificial neural network

LIU Yan-Bei　Associate professor at the School of Life Science, Tiangong University. He received his Ph.D. degree in circuit and system from Tianjin University in 2017. His research interest covers machine learning and medical data analysis

摘要

摘要: 图像分辨率是衡量一幅图像质量的重要标准. 在军事、医学和安防等领域, 高分辨率图像是专业人士分析问题并做出准确判断的前提. 根据成像采集设备、退化因素等条件对低分辨率图像进行超分辨率重建成为一个既具有研究价值又极具挑战性的难点问题. 首先简述了图像超分辨率重建的概念、重建思想和方法分类; 然后重点分析用于单幅图像超分辨率重建的空域方法, 梳理基于插值和基于学习两大类重建方法中的代表性算法及其特点; 之后结合用于超分辨率重建技术的数据集, 重点分析比较了传统超分辨率重建方法和基于深度学习的典型超分辨率重建方法的性能; 最后对图像超分辨率重建未来的发展趋势进行展望.
- 超分辨率重建 /
- 单幅图像 /
- 空域方法 /
- 深度学习
Abstract: Image resolution is an important criterion to measure the quality of an image. High-resolution images are a prerequisite for professionals to analyze problems and make accurate judgments in the fields of military, medicine, and security. The super-resolution reconstruction of low-resolution images according to conditions such as imaging acquisition equipment and degradation factors has become a difficult problem that is both valuable and challenging for research. This paper first briefly describes the concept, reconstruction ideas and method classification of image super-resolution reconstruction. Secondly, the spatial methods for single image super-resolution reconstruction are analyzed, and the representative algorithms and their characteristics of the interpolation-based method and learning-based method are sorted out. Then, combined with the data set used for super-resolution reconstruction technology, the performances of traditional super-resolution reconstruction method and typical super-resolution reconstruction method based on deep learning are analyzed and compared. Finally, the future development trend of image super-resolution reconstruction is prospected.
- Super resolution reconstruction /
- single image /
- spatial method /
- deep learning

HTML全文

图 1 单幅图像SR重建方法分类图

Fig. 1 Classification of single image SR reconstruction method

下载: 全尺寸图片幻灯片

图 2 双三次插值过程示意图

Fig. 2 Schematic diagram of the bicubic interpolation

下载: 全尺寸图片幻灯片

图 3 基于深度学习的SR方法网络结构图

Fig. 3 Network structure of SR method based on deep learning

下载: 全尺寸图片幻灯片

图 4 不同尺度模型SR结构

Fig. 4 SR structure with different scales

下载: 全尺寸图片幻灯片

图 5 SR重建方法本质的联系和差异

Fig. 5 Relations and differences of SR reconstruction methods

下载: 全尺寸图片幻灯片

图 6 基于传统小波变换和与深度学习相结合的小波变换SR重建方法流程图

Fig. 6 SR reconstruction method based on traditional wavelet transform and wavelet transform combined with deep learning

下载: 全尺寸图片幻灯片

表 1 典型深度学习网络内部结构

Table 1 The internal structure of a typical deep learning network

方法	网络结构	作用
VDSR^[78]	残差学习	加快深度网络收敛
DRCN^[79]	递归监督、跳跃连接	减缓梯度爆炸或梯度消失, 存储输入信号用于目标预测
DRRN^[82]	全局残差学习	学习复杂特征, 帮助梯度传播
	局部残差学习	携带丰富的细节信息
	递归块	权值共享, 多路径递归连接
SRDenseNet^[83]	密集跳跃连接	增强不同层间的特征融合
EDSR^[91]	残差块	增强初始层级与深度层级的联系
MemNet^[85]	内存块	自适应地学习不同内存的不同权重
	递归单元	控制应该保留多少长期内存
	门单元	存储多少短期内存
RDN^[86]	残差密集块	读取前一个RDN状态, 增强层间连接
RDN^[86]	连续记忆机制	全局特征融合, 挖掘分层信息
SRFBN^[96]	反馈块、反馈机制	共享权重, 帮助更好的高级信息表达; 高级信息回传给低级信息
RCAN^[99]	通道注意力机制	分级标定图像低级和高级语义信息

下载: 导出CSV

表 2 SR网络输入及层数对照表

Table 2 Comparison of SR network input and layer number

方法	网络输入	网络层数
SRCNN	LR + BI	3
FSRCNN	LR	8
ESPCN	LR	3
VDSR	LR ＋ BI	20
DRCN	LR ＋ BI	20
LapSRN	LR	27
RED	LR	30
DRRN	LR + BI	52
SRDenseNet	LR	64
SRGAN	LR + BI	54
MemNet	LR ＋ BI	80
RDN	LR	20 (RDB)

下载: 导出CSV

表 3 SR重建图像常用质量评价方法

Table 3 Common quality evaluation methods for SR reconstructed images

特点		类别	常用评估方法	适用场景	优缺点		使用方法
主观	全参考	基于评分	MOS/DMOS	不受距离、设备、光照、及观测者的视觉能力、情绪等因素影响的情况	优点: 能够真实的反映图像的直观质量, 评价结果可靠, 无技术障碍.	缺点: 无法应用数学模型对其进行描述, 耗时多、费用高. 易受观测动机、观测环境等诸多因素的影响.	根据评分表分别对参考图像和待测图像评分
客观	全参考 (真值图像 + 失真图像)	基于像素	MSR/PSNR	—	优点: 计算形式上非常简单, 物理意义理解也很清晰.	缺点: 未考虑将人类视觉系统特性, 单纯从数学角度来分析差异, 未与图像的感知质量产生联系.	—
		基于人类视觉系统 (结构和特征)	SSIM/MS-SSIM/ FSIM/VIF/IFC	参考图像完整的情况	优点: 从整体上直接模拟HVS(人类视觉系统)抽取对象结构的人类视觉功能, 更符合视觉感知.	缺点: 从图像像素值的全局统计出发, 未考虑人眼的局部视觉因素, 对于图像局部质量无从把握.	所有像素点对应比较
		基于深度学习	NAR-DCNN^[145]/ LPIPS^[146]	—	—	—	—
	盲参考 (失真图像)	基于感知/概率模型	PI^[147]/Ma^[148]/ NIQE^[149]/ BLIINDS^[150]/ BIQI^[151]/ BRISQUE^[151]	无参考图像的情况. 无需参考图像, 灵活性强.	优点: 直接从原始图像像素学习判别图像特征, 而不使用手工提取特征.	共性: 首先对理想图像的特征做出某种假设, 转化成一个分类或回归问题; 再为该假设建立相应的数学分析模型, 学习特征; 最后通过计算待评图像在该模型下的表现特征, 从而得到图像的质量评价结果.	特征由自然场景统计提取
	盲参考 (失真图像)	基于深度学习 (网络模型)	DB-CNN^[152]/ RankIQA^[153]/ DIQI^[154]	无参考图像的情况. 无需参考图像, 灵活性强.	优点: 直接从原始图像像素学习判别图像特征, 而不使用手工提取特征.		CNN/CNN+回归模型提取特征

下载: 导出CSV

参考文献(154)

[1]	Park S C, Park M K, Kang M G. Super-resolution image reconstruction: A technical overview. IEEE Signal Processing Magazine, 2003, 20(3): 21-36. doi: 10.1109/MSP.2003.1203207
[2]	Morin R, Basarab A, Kouame D. Alternating direction method of multipliers framework for super-resolution in ultrasound imaging. In: Proceedings of the 9th IEEE International Symposium on Biomedical Imaging. Barcelona, Spain: IEEE, 2012. 1595−1598
[3]	Cui J, Wang Y, Huang J, Tan T, Sun Z. An iris image synthesis method based on PCA and super-resolution. In: Proceedings of the 17th International Conference on Pattern Recognition. Cambridge, UK: 2004. 471−474
[4]	Nguyen K, Sridharan S, Denman S, Fookes C. Feature domain super-resolution framework for Gabor-based face and iris recognition. In: Proceedings of the 25th IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE, 2012. 2642−2649
[5]	Harris J L. Diffraction and resolving power. Journal of the Optical Society of America, 1964, 54(7): 931-933. doi: 10.1364/JOSA.54.000931
[6]	Goodman J W. Introduction to Fourier Optics. New York: McGraw-Hill, 1968.
[7]	Tsai R, Huang T. Multiframe image restoration and registration. Computer Vision and Image Processing, 1984, 1(2): 317-339.
[8]	Yang J, Wright J, Huang T S, Ma Y. Image super-resolution as sparse representation of raw image patches. In: Proceedings of the 26th IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Anchorage, USA: IEEE, 2008.
[9]	Yang J, Wright J, Huang T S, Ma Y. Image super-resolution via sparse representation. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873. doi: 10.1109/TIP.2010.2050625
[10]	Nguyan N, Golub G, Milanfar P. Preconditioners for regularized image super resolution. In: Proceedings of the 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Piscataway, USA: IEEE, 1999. 3249−3252
[11]	Yu G, Sapiro G, Mallat S. Solving inverse problems with piecewise linear estimators: From gaussian mixture models to structured sparsity. IEEE Transactions on Image Processing, 2012, 21(5): 2481-2499. doi: 10.1109/TIP.2011.2176743
[12]	Elad M, Feuer A. Restoration of a single super resolution image from several blurred, noisy, and under sampled measured images. IEEE Transactions on Image Processing, 1997, 6(12): 1646-1658. doi: 10.1109/83.650118
[13]	Timofte R, De Smet V, Van Gool L. A+: Adjusted anchored neighborhood regression for fast super-resolution. In: Proceedings of the 12th Asian Conference on Computer Vision. Singapore: 2014. 111−126
[14]	Cui Z, Chang H, Shan S, Zhong B, Chen X. Deep network cascade for image super-resolution. In: Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland: 2014. 49−64
[15]	Song H, Zhang L, Wang P, Zhang K, Li X. AN adaptive L1-L2 hybrid error model to super-resolution. In: Proceedings of the 2010 IEEE International Conference on Image Processing. Hong Kong, China: IEEE, 2010. 2821−2824
[16]	Wang S, Zhang L, Liang Y, Pan Q. Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE, 2012. 2216−2223
[17]	Dong C, Loy C C, He K, Tang X. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 38(2): 295-307.
[18]	Zhu Y, Zhang Y, Yuille A L. Single image super-resolution using deformable patches. In: Proceedings of the 27th IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 2917−2924
[19]	Gao X, Zhang K, Tao D, Li X. Image super-resolution with sparse neighbor embedding. IEEE Transactions on Image Processing, 2012, 21(7): 3194-3205. doi: 10.1109/TIP.2012.2190080
[20]	Dong W, Zhang L, Shi G, Wu X. Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Transactions on Image Processing, 2011, 20(7): 1838-1857. doi: 10.1109/TIP.2011.2108306
[21]	岳波. 基于学习的图像超分辨率重建方法研究[博士论文], 西安电子科技大学, 中国, 2019 Yue Bo. Study on Learning-Based Image Super-Resolution Method [Ph.D. dissertation], Xidian University, China, 2019
[22]	苏衡, 周杰, 张志浩. 超分辨率图像重建方法综述[J]. 自动化学报, 2013, 39(8): 1202-1213. Su Heng, Zhou Jie, Zhang Zhi-Hao. Survey of Super-resolution Image Reconstruction Methods. Acta Automatica Sinica, 2013, 39(8): 1202-1213(in Chinese)
[23]	孙旭, 李晓光, 李嘉锋, 卓力. 基于深度学习的图像超分辨率复原研究进展[J]. 自动化学报, 2017, 43(5): 697-709. Sun Xu, Li Xiao-Guang, Li Jia-Feng, Zhuo Li. Review on Deep Learning Based Image Super-resolution Restoration Algorithms. Acta Automatica Sinica, 2017, 43(5): 697-709(in Chinese)
[24]	Wang Z H, Chen J, C. H. Hoi S. Deep Learning for Image Super-resolution: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence2021, 43(10): 3365-3387
[25]	Baker S, Kanade T. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(9): 1167-1183. doi: 10.1109/TPAMI.2002.1033210
[26]	Gerchberg R W. Super-resolution through error energy reduction. Journal of Modern Optics, 1974, 21(9): 709-720.
[27]	Santis P D, Gori F. On an iterative method for super-resolution[J]. Journal of Modern Optics, 1975, 22(8): 691-695.
[28]	Prashanth H S, Shashidhara H L, Balasubramanya M K N. Image scaling comparison using universal image quality index. In: Proceedings of the 2009 International Conference on Advances in Computing, Control and Telecommunication Technologies. Trivandrum, India: IEEE, 2009. 859−863
[29]	Gribbon K T, Bailey D G. A novel approach to real-time bilinear interpolation. In: Proceedings of the 2nd IEEE International Workshop on Electronic Design, Test and Application. Perth, Australia: IEEE, 2004. 126−131.
[30]	Keys R G. Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1981, 29(6): 1153-1160. doi: 10.1109/TASSP.1981.1163711
[31]	Kwok W, Sun H. Multi-directional interpolation for spatial error concealment. IEEE Transactions on Consumer Electronics, 1993, 39(3): 455-460. doi: 10.1109/30.234620
[32]	Li X, Orchard M T. New edge-directed interpolation. IEEE Transactions on Image Processing, 2001, 10(10): 1521-1527. doi: 10.1109/83.951537
[33]	Chen M J, Huang C H, Lee W L. A fast edge-oriented algorithm for image interpolation. Image and Vision Computing, 2005, 23(9): 791-798. doi: 10.1016/j.imavis.2005.05.005
[34]	Zhang X, Wu X. Image interpolation by adaptive 2D autoregressive modeling and soft-decision estimation. IEEE Transactions on Image Processing, 2008, 17(6): 887-896. doi: 10.1109/TIP.2008.924279
[35]	Hennings-Yeomans P H, Baker S, Kumar B V K V. Simultaneous super-resolution and feature extraction for recognition of low-resolution faces. In: Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, USA: IEEE, 2008.
[36]	Babacan S D, Molina R, Katsaggelos A K. Total variation super resolution using a variational approach. In: Proceedings of the 15th IEEE International Conference on Image Processing. San Diego, USA: IEEE, 2008. 641−644
[37]	Farsiu S, Robinson M D, Elad M, Milanfar P. Fast and robust multiframe super resolution. IEEE Transactions on Image Processing, 2004, 13(10): 1327-1344. doi: 10.1109/TIP.2004.834669
[38]	Aly H A, Dubois E. Image up-sampling using total-variation regularization with a new observation model. IEEE Transactions on Image Processing, 2005, 14(10): 1647-1659. doi: 10.1109/TIP.2005.851684
[39]	Freeman W T, Pasztor E C, Carmichael O T. Learning low-level vision. In: Proceedings of the 7th IEEE International Conference on Computer Vision. Piscataway, USA: 1999. 1182− 1189
[40]	Wang Q, Tang X, Shum H. Patch based blind image super resolution. In: Proceedings of the 10th IEEE International Conference on Computer Vision. Beijing, China: IEEE, 2005. 709−716
[41]	Chang H, Yeung D Y, Xiong Y. Super resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington DC, USA: IEEE, 2004. 1275−1282
[42]	Chan T M, Zhang J, Pu J, Huang H. Neighbor embedding based super-resolution algorithm through edge detection and feature selection. Pattern Recognition Letters, 2009, 30(5): 494-502. doi: 10.1016/j.patrec.2008.11.008
[43]	Gao X, Zhang K, Tao D, Li X. Joint Learning for single-image super-resolution via a coupled constraint. IEEE Transactions on Image Processing, 2012, 21(2): 469-480. doi: 10.1109/TIP.2011.2161482
[44]	Aharon M, Elad M, Bruckstein A. K-SVD: an algorithm for designing overcompletes dictionaries for sparse representation. IEEE Transactions on Signal Processing, 2006, 54(11): 4311-4322. doi: 10.1109/TSP.2006.881199
[45]	Zhang L, Dong W, Zhang D, Shi G. Two-stage image denoising by principal component analysis with local pixel grouping[J]. Pattern Recognition, 2010, 43(4): 1531-1549. doi: 10.1016/j.patcog.2009.09.023
[46]	Tropp J, Gilbert A. Signal Recovery from Random Measurements via Orthogonal Matching pursuit. IEEE Transactions Information Theory, 2007, 53(12): 4655-4666. doi: 10.1109/TIT.2007.909108
[47]	Daubechies, Defrise M, Mol C D. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics, 2004, 57(11): 1413-1457. doi: 10.1002/cpa.20042
[48]	潘宗序, 禹晶, 胡少兴, 孙卫东. 基于多尺度结构自相似性的单幅图像超分辨率算法. 自动化学报, 2014, 40(04): 594-603. Pan Zong-Xu, Yu Jing Hu Shao-Xing, Sun Wei-Dong. Single Image Super Resolution Based on Multi-scale Structural Self-similarity. Acta Automatica Sinica, 2014, 40(04): 594-603.
[49]	Zeyde R, Elad M, Protter M. On single image scale-up using sparse-representations. In: Proceedings of the 7th International Conference on Curves and Surfaces, Curves and Surfaces. Avi-gnon, France: 2012. 711−730
[50]	Yang J, Wang Z, Lin Z, Cohen S, Huang T. Couple dictionary training for image super-resolution[J]. IEEE Transactions on Image Processing, 2012, 21(8): 3467-3487. doi: 10.1109/TIP.2012.2192127
[51]	He L, Qi H, Zaretzki R. Beta process joint dictionary learning for coupled feature spaces with application to single image super-resolution. In: Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: 2013. 345−352
[52]	Yang W, Tian Y, Zhou F. Consistent coding scheme for single image super-resolution via independent dictionaries[J]. IEEE Transactions on Multimedia, 2016, 18 (3): 313-325. doi: 10.1109/TMM.2016.2515997
[53]	Zhao J, Hu H, Cao F. Image super-resolution via adaptive sparse representation[J]. Knowledge-Based Systems, 2017, 124(5): 23-33.
[54]	Wang J, Zhu S, Gong Y. Resolution-invariant image representation and its applications. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Mia-mi, USA: IEEE, 2009. 2512−2519
[55]	Lu X, Yuan H, Yan P, Yuan Y, Li X. Geometry constrained sparse coding for single image super resolution. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA: IEEE, 2012. 1648−1655
[56]	Dong W, Zhang L, Shi G. Centralized sparse representation for image restoration. In: Proceedings of the 2011 IEEE International Conference on Computer Vision. Barcelona, Spain: 2011. 1259−1266
[57]	Dong W S, Zhang L, Lukac R, Shi G. Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Transactions on Image Processing, 2013, 22(4): 1382-1394. doi: 10.1109/TIP.2012.2231086
[58]	Glasner D, Bagon S, Irani M. Super-resolution from a single image. In: Proceedings of the 12th IEEE International Conference on Computer Vision. Kyoto, Japan: IEEE, 2009. 349−356
[59]	Dong W, Zhang L, Shi G, Li X. Nonlocally centralized sparse representation for image restoration. IEEE Transactions on Image Processing, 2013, 22(4): 1620-1630. doi: 10.1109/TIP.2012.2235847
[60]	Yang S. Wang, M. Sun Y, Sun F, Jiao L. Compressive sampling based single-image super-resolution reconstruction by dual-sparsity and non-local similarity regularizer. Pattern Recognition Letters, 2012, 33(9): 1049-1059.
[61]	Li J, Gong W, Li W. Dual-sparsity regularized sparse representation for single image super-resolution. Information Sciences, 2015, 298(3): 257-273.
[62]	Shi J, Qi C. Low-rank sparse representation or single image super-resolution via self-similarity learning. In: Proceedings of the 23rd IEEE International Conference on Image Processing. Pho-enix, USA: IEEE, 2016. 1424−1428
[63]	Li J, Wu J, Deng H, Liu J. A self-learning image super-resolution method via sparse representation and non-local similarity. Neurocomputing, 2016, 184(5): 196-206.
[64]	李进明. 基于稀疏表示的图像超分辨率重建方法研究[博士论文]. 重庆大学, 中国, 2015 Li Jin-Ming. Research on Sparse Representation Based Image Super-Resolution Reconstruction Method[Ph.D. dissertation], Chongqing University, China, 2015
[65]	Lu X, Yuan Y, Yan P. Alternatively constrained dictionary learning for image super resolution. IEEE transactions on Cybernetics, 2014, 44(3): 366-377. doi: 10.1109/TCYB.2013.2256347
[66]	Kim K I, Kwon Y. Single-image super-resolution using sparse regression and natural image prior. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(6): 1127-1133. doi: 10.1109/TPAMI.2010.25
[67]	Kim K I, Kwon Y. Example-based learning for single-image super-resolution. In: Proceedings of the 30th DAGM Symposium on Pattern Recognition. Munich, Germany: 2008. 456−465
[68]	Deng C, Xu J, Zhang K, Tao D, Gao X, Li X. Similarity constraints-based structured output regression machine: An approach to image super-resolution. IEEE Transactions on Neural Networks and Learning Systems, 2016, 27(12): 2472-2485. doi: 10.1109/TNNLS.2015.2468069
[69]	He H, Siu W C. Single image super-resolution using Gaussian process regression. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado, USA: IEEE, 2011. 449−456
[70]	Wang H, Gao X, Zhang K, Li J. Single image super-resolution using Gaussian process regression with dictionary-based sampling and student-t likelihood. IEEE Transactions on Image Processing, 2017, 26(7): 3556-3568.
[71]	Timofte R, De V, Gool L V. Anchored neighborhood regression for fast example-based super-resolution. In: Proceedings of the 14th IEEE International Conference on Computer Vision. Sydney, Australia: IEEE, 2013. 1920−1927
[72]	Timofte R, Van Gool L. Adaptive and weighted collaborative representations for image classification. Pattern Recognition Letters, 2014, 43(1): 127-135.
[73]	Yang C Y, Yang M H. Fast direct super-resolution by simple functions. In: Proceedings of the 14th IEEE International Conference on Computer Vision. Sydney, Australia: 2014. 561−568
[74]	Zhang K, Tao D, Gao X, Li X, Xiong Z. Learning multiple linear mappings for efficient single image super- resolution. IEEE Transactions on Image Processing, 2015, 24(3): 846-861. doi: 10.1109/TIP.2015.2389629
[75]	Sun J, Cao W, Xu Z, Sun J, Cao W, Xu Z, et al. Learning a convolutional neural network for non-uniform motion blur removal. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: 2015. 769−777
[76]	Dong C, Loy C C, Tang X. Accelerating the super-resolution convolutional neural network. In: Proceedings of the 14th European Conference on Computer Vision. Amsterdam, Netherlands: 2016. 391−407
[77]	Shi W, Caballero J, Huszar F, Totz J, Aitken A, Bishop R, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 1874−1883
[78]	Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 1646−1654
[79]	Kim J, Lee J M, Lee K M. Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recogniti-on. Las Vegas, USA: IEEE, 2016. 1637−1645
[80]	Lai W S, Huang J B, Ahuja N, Yang M H. Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 5835−5843
[81]	Mao X J, Shen C, Yang Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Proceedings of the 30th Annual Conference on Neural Information Processing Systems. Barcelona, Spain: 2016. 2810−2818
[82]	Tai Y, Yang J, Liu X. Image super-resolution via deep recursive residual network. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 2790−2798
[83]	Tong T, Li G, Liu X, Gao Q. Image super-resolution using dense skip connections. In: Proceedings of the 16th IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 4799−4807
[84]	Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, et al. Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 105−114
[85]	Tai Y, Yang J, Liu X, Xu C. MemNet: A persistent memory network for image restoration. In: Proceedings of the 16th International Conference on Computer Vision. Venice, Italy: 2017. 4539−4547
[86]	Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y. Residual dense network for image super-resolution. In: Proceedings of the 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 2472−2481
[87]	Wang Z, Liu D, Yang J, Han W, Huang T. Deep networks for image super-resolution with sparse prior. In: Proceedings of the 15th IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015: 370−378
[88]	He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 770−778
[89]	Huang G, Liu Z, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 2261−2269
[90]	Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial nets. In: Proceedings of the 28th Annual Conference on Neural Information Processing Systems. Montreal, Canada: 2014. 2672−2680
[91]	Lim B, Son S, Kim H, Nah S, Lee K M. Enhanced deep residual networks for single image super-resolution. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, USA: 2017. 1132−1140
[92]	Feng Z, Lai J, Xie X, Zhu J. Image super-resolution via a densely connected recursive network. Neurocomputing, 2018, 316(11): 270-276.
[93]	Yu J, Fan Y, Yang J, Xu N. Wide activation for efficient and accurate image super-resolution, Technical report and factsheet [Online], available: https://arxiv.org/abs/1808.08718, December 21, 2018.
[94]	Sha F, Zandavi S M, Chung Y Y. Fast deep parallel residual network for accurate super resolution image processing. Expert Systems with Applications, 2019, 128(8): 157-168.
[95]	Li Z, Li Q, Wu W, Yang J, Li Z, Yang X. Deep recursive up-down sampling networks for single image super-resolution. Neurocomputing, 2020, 398(7): 377-388.
[96]	Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W. Feedback network for image super-resolution. In: Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 3862−3871
[97]	Cao Y, He Z, Ye Z, Li X, Cao Y, Yang J. Fast and accurate single image super-resolution via an energy-aware improved deep residual network. Signal Processing, 2019, 162(9): 115-125.
[98]	Zareapoor M, Celebi M. E, Yang J. Diverse adversarial network for image super resolution. Signal Processing: Image Communication, 2019, 74(5): 191-200.
[99]	Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y. Image super-resolution using very deep residual channel attention networks. In: Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: Springer, 2018. 294−310
[100]	He K, Zhang X, Ren S, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916. doi: 10.1109/TPAMI.2015.2389824
[101]	Hu X, Mu H, Zhang X, Wang Z, Tan T, Sun J. Meta-SR: A magnification-arbitrary network for super-resolution. In: Proce-edings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 1575−1584
[102]	Zhang F, Cai N, Cen G, Li F, Wang H, Chen X. Image super-resolution via a novel cascaded convolutional neural network framework. Signal Processing Image Communication, 2018, 63(4): 9-18.
[103]	Cai J, Zheng H, Yong H, Cao Z, Zhang L. Toward real-world single image super-resolution: A new benchmark and a new model. In: Proceedings of the 17th IEEE/CVF International Conference on Computer Vision. Seoul, South Korea: IEEE, 2019. 3086−3095
[104]	Wang L, Wang Y, Liang Z, Lim Z, Yang J, An W, et al. Learning parallax attention for stereo image super-resolution. In: Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: 2019. 12242−12251
[105]	Pan Z, Li B, Xi T, Fan Y, Zhang G, Liu J, et al. Real image super resolution via heterogeneous model using GP-NAS. In: Proceedings of the the 16th European Conference on Compu-ter Vision. Glasgow, United kingdom: Springer, 2020. 423− 436
[106]	Zhang Z, Wang Z, Lin Z, Qi H. Image super-resolution by neural texture transfer. In: Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019. 7974−7983
[107]	Bulat A, Ynag J, Tzimiropoulos G. To learn image super-resolution, use a GAN to learn how to do image degradation first. In: Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: 2018. 187−202
[108]	Zhang K, Zuo W, Zhang L. Deep plug-and-play super-resolution for arbitrary blur kernels. In: Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019. 1671−1681
[109]	Song X, Dai Y, Zhou D, Liu L, Li W, Li H, et al. Channel attention based iterative residual learning for depth map super-resolution. In: Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Virtual Event: IEEE, 2020. 5631−5640
[110]	Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 2017, 26(7): 3142-3155. doi: 10.1109/TIP.2017.2662206
[111]	Zhang K, Zuo W M, Zhang L. FFDNet: Toward a fast and flexible solution for CNN-based image denoising. IEEE Transactions on Image Processing, 2018, 27(9): 4608-4622. doi: 10.1109/TIP.2018.2839891
[112]	Guo S, Yan Z, Zhang K, Zuo W, Zhang L. Toward convolutional blind denoising of real photographs. In: Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019. 1712−1722
[113]	Zamir S W, Arora A, Khan S, Hayat M, Khan F S, Yang M H, et al. Learning enriched features for real image restoration and enhancement. In: Proceedings of the 16th European Conferen-ce on Computer Vision. Glasgow, UK: 2020. 492−511
[114]	Zhang K, Gool L V, Timofte R. Deep unfolding network for image super-resolution. In: Proceedings of the 33rd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Virtual Event: IEEE, 2020. 3214−3223
[115]	Ford C, Etter D M. Wavelet basis reconstruction of nonuniformly sampled data. IEEE Transactions on Circuits and Systems II Analog and Digital Signal Processing, 1998, 45(8): 1165-1168. doi: 10.1109/82.718832
[116]	Nguyen N, Milanfar P. A wavelet-based interpolation-restoration method for super resolution. Circuits Systems & Signal Processing, 2000, 19(4): 321-338.
[117]	汪雪林, 文伟, 彭思龙. 基于小波域局部高斯模型的图像超分辨率. 中国图象图形学报, 2004, 9(8): 941-946. doi: 10.3969/j.issn.1006-8961.2004.08.008 Wang Xue-Lin, Wen Wei, Peng Si-Long. Image super resolution based on wavelet-domain local gaussian model. Journal of Image and Graphics, 2004, 9(8): 941-946(in Chinese) doi: 10.3969/j.issn.1006-8961.2004.08.008
[118]	Shen L X, Sun Q X. Biorthogonal wavelet system for high-resolution image reconstruction. IEEE Transactions on Signal Processing, 2004, 52(7): 1997-2011. doi: 10.1109/TSP.2004.828939
[119]	Zhao S, Han H, Peng S. Wavelet-domain HMT-based image super-resolution. In: Proceedings of the 2003 IEEE International Conference on Image Processing. Barcelona, Spain: IEEE, 2003. 656−953
[120]	Zhang Q, Wang H, Yang S. Image super-resolution using a wavelet-based generative adversarial network. Computer Vision and Pattern Recognition [Online], available: https://arxiv. org/abs/1907.10213, May 6, 2021.
[121]	Demirel H, Anbarjafari G. Image resolution enhancement by using discrete and stationary wavelet decomposition. IEEE Transactions on Image Processing, 2011, 20(5): 1458-1460. doi: 10.1109/TIP.2010.2087767
[122]	Mallat S. A theory for multiresolution in signal decomposition: the wavelet representation. IEEE Transactions. on Pattern Analysis and Machine Intelligence, 1989, 11(7): 674-683. doi: 10.1109/34.192463
[123]	Chavez-Roman H, Ponomaryov V. Super resolution image generation using wavelet domain interpolation with edge extraction via a sparse representation. IEEE Geoscience and Remote Sensing Letters, 2014, 11(10): 1777-1781. doi: 10.1109/LGRS.2014.2308905
[124]	Patil V H, Bormane D S, Pawar V S. Super-resolution using neural network. In: Proceedings of the 2nd Asia International Conference on Modeling and Simulation. Kuala Lumpur, Mal-aysia: IEEE, 2008. 492−496
[125]	Asokan A, Anitha J. Lifting wavelet and discrete cosine transform-based super-resolution for satellite image fusion. In: Proce-eding of the 2021 International Conference on Computational Methods and Data Engineering. Sonipat, India: Springer, 2021. 5−12
[126]	Ji H, Fermüller C. Robust wavelet-based super-resolution reconstruction: Theory and algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2009, 31(4): 649-660. doi: 10.1109/TPAMI.2008.103
[127]	张丽. 小波变换和深度学习单幅图像超分辨率算法研究[硕士论文], 信阳师范学院, 中国, 2019 Zhang Li. Research on Wavelet Transform and Deep Learning Super-Resolutiom Algorithm for Single Image[Master thesis], Xinyang Normal University, China, 2019
[128]	段立娟, 武春丽, 恩擎, 乔元华, 张韵东, 陈军成. 基于小波域的深度残差网络图像超分辨率算法. 软件学报, 2019, 30(4): 941-953. Duan Li-Juan, Wu Chun-Li En Qing, Qiao Yuan-Hua, Zhang Yun-Dong, Chen Jun-Cheng. Deep residual network in wavelet domain for image super-resolution. Journal of Software, 2019, 30(4): 941-953 (in Chinese)
[129]	孙超, 吕俊伟, 宫剑, 仇荣超, 李健伟, 伍恒. 结合小波变换与深度网络的图像超分辨率方法[J]. 激光与光电子学进展, 2018, 55(121006): 1-8. Sun Chao, Lv Jun-Wei, Gong Jian, Qiu Rong-Chao, Li Jian-Wei, Wu Heng. Image super-resolution method combining wavelet transform with deep network. Laser& Optoelectronics Progress, 2018, 55(121006): 1-8(in Chinese)
[130]	Wang Z, Liu D, Yang J, Han W, Huang T. Deep networks for image super-resolution with sparse prior. In: Proceedings of the 15th IEEE International Conference on Computer Vision. Santiago, Chile: IEEE, 2015. 370−378
[131]	Gregor K, LeCun Y. Learning fast approximations of sparse coding. In: Proceedings of the 27th International Conference on Machine Learning. Haifa, Israel, 2010. 399−406
[132]	Liu D, Wang Z, Wen B, Yang J, Han W, Huang T. Robust single image super-resolution via deep networks with sparse prior. IEEE Transactions on Image Processing, 2016, 25(7): 3194-3207. doi: 10.1109/TIP.2016.2564643
[133]	Wang J, Chen K, Xu R, Liu Z, Loy C C, Lin D. CARAFE: content-aware reassembly of features. In: Proceedings of the 17th IEEE/CVF International Conference on Computer Vision. Seoul, South Korea: IEEE, 2019. 3007−3016
[134]	Nasrollahi K, Moeslund T B. Super-resolution: a comprehensive survey. Machine Vision and Applications, 2014, 25(8): 1423-1468.
[135]	Martin D, Fowlkes C, Tal D. Malik J. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the 8th International Conference on Computer Vision. Vancouver, USA: IEEE 2001. 416−423
[136]	Timofte R, Agustsson E. NTIRE 2017 challenge on single image super-resolution: Dataset and study. In: Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, USA: IEEE, 2017. 1122−1131
[137]	Bevilacqua M, Roumy A, Guillemot C, Morel M L A. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the 23rd British Machine Vision Conference. Surrey, UK: 2012.
[138]	Huang J B, Singh A, Ahuja N. Single image super-resolution from transformed self-exemplars. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015. 5197−5206
[139]	Fujimoto A, Ogawa T, Yamamoto K, Matsui Y, Yamasaki T, Aizawa K. Manga109 dataset and creation of metadata. In: Proceedings of the 1st International Workshop on coMics ANalysis, Processing and Understanding. New York, USA: 2016.
[140]	Sun L, Hays J. Super-resolution from internet-scale scene matching. In: Proceedings of the 2012 IEEE International Conference on Computational Photography. Seattle, USA: IEEE, 2012.
[141]	Wei P, Xie Z, Lu H, Zhan Z, Ye Q, Zuo W, et al. Component divide and-conquer for real-world image super-resolution. In: Proceedings of the 16th European Conference on Computer Vision. Glasgow, UK: 2020. 101−117
[142]	Chen C, Xiong Z, Tian X, Zha Z J, Wu F. Camera lens super-resolution. In: Proceedings of the 32nd IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019. 1652−1660
[143]	Bychkovsky V, Paris S, Chan E, Durand F. Learning photographic global tonal adjustment with a database of input/output image pairs. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado, USA: IEEE, 2011. 97−104
[144]	Wei C, Wang W, Yang W, Liu J. Deep retinex decomposition for low-light enhancement. In: Proceedings of the 29th British Machine Vision Conference. Newcastle, UK: 2018.
[145]	Liang Y. Wang J, Wan X, Gong Y, Zheng N. Image quality assessment using similar scene as reference. In: Proceedings of the 21st ACM Conference on Computer and Communications Security. Scottsdale, USA: 2016. 3−18
[146]	Zhang R, Isola P, Efros A A, Shechtman E, Wang O. The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the 31st IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 586−595
[147]	Blau Y, Mechrez R, Timofte R, Michaeli T, Zelnik-Manor L. The 2018 PIRM challenge on perceptual image super-resolution. In: Proceedings of the 15th European Conference on Computer Vision. Munich, Germany: 2018. 334−355
[148]	Ma C, Yang C Y, Yang X, Ynag M. Learning a no-reference quality metric for single-image super-resolution. Computer Vision and Image Understanding, 2017, 158(5): 1-16.
[149]	Mittal A, Soundararajan R, Bovik A C. Making a "completely blind" image quality analyzer. IEEE Signal Processing Letter, 2013, 20(3): 209-212. doi: 10.1109/LSP.2012.2227726
[150]	Saad M A, Bovik A C, Charrier C. A DCT statistics-based blind image quality index. IEEE Signal Processing Letters, 2010, 17(6): 583-586. doi: 10.1109/LSP.2010.2045550
[151]	Ma K, Wu Q, Wang Z, Duanmu Z, Yong H, Li J, et al. Group MAD competition: A new methodology to compare objective image quality models. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition. Las Veg-as, USA: IEEE, 2016. 1664−1673
[152]	Zhang W, Ma K, Yan J, Deng D, Wang Z. Blind image quality assessment using a deep bilinear convolutional neural network. IEEE Transactions on Circuits and Systems for Video Technology, 30(1): 36−47
[153]	Liu X, Weijer J V D, Bagdanov A D. RankIQA: Learning from rankings for no-reference image quality assessment. In: Proceedings of the 16th IEEE International Conference on Computer Vision. USA: IEEE, 2017. 1040−1049
[154]	Gu K, Zhai G, Yang X, Zhang W. Deep learning network for blind image quality assessment. In: Proceedings of the 2014 IEEE International Conference on Image Processing. USA: 2014. 511−515