Image Texture Enhancement via Upscaling Algorithm Based on Contour Stencils and Self-learning
-
摘要: 提出一种以轮廓模板插值和局部自学习相结合的图像纹理增强超采样算法，有效地恢复了插值图像丢失的细节纹理，抑制了插值图像边缘的扩散.该方法通过局部自相似性在原始低分辨图像中估计高频信息，对轮廓模板插值图像的细节纹理进行了恢复.其中，为了弥补轮廓模板插值缺少先验知识的缺陷，将原始低分辨率图像的高频信息作为先验知识.为了保证估计的高频信息最优，匹配的过程中采用双匹配，相比较于全局搜索和小窗搜索，提高了效率并保证了匹配精度.此外，使用高斯模糊代替了传统提取高频信息的方法，简化了算法的复杂度，提高了准确性和效率.对估计得到的高频信息采用高斯函数加窗，以减小估计出错和重叠区的混叠影响.本文算法的训练库由原始低分辨图像自身和插值图像构成，节省了生成训练库所需的时间和空间.训练库的简化使得高频信息的估计可以多尺度进行，算法效率得到进一步优化.理论分析和实验结果表明，相比传统的基于插值、基于自学习的图像超分辨率方法，本文方法获得更好的实验结果，主观效果得到明显改善，有效地恢复了图像的纹理细节，提高了图像边缘锐度，避免了产生锯齿等人工效应，客观指标得到提高.Abstract: An image texture enhancement via upscaling algorithm based on interpolation of contour stencils and local self-learning is proposed in this paper, by which the lost detail texture of the interpolated image is effectively recovered and the edge diffusion in image magnification is decreased. In this method, the detail texture of the interpolated image is estimated by local self-similarity in the original low-resolution image. To make up the lack of prior knowledge of the interpolation algorithm, the high frequency information of original low-resolution image works as a priori knowledge. In order to estimate the best high-frequency information, the use of dual matches improves efficiency and accuracy, compared to the global search and the small window search. Instead of the traditional high frequency information extraction method, Gaussian blur is used to reduce the complexity of the algorithm. In addition, adding window on the estimated high-frequency information by Gaussian function inhibits the aliasing effect of overlapping area, as well as reduces the influence of the estimation error. To save space and time, the training library in this paper consists of original low-resolution image and the interpolated image. Simplifying training library makes high-frequency information estimated in a single multi-scale, so the algorithm efficiency is further improved. Theoretical analysis and experimental results show that, the proposed method outperforms other comparing algorithms including the traditional interpolation algorithm and the related algorithms based on self-learning in terms of both objective and visual quality of the interpolation image. The method recovers the detail texture of the image effectively, and enhances the sharpness, also avoids producing the artificial such as sawtooth effect.
-
Key words:
- Upscaling /
- image interpolation /
- self-learning /
- texture enhancement
-
图 11 不同高斯半径下提取的高频图
Fig. 11 The extraction of high frequency diagram underdifferent radius of the Gaussian
图 12 不同高斯方差下提取的高频图
Fig. 12 Extraction of high frequency diagram underdifferent Gaussian
图 13 不同方差和不同匹配块下图像的质量
Fig. 13 Image quality under the different variance and the matching
表 1 测试图像分辨率
Table 1 The resolutions of the test images
Child Koala Girl Wheel 分辨率 128×128 161×241 151×225 207×157 
下载: 导出CSV
表 2 客观指标对比
Table 2 The comparison of objective indicators
图像 客观指标 Bicubic Getreuer Glasner Freedman 本文算法 PSNR 37.687 38.298 38.647 24.216 39.339 39.339 SSIM 0.986 0.988 0.989 0.830 0.991 EPI 0.497 0.517 0.562 0.560 0.588 Entropy 5.272 5.270 5.276 5.264 5.259 Clarity 5.260 5.450 5.901 5.064 6.173 Koala PSNR 38.716 39.101 39.483 29.431 39.738 SSIM 0.986 0.988 0.989 0.840 0.990 EPI 0.349 0.376 0.434 0.436 0.464 Entropy 4.819 4.831 4.839 4.885 4.895 Clarity 3.434 3.683 4.262 5.492 4.553 Girl PSNR 40.008 40.617 40.892 26.801 41.642 SSIM 0.991 0.992 0.993 0.860 0.994 EPI 0.444 0.467 0.518 0.533 0.541 Entropy 5.275 5.274 5.281 5.272 5.269 Clarity 3.480 3.643 4.023 3.950 4.219 Wheel PSNR 36.259 36.929 37.308 25.073 38.126 SSIM 0.980 0.983 0.984 0.821 0.986 EPI 0.517 0.551 0.603 0.649 0.663 Entropy 5.353 5.360 5.353 5.387 5.401 Clarity 5.140 5.454 6.029 6.177 6.621
下载: 导出CSV
表 3 所有测试图片的平均指标
Table 3 The average index of all test images
平均指标 PSNR SSIM EPI Entropy Clarity Bicubic 33.401 0.931 0.377 5.094 3.489 Getreuer 33.874 0.938 0.401 5.099 3.698 Glasner 34.179 0.945 0.456 5.098 4.216 Freedman 26.144 0.851 0.427 5.105 3.959
下载: 导出CSV
表 4 效率对比(s)
Table 4 The contrast of efficiency (s)
图像 Glasner Freedman 本文算法 Child 1861 377 93 Koala 4314 875 287 Girl 3923 748 200 Wheel 4819 636 185
下载: 导出CSV
-
[1] Nasrollahi K, Moeslund T B. Super-resolution:a comprehensive survey. Machine Vision and Applications, 2014, 25(6):1423-1468 doi: 10.1007/s00138-014-0623-4 [2] Balure C S, Kini M R. A survey-super resolution techniques for multiple, single, and stereo images. In:Proceedings the 5th International Symposium on Electronic System Design (ISED). Surathkal:IEEE, 2014.215-216 https://www.computer.org/csdl/proceedings/ised/2014/6965/00/index.html [3] 潘宗序, 禹晶, 肖创柏, 孙卫东. 基于多尺度非局部约束的单幅图像超分辨率算法. 自动化学报, 2014, 40(10):2233-2244 http://www.aas.net.cn/CN/abstract/abstract18498.shtmlPan Zong-Xu, Yu Jing, Xiao Chuang-Bai, Sun Wei-Dong. Single-image super-resolution algorithm based on multi-scale nonlocal regularization. Acta Automatica Sinica, 2014, 40(10):2233-2244 http://www.aas.net.cn/CN/abstract/abstract18498.shtml [4] Getreuer P. Contour stencils:total variation along curves for adaptive image interpolation. SIAM Journal on Imaging Sciences, 2011, 4(3):954-979 doi: 10.1137/100802785 [5] 肖进胜, 饶天宇, 贾茜, 唐路敏, 岳显昌. 改进的自适应冲击滤波图像超分辨率插值算法. 计算机学报, 2015, 38(6):1131-1139 http://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201506003.htmXiao Jin-Sheng, Rao Tian-Yu, Jia Qian, Tang Lu-Min, Yue Xian-Chang. Interpolation algorithm based on improved adaptive shock filter in image super-resolution. Chinese Journal of Computers, 2015, 38(6):1131-1139 http://www.cnki.com.cn/Article/CJFDTOTAL-JSJX201506003.htm [6] Freedman G, Fattal R. Image and video upscaling from local self-examples. ACM Transactions on Graphics, 2011, 30(2):Article No.12 http://cn.bing.com/academic/profile?id=1976416062&encoded=0&v=paper_preview&mkt=zh-cn [7] Kawano H, Suetake N, Cha B, Aso T. Sharpness preserving image enlargement by using self-decomposed codebook and Mahalanobis distance. Image and Vision Computing, 2009, 27(6):684-693 doi: 10.1016/j.imavis.2008.07.013 [8] Zhang Y Q, Liu J Y, Yang W H, Guo Z M. Image super-resolution based on structure-modulated sparse representation. IEEE Transactions on Image Processing, 2015, 24(9):2797-2810 doi: 10.1109/TIP.2015.2431435 [9] Zhang Y Q, Xiao J S, Li S H, Shi C Y, Xie G X. Learning block-structured incoherent dictionaries for sparse representation. Science China Information Sciences, 2015, 58(10):1-15 http://cn.bing.com/academic/profile?id=888734479&encoded=0&v=paper_preview&mkt=zh-cn [10] Yang C Y, Huang J B, Yang M H. Exploiting self-similarities for single frame super-resolution. In:Proceedings of the 10th Asian Conference on Computer Vision (ACCV). Queenstown, New Zealand:Springer, 2011.497-510 [11] Glasner D, Bagon S, Irani M. Super-resolution from a single image. In:Proceedings of the 12th IEEE International Conference on Computer Vision (ICCV). Kyoto, Japan:IEEE, 2009.349-356 http://cn.bing.com/academic/profile?id=2017294772&encoded=0&v=paper_preview&mkt=zh-cn [12] Schulter S, Leistner C, Bischof H. Fast and accurate image upscaling with super-resolution forests. In:Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015.3791-3799 [13] Sajjad M, Ejaz N, Baik S W. Multi-kernel based adaptive interpolation for image super-resolution. Multimedia Tools and Applications, 2014, 72(3):2063-2085 doi: 10.1007/s11042-012-1325-4 [14] Barnsley M F. Fractal modeling of real world images. The Science of Fractal Images. New York:Springer, 1988.219-242 [15] Ebrahimi M, Vrscay E R. Solving the inverse problem of image zooming using ‘self-examples’. Image analysis and Recognition. Berlin Heidelberg:Springer, 2007.117-130 [16] Liu C, Shum H Y, Zhang C S. A two-step approach to hallucinating faces:global parametric model and local nonparametric model. In:Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). Kauai, HI, USA:IEEE, 2001. I-192-I-198 [17] Yang M-C, Wang Y-C F. A self-learning approach to single image super-resolution. IEEE Transactions on Multimedia, 2013, 15(3):498-508 doi: 10.1109/TMM.2012.2232646 [18] Wu W, Liu Z, He X H. Learning-based super resolution using kernel partial least squares. Image and Vision Computing, 2011, 29(6):394-406 doi: 10.1016/j.imavis.2011.02.001 [19] Berkeley segmentation database (BSD)[online], available:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/,March14,2014 [20] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment:from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4):600-612 doi: 10.1109/TIP.2003.819861 [21] 肖进胜, 杜康华, 涂超平, 岳显昌. 基于多聚焦图像深度信息提取的背景虚化显示. 自动化学报, 2015, 41(2):304-311 http://www.aas.net.cn/CN/abstract/abstract18609.shtmlXiao Jin-Sheng, Du Kang-Hua, Tu Chao-Ping, Yue Xian-Chang. Bokeh display based on depth information extraction of multi-focus images. Acta Automatica Sinica, 2015, 41(2):304-311 http://www.aas.net.cn/CN/abstract/abstract18609.shtml [22] Xiao J S, Li W H, Liu G X, Shaw S L, Zhang Y Q. Hierarchical tone mapping based on image colour appearance model. IET Computer Vision, 2014, 8(4):358-364 doi: 10.1049/iet-cvi.2013.0230 -
图(13) / 表(4)
计量
- 文章访问数: 2115
- HTML全文浏览量: 278
- PDF下载量: 662
- 被引次数: 0
