-
摘要: 提出一种新的非线性图像插值算法，称为基于预测梯度的图像插值（Image interpolation with predicted gradients，PGI）.首先沿用现有的边缘对比度引导的图像插值（Contrast-guided image interpolation，CGI）算法思想对低分辨率图像中的边缘进行扩散处理，然后预测高分辨率图像中未知像素的性质，最后对边缘像素采用一维有方向的插值，对非边缘像素采用二维无方向的插值.与通常的非线性图像插值算法相比，新算法对图像边缘信息的理解更为完善.与CGI算法相比，由于梯度预测策略的使用，PGI算法能够更有效地确定未知像素的相关性质（是否为边缘像素，以及是边缘像素时其边缘方向）.实验结果表明，PGI算法无论在视觉效果还是客观性测评指标方面均优于现有的图像插值算法.此外，在对彩色图像进行插值时，本文将通常的RGB颜色空间转化为Lab颜色空间，不仅减少了伪彩色的生成，而且降低了算法的时间复杂度.Abstract: A new nonlinear image interpolation algorithm is proposed, referred to as image interpolation with predicted gradients (PGI). First, the idea of contrast-guided image interpolation (CGI) is employed to diffuse the edges in the low-resolution (LR) image. Then, unknown pixels in the high-resolution (HR) image are predicted. Finally, a 1-D directional filter is employed to process edge pixels while a 2-D directionless filter is used to interpolate non-edge pixels. Compared to the common nonlinear image interpolation algorithms, the new algorithm has a better interpretation of image edges. Compared to the CGI, the PGI can predict the property of unknown pixels more precisely (including whether an unknown pixel is an edge pixel or not, and its direction if it is). Experimental results show that PGI has a better performance than the existing algorithms, either with respect to visual effect or in terms of objective criteria. In addition to interpolate color images, the usual RGB space needs to be converted to the Lab space. As a result, pseudo-color can be suppressed and the computational complexity is reduced.
-
Key words:
- Image interpolation /
- predicted gradients /
- contrast /
- gradients /
- edge
1) 本文责任编委 桑农 -
图 2 插值示意图, 其中实心点表示已知的像素(从LR图像中直接复制得到), 空心点表示未知的像素
Fig. 2 Illustration of the interpolation process (the dots denote the known pixels and the circles denote the missing pixels)
图 5 预测结果的细节比较, 箭头方向表示像素梯度方向, 箭头长短表示像素梯度大小
Fig. 5 The detail comparison of prediction results, where the direction and the length of arrow represent the direction and the size of gradient, respectively
图 9 基于LR图像像素性质推测HR图像上各像素性质的思想概要图
Fig. 9 The characteristics prediction of pixels on HR based on the characteristics of pixels on LR
图 11 不同颜色空间上彩色图像插值示意图
Fig. 11 Demonstration of color image interpolation in different color spaces
图 13 各算法对Wheel图像的插值结果比较
Fig. 13 Comparison of interpolation results on Wheel by different interpolation methods
图 14 本文算法和CGI法对测试图像的4倍插值结果比较
Fig. 14 The comparison of test images via CGI and ours with an interpolation factor $4\times 4$
图 15 各算法插值得到的图像边缘比较
Fig. 15 Comparison of interpolation results on edges by different interpolation methods
表 1 不同插值算法基于PSNR的比较(dB)
Table 1 A comparison of different interpolation methods with respect to the PSNR (dB)
测试图像 Bicubic NEDI LMMSE SAI SME NARM CGI CED 本文算法 (1981) (2001) (2006) (2008) (2010) (2013) (2013) (2016) Airplane 29.28 29.76 29.86 30.38 30.29 30.41 30.03 29.97 30.05 Bike 25.96 25.99 26.06 26.99 26.79 26.97 26.71 26.58 26.71 Boats 29.64 29.57 29.66 30.00 30.06 30.26 29.81 29.78 29.79 Bridge 25.85 25.72 25.68 25.94 25.88 25.86 25.70 25.76 25.66 Butterfly 26.17 26.88 26.44 27.40 27.39 27.32 27.68 27.46 27.68 Cameraman 25.26 25.38 25.55 25.77 26.06 25.78 25.75 25.82 25.82 Fence 23.08 21.68 23.09 22.28 23.10 23.21 23.15 23.16 23.14 House 32.06 31.84 32.47 32.73 33.08 33.23 32.70 32.57 32.76 Lena 30.19 30.57 30.50 31.34 30.94 31.38 31.07 31.03 31.11 Parthenon 25.65 25.38 25.74 25.65 25.71 25.81 25.66 25.69 25.65 Station 24.65 25.04 25.07 25.94 26.03 26.16 26.28 26.23 26.39 Wheel 19.59 21.06 19.64 21.53 21.94 20.76 22.38 22.28 22.44 峰值信噪比增益 0 0.12 0.20 0.71 0.82 0.81 0.79 0.74 0.82 
下载: 导出CSV
表 2 不同插值算法基于SSIM的比较(dB)
Table 2 A comparison of different interpolation methods with respect to the SSIM (dB)
测试图像 Bicubic NEDI LMMSE SAI SME NARM CGI CED 本文算法 (1981) (2001) (2006) (2008) (2010) (2013) (2013) (2016) Airplane 0.9261 0.9311 0.9330 0.9374 0.9357 0.9410 0.9348 0.9336 0.9355 Bike 0.8610 0.8494 0.8593 0.8782 0.8741 0.8807 0.8739 0.8702 0.8740 Boats 0.8764 0.8735 0.8752 0.8825 0.8842 0.8888 0.8794 0.8788 0.8796 Bridge 0.7982 0.7823 0.7875 0.7992 0.7989 0.8015 0.7932 0.7941 0.7923 Butterfly 0.9508 0.9562 0.9531 0.9621 0.9599 0.9634 0.9638 0.9626 0.9641 Cameraman 0.8649 0.8647 0.8692 0.8732 0.8730 0.8779 0.8717 0.8711 0.8724 Fence 0.7604 0.7411 0.7573 0.7518 0.7654 0.7734 0.7647 0.7647 0.7649 House 0.8747 0.8722 0.8755 0.8757 0.8793 0.8819 0.8781 0.8773 0.8781 Lena 0.9114 0.9129 0.9118 0.9239 0.9191 0.9243 0.9208 0.9203 0.9217 Parthenon 0.7894 0.7719 0.7886 0.7863 0.7847 0.7919 0.7878 0.7877 0.7873 Station 0.8928 0.9023 0.9028 0.9160 0.9187 0.9216 0.9219 0.9208 0.9235 Wheel 0.7723 0.8227 0.7686 0.8415 0.8406 0.8308 0.8619 0.8584 0.8639 平均值 0.8565 0.8567 0.8568 0.8690 0.8695 0.8731 0.8710 0.8700 0.8714
下载: 导出CSV
表 3 不同插值算法基于EPI的比较(dB)
Table 3 A comparison of different interpolation methods with respect to the EPI (dB)
测试图像 Bicubic NEDI LMMSE SAI SME NARM CGI CED 本文算法 (1981) (2001) (2006) (2008) (2010) (2013) (2013) (2016) Airplane 0.7776 0.7793 0.7452 0.7629 0.7946 0.7730 0.8055 0.8050 0.8080 Bike 0.7725 0.8059 0.7469 0.7808 0.7917 0.8027 0.8267 0.8257 0.8323 Boats 0.7473 0.7359 0.7052 0.7279 0.7671 0.7209 0.7607 0.7586 0.7640 Bridge 0.7009 0.6802 0.6648 0.6855 0.7039 0.7055 0.7149 0.7090 0.7218 Butterfly 0.8406 0.8713 0.8175 0.8516 0.8691 0.8657 0.8856 0.8863 0.8874 Cameraman 0.7342 0.7212 0.6902 0.7099 0.7562 0.7234 0.7528 0.7554 0.7542 Fence 0.7015 0.7901 0.6738 0.7213 0.7309 0.7227 0.7118 0.7110 0.7141 House 0.7508 0.7429 0.7213 0.7400 0.7723 0.7231 0.7580 0.7581 0.7594 Lena 0.7928 0.8078 0.7711 0.7928 0.8075 0.7946 0.8236 0.8234 0.8273 Parthenon 0.7018 0.6996 0.6632 0.6803 0.7175 0.6884 0.7159 0.7142 0.7191 Station 0.8475 0.8781 0.8125 0.8433 0.8684 0.8656 0.8979 0.8968 0.9003 Wheel 0.7310 0.8214 0.6785 0.8036 0.7912 0.7455 0.8159 0.8186 0.8178 平均值 0.7582 0.7778 0.7242 0.7583 0.7809 0.7609 0.7891 0.7885 0.7921
下载: 导出CSV
表 4 不同算法的插值时间比较(s)
Table 4 A comparison of different interpolation methods with respect to the CPU time (s)
测试图像 Bicubic NEDI LMMSE SME NARM CGI CED 本文算法 (1981) (2001) (2006) (2010) (2013) (2013) (2016) Airplane 0.052 12.547 11.501 170.446 387.140 1.662 1.178 1.715 Bike 0.048 28.306 23.947 399.348 1014.306 3.530 2.572 3.766 Boats 0.030 17.657 14.464 235.319 580.087 2.126 1.554 2.279 Bridge 0.030 17.886 14.339 233.430 708.391 2.285 1.633 3.113 Butterfly 0.016 7.175 7.024 109.869 233.126 1.006 0.754 1.133 Cameraman 0.011 3.906 3.506 52.474 127.869 0.518 0.399 0.527 Fence 0.008 4.658 3.484 53.359 154.776 0.540 0.395 0.572 House 0.008 3.780 3.480 52.230 141.061 0.539 0.379 0.549 Lena 0.007 3.990 3.483 52.295 157.384 0.545 0.427 0.598 Parthenon 0.018 8.543 7.192 116.360 332.498 1.136 0.812 1.218 Station 0.009 4.092 3.516 52.557 145.134 0.551 0.427 0.618 Wheel 0.008 2.764 2.309 35.045 98.709 0.375 0.304 0.430 平均值 0.020 9.609 8.187 130.228 340.040 1.234 0.903 1.377
下载: 导出CSV
表 5 CGI算法和PGI算法对彩色图像进行插值的PSNR (dB)和CPU时间(s)比较
Table 5 A comparison of different interpolation methods with respect to the PSNR (dB) and the CPU time (s)
彩色图像 Butterfly Airplane Starfish PSNR CPU time PSNR CPU time PSNR CPU time CGI 27.59 1.5 29.94 4.1 28.91 1.4 本文算法 27.75 0.6 29.95 1.4 29.01 0.6
下载: 导出CSV
-
[1] El-Khamy S E, Hadhoud M M, Dessouky M I, Salam B M, Abd El-Samie F E. Efficient implementation of image interpolation as an inverse problem. Digital Signal Processing, 2005, 15(2):137-152 doi: 10.1016/j.dsp.2004.10.003 [2] Thévenaz P, Blu T, Unser M. Interpolation revisited. IEEE Transactions on Medical Imaging, 2000, 19(7):739-758 doi: 10.1109/42.875199 [3] Keys R. Cubic convolution interpolation for digital image processing. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1981, 29(6):1153-1160 doi: 10.1109/TASSP.1981.1163711 [4] Hwang J W, Lee S L. Adaptive image interpolation based on local gradient features. IEEE Signal Processing Letters, 2004, 11(3):359-362 doi: 10.1109/LSP.2003.821718 [5] Shi J Z, Reichenbach S E. Image interpolation by two-dimensional parametric cubic convolution. IEEE Transactions on Image Processing, 2006, 15(7):1857-1870 doi: 10.1109/TIP.2006.873429 [6] 庞志勇, 谭洪舟, 陈弟虎.一种改进的低成本自适应双三次插值算法及VLSI实现.自动化学报, 2013, 39(4):407-417 http://www.aas.net.cn/CN/abstract/abstract17829.shtmlPang Zhi-Yong, Tan Hong-Zhou, Chen Di-Hu. An improved low-cost adaptive bicubic interpolation arithmetic and VLSI implementation. Acta Automatica Sinica, 2013, 39(4):407-417 http://www.aas.net.cn/CN/abstract/abstract17829.shtml [7] Li X, Orchard M T. New edge-directed interpolation. IEEE Transactions on Image Processing, 2001, 10(10):1521-1527 doi: 10.1109/83.951537 [8] Zhang X J, Wu X L. Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation. IEEE Transactions on Image Processing, 2008, 17(6):887-896 doi: 10.1109/TIP.2008.924279 [9] Chang S G, Cvetkovic Z, Vetterli M. Locally adaptive wavelet-based image interpolation. IEEE Transactions on Image Processing, 2006, 15(6):1471-1485 doi: 10.1109/TIP.2006.871162 [10] 黄海赟, 戚飞虎, 陈剑, 姚志洪.基于小波的医学图像插值.自动化学报, 2002, 28(5):722-728 http://www.aas.net.cn/CN/abstract/abstract15619.shtmlHuang Hai-Yun, Qi Fei-Hu, Chen Jian, Yao Zhi-Hong. A wavelet-based interpolation of medical images. Acta Automatica Sinica, 2002, 28(5):722-728 http://www.aas.net.cn/CN/abstract/abstract15619.shtml [11] 杨云峰, 苏志勋, 胡金燕.一种保持边缘特征的图像插值方法.中国图象图形学报, 2005, 10(10):1248-1251 http://mall.cnki.net/magazine/Article/ZGTB200510006.htmYang Yun-Feng, Su Zhi-Xun, Hu Jin-Yan. A new edge-holding algorithm of image interpolation. Journal of Image and Graphics, 2005, 10(10):1248-1251 http://mall.cnki.net/magazine/Article/ZGTB200510006.htm [12] Jensen K, Anastassiou D. Subpixel edge localization and the interpolation of still images. IEEE Transactions on Image Processing, 1995, 4(3):285-295 doi: 10.1109/83.366477 [13] Sun H B, Zhang F W, Zheng N N. An edge-based adaptive image interpolation and its VLSI architecture. In: Proceedings of the 2012 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). Hollywood, CA, USA: IEEE, 2012. 1-6 [14] Wei Z, Ma K K. Contrast-guided image interpolation. IEEE Transactions on Image Processing, 2013, 22(11):4271-4285 doi: 10.1109/TIP.2013.2271849 [15] Zhang L, Wu X L. An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Transactions on Image Processing, 2006, 15(8):2226-2238 doi: 10.1109/TIP.2006.877407 [16] Yamaguchi T, Ikehara M. The quick and high quality image interpolation for single image using multi-filtering and weighted mean. In: Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP). Phoenix, AZ, USA: IEEE, 2016. 2841-2845 [17] Dong C Q, Portilla J. Maximum likelihood interpolation for aliasing-aware image restoration. In: Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP). Phoenix, AZ, USA: IEEE, 2016. 564-568 [18] Zhu S Y, Zeng B, Zeng L Y, Gabbouj M. Image interpolation based on non-local geometric similarities and directional gradients. IEEE Transactions on Multimedia, 2016, 18(9):1707-1719 doi: 10.1109/TMM.2016.2593039 [19] Dong W S, Zhang L, Lukac R, Shi G M. Sparse representation based image interpolation with nonlocal autoregressive modeling. IEEE Transactions on Image Processing, 2013, 22(4):1382-1394 doi: 10.1109/TIP.2012.2231086 [20] Huang J J, Siu W C, Liu T R. Fast image interpolation via random forests. IEEE Transactions on Image Processing, 2015, 24(10):3232-3245 doi: 10.1109/TIP.2015.2440751 [21] Mallat S, Yu G S. Super-resolution with sparse mixing estimators. IEEE Transactions on Image Processing, 2010, 19(11):2889-2900 doi: 10.1109/TIP.2010.2049927 [22] Romano Y, Protter M, Elad M. Single image interpolation via adaptive nonlocal sparsity-based modeling. IEEE Transactions on Image Processing, 2014, 23(7):3085-3098 doi: 10.1109/TIP.2014.2325774 [23] Hou H, Andrews H. Cubic splines for image interpolation and digital filtering. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1978, 26(6):508-517 doi: 10.1109/TASSP.1978.1163154 [24] Ye W, Ma K K. Convolutional edge diffusion for fast contrast-guided image interpolation. IEEE Signal Processing Letters, 2016, 23(9):1260-1264 doi: 10.1109/LSP.2016.2571738 -
计量
- 文章访问数: 2944
- HTML全文浏览量: 497
- PDF下载量: 765
- 被引次数: 0
