-
摘要: 图像质量评价（Image quality assessment, IQA）的目标是利用设计的计算模型得到与主观评价一致的结果，而人类视觉感知特性是感知图像质量评价的关键.大量研究发现，认知流形和拓扑连续性是人类感知的基础即人类感知局限在低维流形之上.基于图像低维流形特征分析，本文提出了基于流形特征相似度（Manifold feature similarity, MFS）的全参考图像质量评价方法.首先，利用正交局部保持投影算法来模拟大脑的视觉处理过程获取最佳映射矩阵进而得到图像的低维流形特征，通过流形特征的相似度来表征两幅图像的结构差异，从而反映感知质量上的差异.其次，考虑亮度失真对人眼视觉感知的影响，通过图像块均值计算亮度相似度并用于评价图像的亮度失真；最后，结合两个相似度得到图像的客观质量评价值.在四个公开图像测试库上的实验结果表明，所提出方法与现有代表性的图像质量方法相比总体上具有更好的评价结果.Abstract: Image quality assessment (IQA) aims to use computational models to measure the image quality in consistency with subjective evaluation, and human visual perception characteristics play an important role in the design of IQA metrics. From many researches on human visual perception, it has been found that the cognitive manifolds and the topological continuity can be used to describe the human visual perception, that is, human perception lies on the low-dimensional manifold. With this inspiration and manifold analysis of image, a new IQA metric called manifold feature similarity (MFS) is proposed for full-reference image quality assessment. First, orthogonal locality preserving projection algorithm is used to simulate the brain's visual processing process to obtain the best projection matrix so that low-dimensional manifold features of images are obtained. And the similarity of the manifold features is used to measure the structure differences between the two images so as to reflect differences in perceived quality and get a manifold features-based image quality index. Then, to consider the impact of brightness on human visual perception, the block mean values of the image are used to calculate the distortion of the image's brightness and design a brightness-based image quality index. The final quality score is obtained by incorporating these two indices. Extensive experiments on four large scale benchmark databases demonstrate that the proposed IQA metric works better than all state-of-the-art IQA metrics in terms of prediction accuracy.
-
图 1 基于流形特征相似度的图像质量评价准则
Fig. 1 Manifold feature similarity based perceptual image quality index
图 2 用于OLPP训练的图像集S1, 其中的图像均来自IVC的无失真图像
Fig. 2 The set S1 for OLPP, the images in the set were picked from IVC dataset
图 3 用于OLPP训练的图像集S2, 其中的图像均来自TOY的无失真图像
Fig. 3 The set S2 for OLPP, the images in the set were picked from TOY dataset
表 1 应用于图像质量评价算法分析的4个测试图像库
Table 1 The four benchmark datasets for evaluating IQA indices
图像库 参考图像数 失真图像数 失真类型数 主观测试人数 TID2013 25 3000 25 971 TID2008 25 1700 17 838 CSIQ 30 866 6 35 LIVE 29 799 5 161 
下载: 导出CSV
表 2 3种选块策略对应的SROCC值
Table 2 The SROCC of three selection strategies
图像库 不选块 AVE选块 AVE+VS选块 TID2013 0.8655 0.8728 0.8741 TID2008 0.8763 0.8870 0.8893 CSIQ 0.9508 0.9621 0.9615 LIVE 0.9500 0.9600 0.9578
下载: 导出CSV
表 3 不同的PCA白化降维维数下, MFS在4个图像库上SROCC值
Table 3 The SROCC of MFS at different whitening dimensions on four datasets
PCA白化后的空间维数 LIVE CSIQ TID2008 TID2013 8 0.9578 0.9615 0.8893 0.8741 16 0.9509 0.9594 0.8820 0.8585 24 0.9507 0.9587 0.8754 0.8483 32 0.9332 0.9235 0.8205 0.8059 64 0.9283 0.9067 0.7311 0.7444 不降维 0.8163 0.6962 0.2863 0.3864
下载: 导出CSV
表 4 在两个训练集图像库上的SROCC值比较
Table 4 The SROCC of MFS on two training sets
训练集 LIVE CSIQ TID 2008 TID 2013 Average S1 0.9594 0.9615 0.8866 0.8579 0.9168 S2 0.9578 0.9615 0.8893 0.8741 0.9206
下载: 导出CSV
表 5 ω取不同值时, MFS在4个图像库上的SROCC值
Table 5 The SROCC of MFS when ω takes different values
ω 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 LIVE 0.9553 0.9554 0.9557 0.9560 0.9562 0.9565 0.9568 0.9573 0.9578 0.9575 0.9248 CSIQ 0.9516 0.9523 0.9532 0.9542 0.9553 0.9565 0.9579 0.9594 0.9615 0.9635 0.7700 TID 2008 0.8377 0.8439 0.8509 0.8584 0.8661 0.8739 0.8817 0.8879 0.8893 0.8769 0.6598 TID 2013 0.8407 0.8455 0.8505 0.8554 0.8602 0.8650 0.8696 0.8733 0.8741 0.8675 0.7081
下载: 导出CSV
表 6 仅考虑流形特征时MFS的评价性能(ω=0)
Table 6 The performance when just considering the manifold feature (ω=0)
SSIM MS-SSIM IFC VIF VSNR MAD GSM RFSM FSIMc VSI MFS TID 2013 SROCC 0.7471 0.7859 0.5389 0.6769 0.6812 0.7808 0.7946 0.7744 0.8510 0.8965 0.8407 TID 2008 SROCC 0.7749 0.8542 0.5675 0.7491 0.7046 0.8340 0.8504 0.8680 0.8840 0.8979 0.8377 CSIQ SROCC 0.8756 0.9133 0.7671 0.9195 0.8106 0.9466 0.9108 0.9295 0.9310 0.9423 0.9516 LIVE SROCC 0.9479 0.9513 0.9259 0.9636 0.9274 0.9669 0.9561 0.9401 0.9645 0.9524 0.9553 Ave SROCC 0.8363 0.8761 0.6998 0.8272 0.7809 0.8820 0.8779 0.8780 0.9076 0.9222 0.8963
下载: 导出CSV
表 7 11种方法在4个图像库上的整体性能比较(ω=0.8)
Table 7 The total performance comparison of 11 IQA indices (ω=0.8)
SSIM MS-SSIM IFC VIF VSNR MAD GSM RFSM FSIMc VSI MFS TID2013 SROCC 0.7471 0.7859 0.5389 0.6769 0.6812 0.7808 0.7946 0.7744 0.8510 0.8965 0.8741 KROCC 0.5588 0.6407 0.3939 0.5147 0.5084 0.6035 0.6255 0.5951 0.6665 0.7183 0.6862 PLCC 0.7895 0.8329 0.5538 0.7720 0.7402 0.8267 0.8464 0.8333 0.8769 0.9000 0.8856 RMSE 0.7608 0.6861 1.0322 0.7880 0.8392 0.6975 0.6603 0.6852 0.5959 0.5404 0.5757 TID2008 SROCC 0.7749 0.8542 0.5675 0.7491 0.7046 0.8340 0.8504 0.8680 0.8840 0.8979 0.8893 KROCC 0.5768 0.6568 0.4236 0.5860 0.5340 0.6445 0.6596 0.6780 0.6991 0.7123 0.7055 PLCC 0.7732 0.8451 0.7340 0.8084 0.6820 0.8308 0.8422 0.8645 0.8762 0.8762 0.8865 RMSE 0.8511 0.7173 0.9113 0.7899 0.9815 0.7468 0.7235 0.6746 0.6468 0.6466 0.6211 CSIQ SROCC 0.8756 0.9133 0.7671 0.9195 0.8106 0.9466 0.9108 0.9295 0.9310 0.9423 0.9615 KROCC 0.6907 0.7393 0.5897 0.7537 0.6247 0.7970 0.7374 0.7645 0.7690 0.7857 0.8260 PLCC 0.8613 0.8991 0.8384 0.9277 0.8002 0.9502 0.8964 0.9179 0.9192 0.9279 0.9614 RMSE 0.1344 0.1149 0.1431 0.0980 0.1575 0.0818 0.1164 0.1042 0.1034 0.0979 0.0722 LIVE SROCC 0.9479 0.9513 0.9259 0.9636 0.9274 0.9669 0.9561 0.9401 0.9645 0.9524 0.9578 KROCC 0.7963 0.8045 0.7579 0.8282 0.7616 0.8421 0.8150 0.7816 0.8363 0.8058 0.8199 PLCC 0.9449 0.9489 0.9268 0.9604 0.9231 0.9675 0.9512 0.9354 0.9613 0.9482 0.9543 RMSE 8.9455 8.6188 10.264 7.6137 10.506 6.9073 8.4327 9.6642 7.5296 8.6816 8.1691
下载: 导出CSV
表 8 11种方法在特定失真上的SROCC评价值
Table 8 SROCC values of 11 IQA indices for each type of distortions
Type SSIM MS-SSIM IFC VIF VSNR MAD GSM RFSM FSIMc VSI MFS TID2013 AGN 0.8671 0.8646 0.6612 0.8994 0.8271 0.8843 0.9064 0.8878 0.9101 0.9460 0.9153 ANC 0.7726 0.7730 0.5352 0.8299 0.7305 0.8019 0.8175 0.8476 0.8537 0.8705 0.8273 SCN 0.8515 0.8544 0.6601 0.8835 0.8013 0.8911 0.9158 0.8825 0.8900 0.9367 0.9001 MN 0.7767 0.8073 0.6932 0.8450 0.7072 0.7380 0.7293 0.8368 0.8094 0.7697 0.8186 HFN 0.8634 0.8604 0.7406 0.8972 0.8455 0.8876 0.8869 0.9145 0.9040 0.9200 0.9063 IN 0.7503 0.7629 0.6408 0.8537 0.7363 0.2769 0.7965 0.9062 0.8251 0.8741 0.8313 QN 0.8657 0.8706 0.6282 0.7854 0.8357 0.8514 0.8841 0.8968 0.8807 0.8748 0.8421 GB 0.9668 0.9673 0.8907 0.9650 0.9470 0.9319 0.9689 0.9698 0.9551 0.9612 0.9553 DEN 0.9254 0.9268 0.7779 0.8911 0.9081 0.9252 0.9432 0.9359 0.9330 0.9484 0.9178 JPEG 0.9200 0.9265 0.8357 0.9192 0.9008 0.9217 0.9284 0.9398 0.9339 0.9541 0.9377 JP2K 0.9468 0.9504 0.9078 0.9516 0.9273 0.9511 0.9602 0.9518 0.9589 0.9706 0.9633 JGTE 0.8493 0.8475 0.7425 0.8409 0.7908 0.8283 0.8512 0.8312 0.8610 0.9216 0.8885 J2TE 0.8828 0.8889 0.7769 0.8761 0.8407 0.8788 0.9182 0.9061 0.8919 0.9228 0.9081 NEPN 0.7821 0.7968 0.5737 0.7720 0.6653 0.8315 0.8130 0.7705 0.7937 0.8060 0.7727 Block 0.5720 0.4801 0.2414 0.5306 0.1771 0.2812 0.6418 0.0339 0.5532 0.1713 0.1755 MS 0.7752 0.7906 0.5522 0.6276 0.4871 0.6450 0.7875 0.5547 0.7487 0.7700 0.6285 CTC 0.3775 0.4634 0.1798 0.8386 0.3320 0.1972 0.4857 0.3989 0.4679 0.4754 0.4598 CCS 0.4141 0.4099 0.4029 0.3099 0.3677 0.0575 0.3578 0.0204 0.8359 0.8100 0.8102 MGN 0.7803 0.7786 0.6143 0.8468 0.7644 0.8409 0.8348 0.8464 0.8569 0.9117 0.8630 CN 0.8566 0.8528 0.8160 0.8946 0.8683 0.9064 0.9124 0.8917 0.9135 0.9243 0.9052 LCNI 0.9057 0.9068 0.8180 0.9204 0.8821 0.9443 0.9563 0.9010 0.9485 0.9564 0.9290 ICQD 0.8542 0.8555 0.6006 0.8414 0.8667 0.8745 0.8973 0.8959 0.8815 0.8839 0.9072 CHA 0.8775 0.8784 0.8210 0.8848 0.8645 0.8310 0.8823 0.8990 0.8925 0.8906 0.8798 SSR 0.9461 0.9483 0.8885 0.9353 0.9339 0.9567 0.9668 0.9326 0.9576 0.9628 0.9478 TID2008 AGN 0.8107 0.8086 0.5806 0.8797 0.7728 0.8386 0.8606 0.8415 0.8758 0.9229 0.8887 ANC 0.8029 0.8054 0.5460 0.8757 0.7793 0.8255 0.8091 0.8613 0.8931 0.9118 0.8789 SCN 0.8144 0.8209 0.5958 0.8698 0.7665 0.8678 0.8941 0.8468 0.8711 0.9296 0.8951 MN 0.7795 0.8107 0.6732 0.8683 0.7295 0.7336 0.7452 0.8534 0.8264 0.7734 0.8375 HFN 0.8729 0.8694 0.7318 0.9075 0.8811 0.8864 0.8945 0.9182 0.9156 0.9253 0.9225 IN 0.6732 0.6907 0.5345 0.8327 0.6471 0.0650 0.7235 0.8806 0.7719 0.8298 0.7919 QN 0.8531 0.8589 0.5857 0.7970 0.8270 0.8160 0.8800 0.8880 0.8726 0.8731 0.8500 GB 0.9544 0.9563 0.8559 0.9540 0.9330 0.9196 0.9600 0.9409 0.9472 0.9529 0.9501 DEN 0.9530 0.9582 0.7973 0.9161 0.9286 0.9433 0.9725 0.9400 0.9618 0.9693 0.9488 JPEG 0.9252 0.9322 0.8180 0.9168 0.9174 0.9275 0.9393 0.9385 0.9294 0.9616 0.9416 JP2K 0.9625 0.9700 0.9437 0.9709 0.9515 0.9707 0.9758 0.9488 0.9780 0.9848 0.9825 JGTE 0.8678 0.8681 0.7909 0.8585 0.8055 0.8661 0.8790 0.8503 0.8756 0.9160 0.8766 J2TE 0.8577 0.8606 0.7301 0.8501 0.7909 0.8394 0.8936 0.8592 0.8555 0.8942 0.8947 NEPN 0.7107 0.7377 0.8418 0.7619 0.5716 0.8287 0.7386 0.7274 0.7514 0.7699 0.7094 Block 0.8462 0.7546 0.6770 0.8324 0.1926 0.7970 0.8862 0.6258 0.8464 0.6295 0.4698 MS 0.7231 0.7336 0.4250 0.5096 0.3715 0.5163 0.7190 0.4178 0.6554 0.6714 0.4810 CTC 0.5246 0.6381 0.1713 0.8188 0.4239 0.2723 0.6691 0.5823 0.6510 0.6557 0.6348 CSIQ AGWN 0.8974 0.9471 0.8431 0.9575 0.9241 0.9541 0.9440 0.9441 0.9359 0.9636 0.9647 JPEG 0.9546 0.9634 0.9412 0.9705 0.9036 0.9615 0.9632 0.9502 0.9664 0.9618 0.9548 JP2K 0.9606 0.9683 0.9252 0.9672 0.9480 0.9752 0.9648 0.9643 0.9704 0.9694 0.9750 AGPN 0.8922 0.9331 0.8261 0.9511 0.9084 0.9570 0.9387 0.9357 0.9370 0.9638 0.9607 GB 0.9609 0.9711 0.9527 0.9745 0.9446 0.9602 0.9589 0.9634 0.9729 0.9679 0.9758 GCD 0.7922 0.9526 0.4873 0.9345 0.8700 0.9207 0.9354 0.9527 0.9438 0.9504 0.9485 LIVE JP2K 0.9614 0.9627 0.9113 0.9696 0.9551 0.9676 0.9700 0.9323 0.9724 0.9604 0.9645 JPEG 0.9764 0.9815 0.9468 0.9846 0.9657 0.9764 0.9778 0.9584 0.9840 0.9761 0.9759 AGWN 0.9694 0.9733 0.9382 0.9858 0.9785 0.9844 0.9774 0.9799 0.9716 0.9835 0.9868 GB 0.9517 0.9542 0.9584 0.9728 0.9413 0.9465 0.9518 0.9066 0.9708 0.9527 0.9622 FF 0.9556 0.9471 0.9629 0.9650 0.9027 0.9569 0.9402 0.9237 0.9519 0.9430 0.9418
下载: 导出CSV
表 9 11种质量评价方法的时间复杂度
Table 9 Time cost of 11 IQA indices
IQA index Time cost(ms) SSIM 17.3 MS-SSIM 71.2 IFC 538.0 VIF 546.4 VSNR 23.9 MAD 702.3 GSM 17.7 RFSM 49.8 FSIMc 142.5 VSI 105.2 MFS 140.7
下载: 导出CSV
-
[1] Ma L, Deng C W, Ngan K N, Lin W S. Recent advances and challenges of visual signal quality assessment. China Communications, 2013, 10(5):62-278 doi: 10.1109/CC.2013.6520939 [2] Saha A, Wu Q M J. Utilizing image scales towards totally training free blind image quality assessment. IEEE Transactions on Image Processing, 2015, 24(6):1879-1892 doi: 10.1109/TIP.2015.2411436 [3] 王志明.无参考图像质量评价综述.自动化学报, 2015, 41(6):1062-1079 http://www.aas.net.cn/CN/abstract/abstract18682.shtmlWang Zhi-Ming. Review of no-reference image quality assessment. Acta Automatica Sinica, 2015, 41(6):1062-1079 http://www.aas.net.cn/CN/abstract/abstract18682.shtml [4] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment:from error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4):600-612 doi: 10.1109/TIP.2003.819861 [5] Wang Z, Simoncelli E P, Bovik A C. Multiscale structural similarity for image quality assessment. In:Proceedings of the 37th Conference Record of Asilomar Conference on Signals, Systems and Computers. Pacific Grove, CA, USA:IEEE, 2003. 1398-1402 [6] Zhang L, Zhang D, Mou X Q. RFSIM:a feature based image quality assessment metric using Riesz transforms. In:Proceedings of the 17th International Conference on Image Process. Hong Kong, China:IEEE, 2010. 321-324 [7] Zhang L, Zhang D, Mou X Q, Zhang D. FSIM:a feature similarity index for image quality assessment. IEEE Transactions on Image Processing, 2011, 20(8):2378-2386 doi: 10.1109/TIP.2011.2109730 [8] 程光权, 张继东, 成礼智, 黄金才, 刘忠.基于几何结构失真模型的图像质量评价研究.自动化学报, 2011, 37(7):811-819 http://www.aas.net.cn/CN/abstract/abstract17447.shtmlCheng Guang-Quan, Zhang Ji-Dong, Cheng Li-Zhi, Huang Jin-Cai, Liu Zhong. Image quality assessment based on geometric structural distortion model. Acta Automatica Sinica, 2011, 37(7):811-819 http://www.aas.net.cn/CN/abstract/abstract17447.shtml [9] Chandler D M, Hemami S S. VSNR:a wavelet-based visual signal-to-noise ratio for natural images. IEEE Transactions on Image Processing, 2007, 16(9):2284-2298 doi: 10.1109/TIP.2007.901820 [10] Larson E C, Chandler D M. Most apparent distortion:full-reference image quality assessment and the role of strategy. Journal of Electronic Imaging, 2010, 19(1):011006 doi: 10.1117/1.3267105 [11] Sheikh H R, Bovik A C. Image information and visual quality. IEEE Transactions on Image Processing, 2006, 15(2):430-444 doi: 10.1109/TIP.2005.859378 [12] Sheikh H R, Bovik A C, De Veciana G. An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Transactions on Image Processing, 2005, 14(12):2117-2128 doi: 10.1109/TIP.2005.859389 [13] Zhang L, Shen Y, Li H Y. VSI:a visual saliency-induced index for perceptual image quality assessment. IEEE Transactions on Image Processing, 2014, 23(10):4270-4281 doi: 10.1109/TIP.2014.2346028 [14] Liu A M, Lin W S, Narwaria M. Image quality assessment based on gradient similarity. IEEE Transactions on Image Processing, 2012, 21(4):1500-1512 doi: 10.1109/TIP.2011.2175935 [15] Seung H S, Lee D D. The manifold ways of perception. Science, 2000, 290(5500):2268-2269 doi: 10.1126/science.290.5500.2268 [16] de Silva V, Tenenbaum J B. Global versus local methods in nonlinear dimensionality reduction. In:Proceedings of the 2002 Advances in Neural Information Processing Systems. Cambridge, MA:MIT Press, 2002. 705-712 [17] Cai D, He X F, Han J W, Zhang H J. Orthogonal Laplacianfaces for face recognition. IEEE Transactions on Image Processing, 2006, 15(11):3608-3614 doi: 10.1109/TIP.2006.881945 [18] Charrier C, Lebrun G, Lezoray O. Image quality assessment with manifold and machine learning. In:Proceedings of the 2009 SPIE 7242, Image Quality and System Performance VI. San Jose, CA:SPIE, 2009. http://cn.bing.com/academic/profile?id=2042873693&encoded=0&v=paper_preview&mkt=zh-cn [19] Taube J S. Head direction cells and the neurophysiological basis for a sense of direction. Progress in Neurobiology, 1998, 55(3):225-256 doi: 10.1016/S0301-0082(98)00004-5 [20] Simoncelli E P, Olshausen B A. Natural image statistics and neural representation. Annual Review of Neuroscience, 2001, 24:1193-1216 doi: 10.1146/annurev.neuro.24.1.1193 [21] Engelke U, Kaprykowsky H, Zepernick H, Ndjiki-Nya P. Visual attention in quality assessment. IEEE Signal Processing Magazine, 2011, 28(6):50-59 doi: 10.1109/MSP.2011.942473 [22] Moorthy A K, Bovik A C. Visual importance pooling for image quality assessment. IEEE Journal of Selected Topics in Signal Processing, 2009, 3(2):193-201 doi: 10.1109/JSTSP.2009.2015374 [23] Zhang L, Gu Z Y, Li H Y. SDSP:a novel saliency detection method by combining simple priors. In:Proceedings of the 20th IEEE International Conference on Image Processing. Melbourne, VIC:IEEE, 2013. 171-175 [24] Sheikh H R, Sabir M F, Bovik A C. A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Transactions on Image Processing, 2006, 15(11):3440-3451 doi: 10.1109/TIP.2006.881959 [25] Ponomarenko N N, Lukin V V, Zelensky A, Egiazarian K, Carli M, Battisti F. TID2008——a database for evaluation of full-reference visual quality assessment metrics. Advances of Modern Radioelectronics, 2009, 10:30-45 http://cn.bing.com/academic/profile?id=2324971623&encoded=0&v=paper_preview&mkt=zh-cn [26] Ponomarenko N, Jin L, Ieremeiev O, Lukin V, Egiazarian K, Astola J, Vozel B, Chehdi K, Carli M, Battisti F, Kuo C C J. Image database TID2013:peculiarities, results and perspectives. Signal Processing:Image Communication, 2015, 30:57-77 doi: 10.1016/j.image.2014.10.009 [27] VQEG. Final report from the video quality experts group on the validation of objective models of video quality assessment[Online], available:http://www.vqeg.org/, November 3, 2015 -
图(4) / 表(9)
计量
- 文章访问数: 3098
- HTML全文浏览量: 399
- PDF下载量: 1092
- 被引次数: 0
