Asymmetrically Distorted Stereoscopic Image Quality Assessment Based on Ocular Dominance
-
摘要: 针对现有立体图像质量评价算法对非对称失真立体图像的评价准确性及执行效率较低的问题，提出一种基于眼优势的非对称失真立体图像质量评价算法.首先采用梯度幅值响应来模拟左右眼输入的刺激强度，并根据人类视觉系统的眼优势原理分别以左和右视点图像作为主视图合成两幅融合图像；其次，利用旋转不变统一局部二值模式直方图、皮尔逊线性相关系数以及非对称广义高斯模型，获取左右融合图像以及左右梯度幅值响应图像中的多种能够反映立体图像质量好坏的特征；最后，利用自适应增强的支持向量回归模型将感知特征向量映射为图像质量值.在四个基准测试数据库上的实验结果表明：本文所提出算法大幅提升了非对称失真立体图像的评价准确性，且具有较高的执行效率.这些优势说明本文算法所提取的特征描述能力更强，质量映射模型的稳定性更好.
-
关键词:
- 立体图像质量评价 /
- 非对称失真 /
- 眼优势 /
- 双融合图像 /
- 自适应增强—支持向量回归
Abstract: Aiming at overcoming the low prediction accuracy and efficiency of current stereoscopic image quality assessment (SIQA) algorithms in evaluating the asymmetrically distorted stereoscopic images, this paper proposed an asymmetrically distorted SIQA algorithm based on ocular dominance theory. The proposed algorithm first uses the gradient magnitude response images (GMRIs) of the left and right views to model stimulus strength, and synthesizes two fusion images (FIs) by taking the left and the right views as dominant eye input, respectively, according to the theory of ocular dominance in human visual system. Then, the global histogram representations of the rotation invariant uniform local binary patterns, the Pearson linear correlation coefficient, and the asymmetric generalized Gaussian distribution model are employed to extract various perceptual features from the two GMRIs and the two FIs. Finally, with a pre-trained AdaBoosting support vector regression model, the extracted quality-aware feature vector of the stereoscopic image is mapped into the image quality score. Experimental results on four benchmark stereoscopic image databases show that, compared with the state-of-the-art SIQA algorithms, the prediction accuracy on asymmetrically distorted images of the proposed algorithm is greatly improved, and the implementation efficiency of the proposed algorithm is high. The results indicate that the extracted features of the proposed algorithm are more descriptive, and the quality mapping model is more stable.1) 本文责任编委 刘跃虎 -
图 1 基于眼优势的非对称失真立体图像质量评价算法框图
Fig. 1 The diagram of asymmetrically distorted SIQA algorithm based on ocular dominance
图 2 失真图像及其GMRI和Gabor滤波图像
Fig. 2 The distorted image, the GMRI, and the Gabor response image
图 4 各类失真图像及其左右融合图像(从左至右的图像依次为左视点、右视点、左融合和右融合图像)
Fig. 4 The distorted images and their left and right fusion images for various distortion types (Images from left to right are left view, right view, left fusion image, and right fusion image)
图 5 各类失真图像的左右融合图像的RIU-LBP直方图
Fig. 5 RIU-LBP histograms of the left and right fusion images for different distortion types
图 6 左和右融合图像的邻域像素之间PLCC值取值变化图
Fig. 6 The variation of the PLCC values between neighboring pixels in the left and right fusion images
图 7 归一化GMRIs的统计分布(以右视点为例)
Fig. 7 The distribution of normalized GMRIs (take the right view as an example)
图 8 GMRI在水平方向的邻域像素乘积图统计分布(以右视点为例)
Fig. 8 The distribution of the neighboring products of GMRI along horizontal direction (take the right view as an example)
图 10 各对比算法在IVC Phase Ⅰ数据库上预测值与MOS值的散点分布图
Fig. 10 The scatter plots of the MOS values versus the quality scores predicted by the competing algorithms on IVC Phase Ⅰ database
图 11 采用不同个数SVR在独立数据库和交叉数据库验证的结果对比
Fig. 11 The comparison of experiments on individual and cross database tests when using different numbers of SVR
表 1 失真立体图像的两幅融合图像与左右视点图像之间的相似性和相关性
Table 1 The similarity and correlation between the two fusion images and the stereo pair
图像 相似性 相关性 对称失真 非对称失真 混合 对称失真 非对称失真 混合 I-L I-R I-L I-R I-L I-R I-L I-R I-L I-R I-L I-R FI-L 0.6404 0.4091 0.6726 0.3946 0.6555 0.3969 0.7561 0.5815 0.8134 0.5493 0.7950 0.5531 FI-R 0.4047 0.6264 0.3971 0.6135 0.4005 0.6214 0.5705 0.7481 0.5766 0.7205 0.5766 0.7448 
下载: 导出CSV
表 2 各算法在基准测试数据库中的非对称失真图像部分预测结果比较
Table 2 The comparison between SIQA algorithms on the asymmetrically distorted images in benchmark databases
算法 IVC Phase Ⅰ (252) IVC Phase Ⅱ (330) LIVE Phase Ⅱ (280) SROCC PLCC RMSE SROCC PLCC RMSE SROCC PLCC RMSE SS[25] 0.5372 0.5427 11.1710 0.4029 0.4080 16.3354 0.7694 0.7691 6.9152 MS[31] 0.5109 0.5206 11.1025 0.4501 0.4583 15.8435 0.7207 0.6294 7.4424 BR[27] 0.9349 0.9453 4.8005 0.9206 0.9377 6.4756 0.9336 0.9410 3.8740 CF[14] 0.4147 0.4575 11.9912 0.3302 0.3407 16.2554 0.8883 0.7701 4.8922 ID[9] 0.9383 0.9493 4.5700 0.9310 0.9392 6.2879 0.9019 0.8981 − SI[18] 0.9336 0.9481 4.6602 0.9217 0.9343 6.6306 0.9402 0.9457 3.6905 文献[32] 0.8600 0.8740 − 0.8010 0.8440 − 0.8920 0.9570 − ODAD 0.9595 0.9692 3.5685 0.9643 0.9703 4.4826 0.9397 0.9482 3.6014
下载: 导出CSV
表 3 各算法在基准测试数据库中的对称失真图像部分预测结果比较
Table 3 The comparison between SIQA algorithms on the symmetrically distorted images in benchmark databases
算法 IVC Phase Ⅰ (78) IVC Phase Ⅱ (130) LIVE Phase Ⅱ (80) SROCC PLCC RMSE SROCC PLCC RMSE SROCC PLCC RMSE SS[25] 0.7517 0.7082 9.5359 0.5933 0.5775 13.2300 0.8447 0.8360 5.8319 MS[31] 0.6416 0.6298 10.6055 0.5169 0.5263 13.7357 0.9185 0.7951 4.1896 BR[27] 0.9393 0.9655 5.4399 0.9460 0.9611 5.7405 0.9147 0.9336 4.2804 CF[14] 0.7361 0.7944 9.9160 0.4864 0.5247 14.1269 0.9173 0.8134 4.0668 ID[9] 0.9669 0.9778 3.7110 0.9696 0.9802 4.0223 0.9234 0.9372 − SI[18] 0.9464 0.9696 5.0625 0.9463 0.9593 5.7976 0.9235 0.9433 3.9317 文献[32] 0.9100 0.9020 − 0.9140 0.9150 − 0.9280 0.9350 − ODAD 0.9464 0.9682 5.1764 0.9672 0.9700 4.4689 0.9071 0.9287 4.4932
下载: 导出CSV
表 4 各算法在对称与非对称失真数据库上的预测结果比较
Table 4 The comparison between SIQA algorithms on the symmetrically and asymmetrically distorted databases
算法 IVC Phase Ⅰ (330) IVC Phase Ⅱ (460) LIVE Phase Ⅱ (360) SROCC PLCC RMSE SROCC PLCC RMSE SROCC PLCC RMSE SS[25] 0.5963 0.6074 11.0735 0.4671 0.4749 15.9686 0.7925 0.7835 6.2681 MS[31] 0.5479 0.5647 11.2988 0.4749 0.4868 15.7450 0.7707 0.6594 6.9380 BR[27] 0.9352 0.9501 4.9508 0.9326 0.9453 6.2938 0.9375 0.9427 3.8404 CF[14] 0.5690 0.6740 11.6230 0.4440 0.5690 15.7400 0.9013 0.7781 4.7464 CN[15] 0.9110 0.9260 6.2320 0.8840 0.8820 8.9610 0.8800 0.8800 5.1020 Su[16] − − − − − − 0.9050 0.9130 4.6570 ID[9] 0.9502 0.9609 4.3588 0.9437 0.9533 5.7809 0.9159 0.9188 − SI[18] 0.9442 0.9570 4.6212 0.9369 0.9488 6.0713 0.9471 0.9519 3.5482 文献[32] 0.9040 0.8980 − 0.8900 0.8660 − 0.9180 0.8950 3.2100 文献[33] − − − − − − 0.9233 0.9297 − ODAD 0.9645 0.9729 3.4945 0.9696 0.9745 4.1355 0.9403 0.9453 3.4609
下载: 导出CSV
表 5 各算法在LIVE Phase Ⅰ数据库上的SROCC、PLCC和RMSE比较
Table 5 The comparison between SIQA algorithms in terms of SROCC, PLCC, and RMSE on LIVE Phase Ⅰ database
下载: 导出CSV
表 6 交叉验证实验结果
Table 6 The experimental results on cross database tests
算法 IVC Phase Ⅰ LIVE Phase Ⅱ LIVE Phase Ⅰ LIVE Phase Ⅱ IVC Phase Ⅱ LIVE Phase Ⅰ IVC Phase Ⅰ IVC Phase Ⅱ SROCC PLCC SROCC PLCC SROCC PLCC SROCC PLCC SROCC PLCC SROCC PLCC BR[27] 0.5878 0.6129 0.3168 0.2999 0.8413 0.8699 0.6568 0.6708 0.7585 0.7978 0.5344 0.5239 SI[18] 0.6248 0.6515 0.3311 0.3324 0.8165 0.8419 0.9054 0.9045 0.6694 0.6470 0.4854 0.4748 ODAD 0.5784 0.6360 0.3542 0.3591 0.8390 0.8458 0.9048 0.9050 0.6133 0.5785 0.6123 0.6057
下载: 导出CSV
表 7 各部分特征对ODAD算法性能影响
Table 7 The influence of each part of features on the performance of ODAD
算法 LIVE Phase Ⅰ LIVE Phase Ⅱ IVC Phase Ⅰ IVC Phase Ⅱ SROCC PLCC RMSE SROCC PLCC RMSE SROCC RMSE PLCC RMSE SROCC RMSE ODAD-GM 0.9531 0.9587 4.6525 0.9400 0.9452 3.9110 0.9556 0.9618 4.3948 0.9589 0.9651 5.0795 ODAD-CL 0.9586 0.9629 4.5157 0.9401 0.9458 3.6589 0.9546 0.9647 4.1464 0.9582 0.9655 5.1122 ODAD-CR 0.9585 0.9645 4.3886 0.9400 0.9453 3.7114 0.9519 0.9641 4.2672 0.9560 0.9645 5.2156 ODAD-148 0.9570 0.9631 4.5024 0.9408 0.9468 3.7199 0.9536 0.9647 4.2296 0.9566 0.9646 5.0769 ODAD 0.9596 0.9665 4.0745 0.9403 0.9453 3.4609 0.9645 0.9729 3.4945 0.9696 0.9745 4.1355
下载: 导出CSV
-
[1] Su C C, Moorthy A K, Bovik A C. Visual quality assessment of stereoscopic image and video:challenges, advances, and future trends. Visual Signal Quality Assessment, Cham, Springer, 2015. 185-212 [2] 王志明.无参考图像质量评价综述.自动化学报, 2015, 41(6):1062-1079 http://www.aas.net.cn/CN/abstract/abstract18682.shtmlWang Zhi-Ming. Review of no-reference image quality assessment. Acta Automatica Sinica, 2015, 41(6):1062-1079 http://www.aas.net.cn/CN/abstract/abstract18682.shtml [3] 徐少平, 林官喜, 曾小霞, 姜尹楠, 唐祎玲.立体图像质量感知特征提取的研究进展.计算机工程, 2018, 44(6):239-248 doi: 10.3969/j.issn.1000-3428.2018.06.041Xu Shao-Ping, Lin Guan-Xi, Zeng Xiao-Xia, Jiang Yin-Nan, Tang Yi-Ling. Quality-aware feature extraction in the stereoscopic image quality assessment algorithm. Computer Engineering, 2018, 44(6):239-248 doi: 10.3969/j.issn.1000-3428.2018.06.041 [4] 陈勇, 吴明明, 房昊, 刘焕淋.基于差异激励的无参考图像质量评价[网络出版].自动化学报, http://kns.cnki.net, 2019年1月22日Chen Yong, Wu Ming-Ming, Fang Hao, Liu Huan-Lin. No-reference image quality assessment based on differential excitation[Online]. Acta Automatica Sinica, available:http://kns.cnki.net, January 22, 2019 [5] Campisi P, Callet P L, Marint E. Stereoscopic images quality assessment. In:Proceedings of the 15th European Signal Processing. Poznan, Poland:IEEE, 2007. 2110-2114 [6] 张明琦, 范影乐, 武薇.基于初级视通路视觉感知机制的轮廓检测方法[网络出版].自动化学报, http://kns.cnki.net, 2018年10月11日Zhang Ming-Qi, Fan Ying-Le, Wu Wei. A contour detection method based on visual perception mechanism in primary visual pathway[Online]. Acta Automatica Sinica, available:http://kns.cnki.net, October 11, 2018 [7] Hubel D H, Wiesel T N. Ferrier Lecture:functional architecture of macaque monkey visual cortex. Proceedings of the Royal Society B:Biological Sciences, 1977, 198(1130):1-59 doi: 10.1098/rspb.1977.0085 [8] Menon R S, Ogawa S, Strupp J P, Ugurbil K. Ocular dominance in human V1 demonstrated by functional magnetic resonance imaging. Journal of Neurophysiology, 1997, 77(5):2780-2787 doi: 10.1152/jn.1997.77.5.2780 [9] Wang J, Zeng K, Wang Z. Quality prediction of asymmetrically distorted stereoscopic images from single views. In:Proceeding of the 2014 IEEE International Conference on Multimedia and Expo. Chengdu, Sichuan, China:IEEE, 2014. 14-18 [10] Shao F, Zhang Z, Jiang Q, Lin W, Jiang G. Toward domain transfer for no-reference quality prediction of asymmetrically distorted stereoscopic images. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 28(3):573-585 doi: 10.1109/TCSVT.2016.2628082 [11] Zhou W, Zhang S, Pan T, Yu L, Qiu W, Zhou Y, et al. Blind 3D image quality assessment based on self-similarity of binocular features. Neurocomputing, 2017, 224(8):128-134 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=f025746f01fdff2db844e62be75977cf [12] Shao F, Lin W, Jiang G, Dai Q. Models of monocular and binocular visual perception in quality assessment of stereoscopic images. IEEE Transactions on Computational Imaging, 2016, 2(2):123-135 doi: 10.1109/TCI.2016.2538720 [13] Maalouf A, Larabi M C. CYCLOP:a stereo color image quality assessment metric. In:Proceedings of the 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Prague, Czech Republic:IEEE, 2011. 1161-1164 [14] Chen M J, Su C C, Kwon D K, Cormack L K, Bovik A C. Fullreference quality assessment of stereopairs accounting for rivalry. Signal Processing:Image Communication, 2013, 28(9):1143-1155 doi: 10.1016/j.image.2013.05.006 [15] Chen M J, Cormack L K, Bovik A C. No-reference quality assessment of natural stereopairs. IEEE Transactions on Image Processing, 2013, 22(9):3379-3391 doi: 10.1109/TIP.2013.2267393 [16] Su C C, Cormack L K, Bovik A C. Oriented correlation models of distorted natural images with application to natural stereopair quality evaluation. IEEE Transactions on Image Processing, 2015, 24(5):1685-1699 doi: 10.1109/TIP.2015.2409558 [17] Shen L, Fang R, Yao Y, Geng X, Wu D. No-reference stereoscopic image quality assessment based on image distortion and stereo perceptual information. IEEE Transactions on Emerging Topics in Computational Intelligence, 2019, 3(1):59-72 doi: 10.1109/TETCI.2018.2804885 [18] Liu L, Liu B, Su C C, Huang H, Bovik A C. Binocular spatial activity and reverse saliency driven no-reference stereopair quality assessment. Signal Processing Image Communication, 2017, 58(10):287-299 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=daf1d34098fc5b22df02bd3586ef4ee0 [19] 杨小康, 刘敏, 翟广涛, 顾锞.基于非线性眼优势视差补偿的立体图像质量评价方法. CN, Patent 104240255A, 2014年12月24日Yang Xiao-Kang, Liu Min, Zhai Guang-Tao, Gu Ke. Stereoscopic image quality evaluation method based on nonlinear ocular dominance parallax compensation. CN Patent 104240255A. December 24, 2014 [20] 杨小康, 刘敏, 翟广涛, 顾锞.基于眼优势理论和视差补偿的立体图像质量评价方法. CN Patent 104243977B. 2014年12月24日Yang Xiao-Kang, Liu Min, Zhai Guang-Tao, Gu Ke. Stereoscopic image quality evaluation method based on ocular dominance theory and parallax compensation. CN Patent 104243977B. December 24, 2014 [21] Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7):971-987 doi: 10.1109/TPAMI.2002.1017623 [22] Zhou W, Yu L, Qiu W, Zhou Y, Wu M. Local gradient patterns (LGP):an effective local-statistical-feature extraction scheme for no-reference image quality assessment. Information Sciences, 2017, 397-398:1-14 doi: 10.1016/j.ins.2017.02.049 [23] Oszust M. No-Reference image quality assessment using image statistics and robust feature descriptors. IEEE Signal Processing Letters, 2017, 24(11):1656-1660 doi: 10.1109/LSP.2017.2754539 [24] Hubel D H, Wiesel T N. Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. The Journal of Physiology, 1962, 160(1):106-154 doi: 10.1113/jphysiol.1962.sp006837 [25] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment:from error visibility to structural similarity. IEEE Transaction Image Processing, 2004, 13(4):600-612 doi: 10.1089-fpd.2009.0394/ [26] Wang J, Rehman A, Zeng K, Wang S, Wang Z. Quality prediction of asymmetrically distorted stereoscopic 3D images. IEEE Transactions on Image Processing, 2015, 24(11):3400-3414 doi: 10.1109/TIP.2015.2446942 [27] Mittal A, Moorthy A K, Bovik A C. No-reference image quality assessment in the spatial domain. IEEE Transactions on Image Processing, 2012, 21(12):4695-4708 doi: 10.1109/TIP.2012.2214050 [28] Moorthy A K, Su C C, Mittal A, Bovik A C. Subjective evaluation of stereoscopic image quality. Signal Processing:Image Communication, 2013, 28(8):870-883 doi: 10.1016/j.image.2012.08.004 [29] Singer W, Gray C M. Visual feature integration and the temporal correlation hypothesis. Annual Review of Neuroscience, 1995, 18(1):555-586 doi: 10.1146/annurev.ne.18.030195.003011 [30] Chang C C, Lin C J. LIBSVM:a library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3):Article No.27 http://d.old.wanfangdata.com.cn/Periodical/jdq201315008 [31] Wang Z, Simoncelli E R, Bovik A C. Multiscale structural similarity for image quality assessment. In:Proceedings of IEEE 37th asilomar conference on Signals, Systems and Computers, CA, USA:Pacific Grove, 2004. 1398-1402 https://ieeexplore.ieee.org/document/1292216 [32] Fezza S A, Chetouani A, Larabi M C. Using distortion and asymmetry determination for blind stereoscopic image quality assessment strategy. Journal of Visual Communication and Image Representation, 2017, 49:115-128 doi: 10.1016/j.jvcir.2017.08.009 [33] Ding Y, Deng R, Xie X, Xu X, Zhao Y, Chen X, et al. No-reference stereoscopic image quality assessment using convolutional neural network for adaptive feature extraction. IEEE Access, 2018, 6:37595-37603 doi: 10.1109/ACCESS.2018.2851255 -
计量
- 文章访问数: 2232
- HTML全文浏览量: 421
- PDF下载量: 138
- 被引次数: 0
