2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

一种基于双层框架的仿射类图像抠像方法

姚桂林 赵志杰 苏晓东 辛海涛 胡文 秦相林

姚桂林, 赵志杰, 苏晓东, 辛海涛, 胡文, 秦相林. 一种基于双层框架的仿射类图像抠像方法. 自动化学报, 2021, 47(1): 209-223 doi: 10.16383/j.aas.c180356
引用本文: 姚桂林, 赵志杰, 苏晓东, 辛海涛, 胡文, 秦相林. 一种基于双层框架的仿射类图像抠像方法. 自动化学报, 2021, 47(1): 209-223 doi: 10.16383/j.aas.c180356
Yao Gui-Lin, Zhao Zhi-Jie, Su Xiao-Dong, Xin Hai-Tao, Hu Wen, Qin Xiang-Lin. A hierarchical framework on affinity based image matting. Acta Automatica Sinica, 2021, 47(1): 209-223 doi: 10.16383/j.aas.c180356
Citation: Yao Gui-Lin, Zhao Zhi-Jie, Su Xiao-Dong, Xin Hai-Tao, Hu Wen, Qin Xiang-Lin. A hierarchical framework on affinity based image matting. Acta Automatica Sinica, 2021, 47(1): 209-223 doi: 10.16383/j.aas.c180356

一种基于双层框架的仿射类图像抠像方法

doi: 10.16383/j.aas.c180356
基金项目: 

黑龙江省自然科学基金 F2018021

黑龙江省自然科学基金 LH2019F044

哈尔滨商业大学校级科研项目 18XN021

哈尔滨商业大学校级科研项目 2016TD001

哈尔滨商业大学青年创新人才支持计划 2019CX02

黑龙江省哲学社会科学研究规划项目 18GLB029

详细信息
    作者简介:

    姚桂林   哈尔滨商业大学副教授.主要研究方向为图像处理, 人工智能.E-mail: glyao@hrbcu.edu.cn

    赵志杰   哈尔滨商业大学教授.主要研究方向为图像处理, 智能信息处理.E-mail: zhaozj@hrbcu.edu.cn

    辛海涛  哈尔滨商业大学教授.主要研究方向为智能信息处理, 大数据分析. E-mail: xht@hrbcu.edu.cn

    胡文  哈尔滨商业大学教授.主要研究方向为商业智能, 嵌入式技术, 社会计算. E-mail: huw@hrbcu.edu.cn

    秦相林  哈尔滨商业大学教授.主要研究方向为图像处理, 计算机网络. E-mail: qxl@hrbcu.edu.cn

    通讯作者:

    苏晓东  哈尔滨商业大学教授.主要研究方向为人工智能, 智能信息处理.本文通信作者. E-mail: suxd@hrbcu.edu.cn

  • 本文责任编委 刘跃虎

A Hierarchical Framework on Affinity Based Image Matting

Funds: 

Heilongjiang Provincial Natural Science Foundation F2018021

Heilongjiang Provincial Natural Science Foundation LH2019F044

Technological Project of Harbin University of Commerce 18XN021

Technological Project of Harbin University of Commerce 2016TD001

Young Creative Talents Support Project of Harbin University of Commerce 2019CX02

Heilongjiang Provincial Philosophy and Social Science Planning Project 18GLB029

More Information
    Author Bio:

    YAO Gui-Lin   Associate professor at Harbin University of Commerce. His research interest covers image processing and artificial intelligence

    ZHAO Zhi-Jie   Professor at Harbin University of Commerce. His research interest covers image processing and intelligent information processing

    XIN Hai-Tao  Professor at Harbin University of Commerce. His research interest covers intelligent information processing and big data analysis

    HU Wen  Professor at Harbin University of Commerce. His research interest covers business intelligence, embedded technology and social computing

    QIN Xiang-Lin  Professor at Harbin University of Commerce. His research interest covers image processing and computer network

    Corresponding author: SU Xiao-Dong  Professor at Harbin University of Commerce. His research interest covers artificial intelligence and intelligent information processing. Corresponding author of this paper
  • Recommended by Associate Editor LIU Yue-Hu
  • 摘要: 仿射类抠像方法主要分为KNN (K-nearest neighbor)类和Matting Laplacian类方法, 本文结合这2种方法的优点提出了一种基于仿射类的双层次抠像方法.其中, 第一层为绝对像素的划分层次或预处理层次, 采用了基于KNN类简单权重与相对远距离的搜索方法, 并结合初始Trimap未知区域大小无关的方式; 第二层为混合像素的计算层次或最终抠像层次, 充分利用了第一层计算获得的剩余混合像素的宽度, 自适应地调整Matting Laplacian中的颜色线性模型所构成颜色近邻的核宽度.每个层次均按图像的全局颜色重叠程度相应调整合理的搜索范围.本文的实验具备以下特点: 1)预处理层次之后采用了若干典型的后续抠像方法, 以展现本文方法相比于其他预处理方法对后续抠像操作步骤的优越性和兼容性; 2)最终抠像层次引入了若干其他抠像方法, 以验证本文抠像方法的优越性.实验表明, 相比于其他单层次的仿射类方法, 无论对于计算绝对像素还是混合像素, 本文方法都可以大幅提升计算结果的准确率.
    Recommended by Associate Editor LIU Yue-Hu
    1)  本文责任编委 刘跃虎
  • 图  1  图像抠像问题的基本输入输出和本文的双层次抠像结构

    Fig.  1  Input and output of image matting and the hierarchical framework of our method

    图  2  仿射类方法的2种分类方式对应的算法及适用范围(其中Lap表明在原简单权重方法的基础上采用Matting Laplacian)

    Fig.  2  Two types of affinity based matting method and the corresponding algorithms and application scopes (where Lap denotes the application of matting Laplacian based on simple weight methods)

    图  3  各$\alpha^{\rm{true}}$区间中5种传统仿射类算法$\alpha$结果的平均MSE比较(其中$x$轴坐标中的0.0表示$0.0\sim0.05$区间等)

    Fig.  3  MSE comparison on five traditional affinity based matting algorithms in each $\alpha^{\rm{true}}$ interval (where 0.0 in $x$-label denotes the range of $0.0\sim0.05$, etc)

    图  4  以背景为例, 未知点的各种搜索方式与搜索范围

    Fig.  4  Various kinds of searching manners and searching ranges for an unknown pixel

    图  5  全局低重合度图像中本文预处理方法与其他2种单一搜索方式的结果比较(前2列与后2列分别显示了远距离与近距离搜索方法较好的例子, 图像中的线条表示前景与背景边界)

    Fig.  5  Comparison of pre-processing results between our method and two unary searching methods in low global overlapped cases (where the first and last two rows show good results in methods with long and short searching ranges respectively, and the lines in the input images show known foreground and background boundaries)

    图  6  全局高重合度图像中本文预处理方法与其他2种单一搜索方式的结果比较

    Fig.  6  Comparison of pre-processing results between our method and two unary searching methods in high global overlapped cases

    图  7  预处理后的各种Trimap情况

    Fig.  7  Various kinds of Trimap after pre-processing

    图  8  9种预处理方法与无预处理方法在一些局部图像中的代价比较($\times10^{-4}$)

    Fig.  8  Comparison on cost of several local images in 9 pre-processing methods and no pre-processing method ($\times10^{-4}$)

    图  9  3种型号的Trimap下, 对所有训练图像抠像结果的平均MSE比较, 格式为:算法名称$^{\text{排名}}$, 排名为在8种后续抠像算法下, 每种预处理方法在10种预处理方法中的平均排名

    Fig.  9  Average MSE Comparisons on the matting results of all the training images over three types of Trimap, where the format is [algorithm name]$^{\rm{rank}}$ and the rank denotes the average rank for each of the pre-processing methods out of 10 over the 8 matting methods

    图  10  对于图 7的前6个局部图像及预处理后的Trimap, 采用10种方法进行抠像计算后的MSE比较, 其中前2个例子为硬边界, 中间2个例子为软边界, 后2个例子为长毛发边缘与前景空洞, 且这些例子中未知区域的宽度也逐渐增大

    Fig.  10  MSE comparison on matting results of the first 6 local images in Fig. 7 for 10 matting methods, where the first, median, and last 2 cases are hard boundaries, soft boundaries, and long hair edges and foreground holes respectively, in which the sizes of unknown regions gradually enlarge

    图  11  3种型号的Trimap下, 在各$\alpha$区间上, 10种混合像素计算方法对所有训练图像的抠像结果的平均MSE比较, 其中$x$坐标轴中的0.15表示$0.15\sim0.25$区间等

    Fig.  11  Average MSE comparisons on matting results of all the training images for 10 matting methods over 3 types of Trimap in each $\alpha$ range, where 0.15 in $x$-label denotes the range of $0.15\sim0.25$, etc.

    图  12  10种仿射类抠像方法的3个实例

    Fig.  12  3 Cases of matting results for 10 a–nity based matting methods

    表  1  9种预处理方法与未预处理方法在3种型号Trimap之下对所有训练图像的代价值之和的比较

    Table  1  Comparison on sum cost of all the training images within 9 pre-processing methods and no pre-processing method over three types of Trimap

    预处理方法 代价值($\times10^{-3}$)Trimap型号
    巨大
    无预处理 36.6 22.4 13.7
    Large Kernel 14.6 11.0 8.3
    Nonlocal 17.3 10.2 6.8
    Closed Form 16.4 10.4 6.6
    KNN-0.5 16.3 9.4 5.5
    KNN 14.6 9.1 5.8
    KNN-0.01 11.4 8.5 5.9
    CCM 10.3 8.4 6.3
    KNN-0.1 9.8 6.3 4.7
    本文方法 9.3 6.3 4.6
    下载: 导出CSV

    表  2  10种仿射类方法最终抠像结果的MSE比较

    Table  2  Final MSE comparison on matting results for 10 affinity based matting methods

    算法 MSE($\times10^{-2}$)&Trimap型号
    巨大
    Nonlocal 5 13.9 8.8 5.2
    Nonlocal 10 10.5 6.7 4.0
    CCM 7.6 6.1 3.8
    KNN-0.01 7.5 5.6 2.4
    Nonlocal 20 7.5 4.7 3.0
    Closed Form 6.8 4.5 2.6
    KNN-0.5 6.1 3.8 2.4
    KNN 5.8 4.0 2.3
    KNN-0.1 4.9 3.6 2.4
    本文方法 3.2 2.5 2.2
    下载: 导出CSV
  • [1] Levin A, Lischinski D, Weiss Y. A closed-form solution to natural image matting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2): 228-242 doi: 10.1109/TPAMI.2007.1177
    [2] He K M, Sun J, Tang X O. Fast matting using large kernel matting Laplacian matrices. In: Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE, 2010. 2165-2172
    [3] Lee P, Wu Y. Nonlocal matting. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA: IEEE, 2011. 2193-2200
    [4] Chen Q F, Li D Z Y, Tang C K. KNN matting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(9): 2175-2188 doi: 10.1109/TPAMI.2013.18
    [5] Shi Y F, Au O C, Pang J H, Tang K T, Sun W X, Zhang H, et al. Color clustering matting. In: Proceedings of the 2013 IEEE International Conference on Multimedia and Expo. San Jose, USA: IEEE, 2013. 1-6
    [6] Grady L, Schiwietz T, Aharon S, Westermann R. Random walks for interactive alpha matting. In: Proceedings of the 2005 International Conference on Visualization, Imaging and Image Processing. Benidorm, Spain: ACTA, 2005. 423-429
    [7] Zhang Z P, Zhu Q S, Xie Y Q. Learning based alpha matting using support vector regression. In: Proceedings of the 2012 19th IEEE International Conference on Image Processing, Orlando, USA: IEEE, 2012. 2109-2112
    [8] Chuang Y Y, Curless B, Salesin D H, Szeliski R. A Bayesian approach to digital matting. In: Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition. Kauai, USA: IEEE, 2001. 264-271
    [9] Wang J, Cohen M F. An iterative optimization approach for unified image segmentation and matting. In: Proceedings of the 10th IEEE International Conference on Computer Vision. Beijing, China: IEEE, 2005. 936-943
    [10] Wang J, Cohen M F. Optimized color sampling for robust matting. In: Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, USA: IEEE, 2007. 1-8
    [11] Rhemann C, Rother C, Gelautz M. Improving color modeling for alpha matting. In: Proceedings of British Machine Vision Conference. Leeds, UK: BMVA, 2008. 1155-1164
    [12] He K M, Rhemann C, Rother C, Tang X O, Sun J. A global sampling method for alpha matting. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA: IEEE, 2011. 2049-2056
    [13] Shahrian E, Rajan D, Price B, Cohen S. Improving image matting using comprehensive sampling sets. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA: IEEE, 2013. 636-643
    [14] Varnousfaderani E S, Rajan D. Weighted color and texture sample selection for image matting. IEEE Transactions on Image Processing, 2013, 22(11): 4260-4270 doi: 10.1109/TIP.2013.2271549
    [15] Karacan L, Erdem A, Erdem E. Image Matting with KL-Divergence Based Sparse Sampling. In: Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Chile: IEEE, 2015. 424-432
    [16] Gastal E S L, Oliveira M M. Shared sampling for real-time alpha matting. Computer Graphics Forum, 2010, 29(2): 575-584 doi: 10.1111/j.1467-8659.2009.01627.x
    [17] Rhemann C, Rother C, Rav-Acha A, Sharp T. High resolution matting via interactive trimap segmentation. In: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, USA: IEEE, 2008. 1-8
    [18] 姚桂林, 姚鸿勋.基于仿射方法的图像抠图算法综述. 计算机辅助设计与图形学学报, 2016, 28(4): 677-692 doi: 10.3969/j.issn.1003-9775.2016.04.019

    Yao Gui-Lin, Yao Hong-Xun. A Review on Affinity Based Algorithms in Image Matting. Journal of Computer-Aided Design & Computer Graphics, 2016, 28(4): 677-692 doi: 10.3969/j.issn.1003-9775.2016.04.019
    [19] Johnson J, Varnousfaderani E S, Cholakkal H, Rajan D. Sparse Coding for Alpha Matting. IEEE Transactions on Image Processing, 2016, 25(7): 3032-3043 doi: 10.1109/TIP.2016.2555705
    [20] Cho D, Tai Y W, Kweon I S. Deep convolutional neural network for natural image matting using initial alpha mattes. IEEE Transactions on Image Processing, 2019, 28(3): 1054-1067 doi: 10.1109/TIP.2018.2872925
    [21] Shen X Y, Tao X, Gao H Y, Zhou C, Jia J Y. Deep automatic portrait matting. In: Proceedings of the 14th European Conference on Computer Vision. Amsterdam, Netherlands: Springer, 2016. 92-107
    [22] Xu N, Price B, Cohen S, Huang T. Deep image matting. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 311-320
    [23] Rhemann C, Rother C, Wang J, Gelautz M, Kohli P, Rott P. A perceptually motivated online benchmark for image matting. In: Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA: IEEE, 2009. 1826-1833
    [24] Yao G L. A survey on pre-processing in image matting. Journal of Computer Science and Technology, 2017, 32(1): 122-138 doi: 10.1007/s11390-017-1709-z
    [25] Yao G L, Zhao Z J, Liu S H. A comprehensive survey on sampling based image matting. Computer Graphics Forum. 2017, 36(8): 613-628 doi: 10.1111/cgf.13156
    [26] Chen X, He F Z, Yu H P. A matting method based on full feature coverage. Multimedia Tools and Applications, 2018: 1-29
    [27] 姚桂林, 刘绍辉, 李敏, 姚鸿勋, 胡文.后处理在数字抠像中的应用与解析. 电子学报, 2017, 45(3): 719-729 doi: 10.3969/j.issn.0372-2112.2017.03.032

    Yao Gui-Lin, Liu Shao-Hui, Li Min, Yao Hong-Xun, Hu Wen. Applications and analyses on post processing in image matting. Acta Electronica Sinica, 2017, 45(3): 719-729 doi: 10.3969/j.issn.0372-2112.2017.03.032
    [28] 张展鹏, 朱青松, 谢耀钦.数字抠像的最新研究进展. 自动化学报, 2012, 38(10): 1571-1584 doi: 10.3724/SP.J.1004.2012.01571

    Zhang Zhan-Peng, Zhu Qing-Song, Xie Yao-Qin. The latest research progress on digital matting. Acta Automatica Sinica, 2012, 38(10): 1571-1584 doi: 10.3724/SP.J.1004.2012.01571
    [29] Zhou Y, He F Z, Hou N, Qiu Y M. Parallel ant colony optimization on multi-core SIMD CPUs. Future Generation Computer Systems, 2018, 79: 473-487 doi: 10.1016/j.future.2017.09.073
    [30] Zhou Y, He F Z, Qiu Y M. Dynamic strategy based parallel ant colony optimization on GPUs for TSPs. Science China Information Sciences, 2017, 60(6): 260-262
  • 加载中
图(12) / 表(2)
计量
  • 文章访问数:  60
  • HTML全文浏览量:  55
  • PDF下载量:  42
  • 被引次数: 0
出版历程
  • 收稿日期:  2018-05-28
  • 录用日期:  2019-01-02
  • 刊出日期:  2021-01-29

目录

    /

    返回文章
    返回