Dynamic Occlusion Avoidance Approach by Means of Occlusion Region Model and Object Motion Estimation
-
摘要: 对于运动视觉目标，如何对遮挡区域进行规避是视觉领域一个具有挑战性的问题.本文提出了一种新颖的基于运动视觉目标深度图像利用遮挡信息实现动态遮挡规避的方法.该方法主要利用遮挡区域最佳观测方位模型和视觉目标运动估计方程，通过合理规划摄像机的观测方位逐渐完成对遮挡区域的观测.主要贡献在于：1）提出了深度图像遮挡边界中关键点的概念，利用其构建关键线段对遮挡区域进行快速建模；2）基于关键线段和遮挡区域建模结果，提出了一种构建遮挡区域最佳观测方位模型的方法；3）提出一种混合曲率特征，通过计算深度图像对应的混合曲率矩阵，增加了图像匹配过程中提取特征点的数量，有利于准确估计视觉目标的运动.实验结果验证了所提方法的可行性和有效性.Abstract: How to avoid the occlusion region of a moving visual object is a challenging problem in the visual field. Based on the occlusion information in the depth image of a moving visual object, this paper proposes a novel dynamic occlusion avoidance approach which plans the camera position by utilizing the best view model of occlusion area and the visual target motion estimation equation. This work has three contributions. The first one is the concept of key point, which constitutes the key line segment to construct the model of occlusion region. The second one is the approach for constructing the best view model of occlusion region based on the key line segment and the occlusion region model. The third one is the feature of mixed curvature. The number of feature points extracted in the process of image matching is increased by calculating the mixed curvature matrix corresponding to the depth image, which is conducive to estimating motion of visual object accurately. Experimental results demonstrate the feasibility and effectiveness of the proposed approach.
-
Key words:
- Depth image /
- occlusion information /
- occlusion region modeling /
- next best view /
- motion estimation
1) 本文责任编委 王亮 -
图 5 当前摄像机观测中心点与关键线段的位置关系
Fig. 5 The position relation between the observation center point of current camera and the key line segments
图 6 Bunny的深度图像及其对应的混合曲率矩阵图像
Fig. 6 The depth image of Bunny and its corresponding mixed curvature matrix image
图 10 不同曲率矩阵的可视化结果
Fig. 10 The result of visualization for different curvature matrices
图 12 两种方法在下一最佳观测方位下获取的视觉目标深度图像
Fig. 12 The depth images of visual object in next best view for two methods
表 1 不同曲率矩阵提取特征点结果的量化评估
Table 1 Quantitative evaluation of the result of feature points extracted by different curvature matrices
视觉目标名称 ${N_{\rm C}}$ ${N_{\rm G}}$ ${N_{\rm GC}}$ Bunny 210 109 387 Duck 201 95 378 Mole 144 110 277 Rocker 105 46 148 Knot 175 87 298 
下载: 导出CSV
表 2 视觉目标Dragon做平移运动时的动态遮挡规避过程
Table 2 The dynamic occlusion avoidance process of visual object Dragon with translation motion
深度图像 筛选后特征点匹配图像 筛选后匹配点数 观测方向 观测位置(mm) 观测面积(mm${^2}$) 第1幅 
[-190.00,
-200.00,
-118.00][-190.00,
-200.00,
118.00]第2幅 
125 [-190.00,
200.00,
-118.00][190.00,
-200.00,
-118.00]79.85 第3幅 
130 [-177.41,
212.58,
-114.91][178.12,
-212.42,
114.78]142.64 ⋮ 第6幅 
131 [-138.89,
245.81,
-101.09][139.54,
-245.54,
101.32]343.87 ⋮ 第10幅 
119 [-77.45,
297.12,
-78.71][77.94,
-278.79,
78.89]653.20 ⋮ 第15幅 
121 [-1.99,
296.93,
-46.55][2.96,
-296.31,
47.01]963.37 第16幅 
[5.47,
297.18,
-43.25][-4.78,
-296.82,
43.57]990.57
下载: 导出CSV
表 3 视觉目标Bunny做旋转运动时的动态遮挡规避过程
Table 3 The dynamic occlusion avoidance process of visual object Bunny with rotation motion
深度图像 筛选后特征点匹配图像 筛选后匹配点数 观测方向 观测位置(mm) 观测面积(mm${^2}$) 第1幅 
[196.00,
-170.00,
-150.00][-196.00
, 170.00,
150.00]第2幅 
212 [196.00,
-170.00,
-150.00][-196.00,
170.00,
150.00]476.11 第3幅 
143 [209.64,
-197.55,
-81.44][-208.74,
195.66,
89.22]1139.89 ⋮ 第5幅 
167 [211.73, [-210.13, 2165.26 -216.22, 212.87, -10.29] 18.59] ⋮ 第9幅 
181 [198.26, [-195.49, 3131.86 -220.89, 217.06, -72.18] -66.95] ⋮ 第14幅 
208 [149.48, [-148.50, 3 902.76] -192.54, 191.19, 178.01] -176.66] 第15幅 
[145.25, [-144.46, 3 928.31 -188.99, 188.14, 184.46] -183.16]
下载: 导出CSV
表 4 视觉目标Bunny做平移和旋转运动时的动态遮挡规避过程
Table 4 The dynamic occlusion avoidance process of visual object Bunny with both translation and rotation motion
深度图像 筛选后特征点匹配图像 筛选后匹配点数 观测方向 观测位置(mm) 观测面积(mm${^2}$) 第1幅 
[-190.00, [190.00, 200.00, -200.00, -118.00] -118.00] 第2幅 
224 [-190.00, [190.00, 192.57 200.00, -200.00, -118.00] -118.00] 第3幅 
193 [-161.66, [164.35, 301.96 212.61, -211.53, -137.59] 135.16] ⋮ 第7幅 
204 [-93.59, [95.53, 815.42 227.26, -226.39, -173.45] 172.17] ⋮ 第15幅 
201 [18.87, [-17.23, 1 476.87 211.37, -201.95, -213.50] 212.67] ⋮ 第20幅 
202 [63.67, [-62.42, 1 678.43 191.36, -191.18, -223.15] 222.66] 第21幅 
[71.12, [-69.81, 1 707.26 187.29, -187.05, -224.59] 223.98]
下载: 导出CSV
表 5 视觉目标Printer做平移运动时的动态遮挡规避过程
Table 5 The dynamic occlusion avoidance process of visual object Printer with translation motion
深度图像 筛选后特征点匹配图像 筛选后匹配点数 观测方向 观测位置(mm) 观测面积(mm${^2}$) 第1幅 
[0.00, [0.00, -1.00, 1.00, -1 200.00] 1 200.00] 第2幅 
73 [0.00, [0.00, 321.47 -1.00, 1.00, -1 200.00] 1 200.00] ⋮ 第4幅 
85 [4.45, [-3.12, 587.21 -126.90, 127.21, -917.06] 917.43] ⋮ 第7幅 
133 [-14.13, [-13.78, 1 102.06 -350.78, 351.64, -682.74] 682.95] ⋮ 第18幅 
187 [49.91, [-49.85, 4 537.99 -1 089.01, 1 088.96, -151.39] 151.51] 第19幅 
[53.07 [-52.93, 4 566.16 -1 191.18 1 191.83, -129.07] 129.37]
下载: 导出CSV
表 6 视觉目标Kettle做旋转运动时的动态遮挡规避过程
Table 6 The dynamic occlusion avoidance process of visual object Kettle with rotation motion
深度图像 筛选后特征点匹配图像 筛选后匹配点数 观测方向 观测位置(mm) 观测面积(mm${^2}$) 第1幅 
[0.00, [0.00, 1.00, -1.00, -600.00] 600.00] 第2幅 
92 [0.00, [0.00, 65.31 1.00, -1.00, -600.00] 600.00] 第3幅 
160 [25.11, [-24.37, 112.74 12.97, -11.56, -583.63] 581.70] ⋮ 第7幅 
182 [368.26, [-367.81, 581.09 124.46, -123.11, -398.30] 397.05] ⋮ 第11幅 
192 [601.97, [-601.32, 844.53 274.61, -274.01, -200.72] 200.08] 第12幅 
[599.19, [-598.77, 871.52 272.53, -271.21, -201.67] 201.16]
下载: 导出CSV
表 7 视觉目标Plant同时做旋转和平移运动时的动态遮挡规避过程
Table 7 The dynamic occlusion avoidance process of visual object Plant with both translation and rotation motion
深度图像 筛选后特征点匹配图像 筛选后匹配点数 观测方向 观测位置(mm) 观测面积(mm${^2}$) 第1幅 
[0.00 [0.00, 1.00, -1.00, -650.00] 650.00] 第2幅 
167 [0.00, [0.00, 89.47 1.00, -1.00, -650.00] 650.00] ⋮ 第5幅 
209 [151.02, [-149.56, 237.01 -126.37, 127.21, -471.21] 471.44] ⋮ 第7幅 
257 [218.62, [-217.38, 437.91 -251.09, 251.64, -367.44] 369.18] ⋮ 第15幅 
119 [434.17, [-436.55, 1 123.24 -648.52, 648.77, -32.81] 33.81] 第16幅 
[435.89, [-437.68, 1 152.32 -648.97, 649.10, -30.88] 31.14]
下载: 导出CSV
表 8 两种建模方法的时间消耗 (ms)
Table 8 Time consumption of two modeling methods (ms)
方法名称 时间消耗 平均时间消耗 Bunny Duck Mole Rocker Dragon 文献[13] 69.51 40.03 56.04 42.87 83.03 58.29 本文 6.83 4.19 6.96 4.77 8.28 6.21
下载: 导出CSV
表 9 两种方法的下一最佳观测方位实验结果量化评估
Table 9 The quantitative evaluation of experimental results in next best view for two methods
视觉目标名称 文献[13]方法 本文方法 ${N_{\rm nbv}}$ ${N_{\rm o}}$ ${N_{\rm new}}$ ${R_{\rm o}}$ ${R_{\rm new}}$ ${N_{\rm nbv}}$ ${N_{\rm o}}$ ${N_{\rm new}}$ ${R_{\rm o}}$ ${R_{\rm new}}$ Bunny 16 071 1 835 14 236 11.42 88.58 16 087 1 023 15 064 6.36 93.64 Duck 17 622 5 187 12 444 29.38 70.62 17 418 4 286 13 132 24.61 75.39 Mole 12 354 1 930 10 424 15.62 84.38 15 996 1 942 14 054 12.14 87.86 Rocker 10 291 346 9 945 3.36 96.64 9 880 309 9 571 3.13 96.87 Dragon 9 090 416 8 674 4.58 95.42 9 358 317 9 041 3.39 96.61
下载: 导出CSV
表 10 两种运动估计方法的对比结果
Table 10 The comparison results of two motion estimation methods
运动方式 方法 结果(mm) 消耗时间(ms) 以3mm/s速度沿向量[1, 0, 0]${^{\rm T}}$平移 Ground truth [0.3, -1, 300]${^{\rm T}}$ 文献[13] [0.398, -1.158, 300.114]${^{\rm T}}$ 434.14 本文方法 [0.341, -1.026, 300.015]${^{\rm T}}$ 119.55 以2/s速度绕向量[2, 1, 0]${^{\rm T}}$旋转 Ground truth [4.682, -10.364, 299.789]${^{\rm T}}$ 文献[13] [3.541, -7.003, 299.551]${^{\rm T}}$ 573.09 本文方法 [4.531, -9.158, 299.865]${^{\rm T}}$ 212.95 以3mm/s速度沿向量 Ground truth [4.982, -10.364, 299.789]${^{\rm T}}$ [1, 0, 0]${^{\rm T}}$平移的同时 文献[13] [3.671, -7.119, 299.518]${^{\rm T}}$ 473.66 以2/s度绕向量[ 2, 1, 0]${^{\rm T}}$旋转 本文方法 [4.768, -9.690, 299.606]${^{\rm T}}$ 238.62
下载: 导出CSV
表 11 两种动态遮挡规避方法的量化评估
Table 11 The quantitative evaluation of dynamic occlusion avoidance for two methods
视觉目标 实验组别 文献[13]方法 本文方法 ${\sum_{j=1}^N{S_j}}$ ${S_{\rm purpose}}$ ${S_{\rm view}}$ ${\eta}$ ${\bar T}$ ${\sum_{j=1}^N{S_j}}$ ${S_{\rm purpose}}$ ${S_{\rm view}}$ ${\eta}$ ${\bar T}$- (mm${^2}$) (mm${^2}$) (mm${^2}$) $(\%)$ (s) (mm${^2}$) (mm${^2}$) (mm${^2}$) $(\%)$ (s)- Bunny 1 7 289.26 2 189.12 3 025.58 94.87 0.93 7 588.85 3 729.86 3 682.89 98.74 0.29 2 7 289.26 3 189.12 2 654.63 83.24 0.97 7 588.85 3729.86 3 467.21 92.96 0.35- 3 5 028.92 3 238.47 3 017.48 93.18 0.91 4 892.33 3 556.86 3 510.18 98.69 0.36- 4 4 414.50 1 758.18 1 553.48 88.36 0.95 4 575.69 1 947.95 1 837.64 94.34 0.37- Rocker 5 1 905.09 1 440.92 1 363.45 94.62 0.93 2 007.43 1 567.96 1 485.38 94.73 0.33- 6 3 433.54 2 707.33 1 947.98 71.95 0.98 3 364.32 2 832.76 2 396.03 84.58 0.40 Duck 7 1 911.10 1 025.27 733.58 71.55 1.02 1 898.01 1 071.93 830.93 77.52 0.39 8 3 307.46 2 788.37 2 748.35 98.56 0.93 3 316.25 2 928.41 2 793.99 95.41 0.34 Mole 9 320.01 221.68 133.09 60.04 1.02 352.18 262.3 186.35 71.04 0.38 Dragon 10 5 736.48 1 948.61 1 485.98 76.26 1.10 5 831.1 2 061.78 1 710.62 82.87 0.36
下载: 导出CSV
-
[1] Koller D, Weber J, Malik J. Robust multiple car tracking with occlusion reasoning. In: Proceedings of the 3rd European Conference on Computer Vision. Stockholm, Sweden: Springer-Verlag, 1994. 189-196 [2] Senior A, Hampapur A, Tian Y L, Brown L, Pankanti S, Bolle R. Appearance models for occlusion handling. Image and Vision Computing, 2006, 24(11):1233-1243 doi: 10.1016/j.imavis.2005.06.007 [3] Chang T H, Gong S G, Ong E J. Tracking multiple people under occlusion using multiple cameras. In: Proceedings of the 2000 British Machine Vision Conference. Bristol, UK, Greece: BMVC, 2000. 566-575 [4] Maver J, Bajcsy R. Occlusions as a guide for planning the next view. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(5):417-433 doi: 10.1109/34.211463 [5] Li Y F, Liu Z G. Information entropy-based viewpoint planning for 3-D object reconstruction. IEEE Transactions on Robotics, 2005, 21(3):324-337 doi: 10.1109/TRO.2004.837239 [6] Banta J E, Wong L R, Dumont C, Abidi M A. A next-best-view system for autonomous 3-D object reconstruction. IEEE Transactions on Systems, Man, and Cybernetics—Part A:Systems and Humans, 2000, 30(5):589-598 doi: 10.1109/3468.867866 [7] Wu B, Sun X, Wu Q C, Yan M L, Wang H Q, Fu K. Building reconstruction from high-resolution multiview aerial imagery. IEEE Geoscience and Remote Sensing Letters, 2015, 12(4):855-859 doi: 10.1109/LGRS.2014.2364309 [8] Pito R. A solution to the next best view problem for automated surface acquisition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 21(10):1016-1030 doi: 10.1109/34.799908 [9] Potthast C, Sukhatme G S. A probabilistic framework for next best view estimation in a cluttered environment. Journal of Visual Communication and Image Representation, 2014, 25(1):148-164 doi: 10.1016/j.jvcir.2013.07.006 [10] 张世辉, 刘建新, 孔令富.基于深度图像利用遮挡信息确定下一最佳观测方位.计算机学报, 2015, 38(12):2450-2463 http://d.old.wanfangdata.com.cn/Periodical/jsjxb201512010Zhang Shi-Hui, Liu Jian-Xin, Kong Ling-Fu. Determining next best view based on occlusion information in depth image. Chinese Journal of Computers, 2015, 38(12):2450-2463 http://d.old.wanfangdata.com.cn/Periodical/jsjxb201512010 [11] Doumanoglou A, Kouskouridas R, Malassiotis S, Kim T K. Recovering 6D object pose and predicting next-best-view in the crowd. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016. 3583-3592 [12] Bissmarck F, Svensson M, Tolt G. Efficient algorithms for Next Best View evaluation. In: Proceedings of the 2015 IEEE International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany: IEEE, 2015. 5876-5883 [13] Zhang S H, He H, Zhang Y C, Li X, Sang Y. Dynamic self-occlusion avoidance approach based on the depth image sequence of moving visual object. Mathematical Problems in Engineering, 2016, 2016:Article No. 4783794 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=Doaj000004753010 [14] Zhang S H, Liu J X. A self-occlusion detection approach based on depth image using SVM. International Journal of Advanced Robotic Systems, 2012, 9(6):Article No. 230 doi: 10.5772/53823 [15] 张世辉, 韩德伟, 何欢.利用视觉目标遮挡和轮廓信息确定下一最佳观测方位.电子与信息学报, 2015, 37(12):2921-2928 http://d.old.wanfangdata.com.cn/Periodical/dzkxxk201512019Zhang Shi-Hui, Han De-Wei, He Huan. Determining next best view using occlusion and contour information of visual object. Journal of Electronics and Information Technology, 2015, 37(12):2921-2928 http://d.old.wanfangdata.com.cn/Periodical/dzkxxk201512019 [16] Besl P J, Jain R C. Segmentation through variable-order surface fitting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1988, 10(2):167-192 doi: 10.1109/34.3881 [17] Bay H, Tuytelaars T, Gool L V. SURF: speeded up robust features. In: Proceedings of the 9th European Conference on Computer Vision and Image Understanding. Graz, Austria: ECCV, 2006, 110: 404-417 [18] 张东兴, 祝明波, 邹建武, 李相平.基于相似三角形的SIFT错误匹配点剔除算法研究.计算机工程与科学, 2012, 34(4):66-70 doi: 10.3969/j.issn.1007-130X.2012.04.014Zhang Dong-Xing, Zhu Ming-Bo, Zou Jian-Wu, Li Xiang-Ping. Research on wrong match pairs elimination based on similar triangles in the SIFT algorithm. Computer Engineering and Science, 2012, 34(4):66-70 doi: 10.3969/j.issn.1007-130X.2012.04.014 [19] 赵峰, 黄庆明, 高文.一种基于奇异值分解的图像匹配算法.计算机研究与发展, 2010, 47(1):23-32 http://d.old.wanfangdata.com.cn/Periodical/jsjyjyfz201001004Zhao Feng, Huang Qing-Ming, Gao Wen. An image matching algorithm based on singular value decomposition. Journal of Computer Research and Development, 2010, 47(1):23-32 http://d.old.wanfangdata.com.cn/Periodical/jsjyjyfz201001004 [20] Hetzel G, Leibe B, Levi P, Schiele B. 3D object recognition from range images using local feature histograms. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Kauai, HI, USA: IEEE, 2001. 394-399 -
计量
- 文章访问数: 2635
- HTML全文浏览量: 175
- PDF下载量: 376
- 被引次数: 0
