基于随机森林误分类处理的3D人体姿态估计

蔡轶珩; 王雪艳; 马杰; 孔欣然

doi:10.16383/j.aas.c180314

基于随机森林误分类处理的3D人体姿态估计

doi: 10.16383/j.aas.c180314

蔡轶珩^1, ,,
王雪艳^1,,
马杰^1,,
孔欣然^1,

1.
北京工业大学信息学部信号与信息处理研究室北京 100124

基金项目:

科技部国家重点研发计划课题 2017YFC1703302

北京市教委科技项目 KM201710005028

详细信息

作者简介:
王雪艳北京工业大学信息学部研究生. 2016年获得河北工程大学信息与电气工程学院学士学位.主要研究方向为图像与视频处理. E-mail: xinxiY23@126.com

马杰北京工业大学信息学部研究生. 2016年获得北京工业大学信息学部学士学位.主要研究方向为图像与视频信号处理. E-mail: 13241247924@163.com

孔欣然北京工业大学信息学部研究生. 2016年获得北京工业大学信息学部学士学位.主要研究方向为图像与视频处理.E-mail:duzouran@163.com

通讯作者:
蔡轶珩北京工业大学信息学部副教授.美国罗切斯特大学访问学者. 1998年获得合肥工业大学精密仪器专业硕士学位. 2007年获得北京工业大学智能化信息处理专业博士学位.主要研究方向为医学图像信息处理, 光度立体三维表面重建, 视觉感知信息处理.本文通信作者. E-mail: caiyiheng@bjut.edu.cn

计量
- 文章访问数: 1811
- HTML全文浏览量: 143
- PDF下载量: 214
- 被引次数: 0
出版历程
- 收稿日期: 2018-05-16
- 录用日期: 2018-10-06
- 刊出日期: 2020-07-24

3D Human Pose Estimation Based on Random Forest Misclassiflcation Processing Mechanism

CAI Yi-Heng^{1
, ,},
WANG Xue-Yan^1
,,
MA Jie^1
,,
KONG Xin-Ran^1
,

1.
Signal and Information Processing Laboratory, Department of Information, Beijing University of Technology, Beijing 100124

Funds:

National Key Research and Development Program 2017YFC1703302

Science and Technology Projects of Beijing Municipal Education Commission of China KM201710005028

More Information

Author Bio:
WANG Xue-Yan Master student in the Department of Information, Beijing University of Technology. She received her bachelor degree from the College of Information and Electrical Engineering, Hebei University of Engineering in 2016. Her research interest covers image and video processing

MA Jie Master student in the Department of Information, Beijing University of Technology. He received his bachelor degree from the College of Department of Information, Beijing University of Technology in 2016. His research interest covers image and video signal processing

KONG Xin-Ran Master student in the Department of Information, Beijing University of Technology. She received her bachelor degree from the College of Department of Information, Beijing University of Technology in 2016. Her research interest covers image and video processing

Corresponding author: CAI Yi-Heng Associate professor in the Department of Information, Beijing University of Technology. Visiting scholar in the University of Rochester at USA. She received her master degree in precision instruments from Southeast University in 1998, and Ph. D. degree in intelligent information processing from Beijing University of Technology in 2007. Her research interest covers medical image information processing, photometric three dimensional surface reconstruction, and visual perception information processing. Corresponding author of this paper

摘要

摘要: 为解决基于随机森林的3D人体姿态估计算法容易出现的误分类问题, 提出一种基于自适应融合特征提取和误分类处理机制的改进算法.该算法利用自适应融合特征提取方法自适应提取深度融合特征, 此特征可表达图像距离信息和部位尺寸信息, 增强特征的表征能力; 针对识别部位误分类问题, 分别从识别部位误分点聚集情况和迭代整合思想出发, 提出误分类处理机制, 改善部位识别结果; 最后提出可进一步处理误分点的改进主方向分析(Principal direction analysis, PDA)算法, 自适应计算出部位主方向向量, 实现3D人体姿态估计.结果表明, 该算法能有效去除部位误分点, 并显著改善了3D人体姿态估计.
- 人体姿态估计 /
- 随机森林 /
- 误分类处理 /
- 主方向分析
Abstract: This paper proposed an improved method which can reduce the misclassification in human pose estimation based on random forest and increase the accuracy, included adaptive fusion feature extraction and misclassification processing mechanism. Firstly, we improved the method of feature extraction to adaptive extract deep fusion feature with adaptive feature fusion extractive method, so that, both distance information and part information could enhance feature expression. Furthermore, owing to inspiration from error cluster analysis and iteration thought, the misclassification processing mechanism is proposed to handle misclassi-fication appearance. Finally, we achieved accurate human pose estimation from single depth images by applying the principal direction vector based on the improved principal direction analysis (PDA) algorithm. The experimental results demonstrated that this algorithm can efficiently eliminate several misclassifications and improve the accuracy of the 3D pose estimation.
- Human pose estimation /
- random forest /
- misclassification processing /
- principal direction analysis (PDA)
Recommended by Associate Editor HUANG Qing-Ming
注释:

1) 本文责任编委黄庆明

HTML全文

图 1 合成深度图像数据库

Fig. 1 Synthetic depth image dataset

下载: 全尺寸图片幻灯片

图 2 算法整体流程图

Fig. 2 Overview of proposed technique

下载: 全尺寸图片幻灯片

图 3 偏移向量对示意图

Fig. 3 Offset vector pair

下载: 全尺寸图片幻灯片

图 4 合成图像部位尺寸示意图

Fig. 4 Part size of the synthetic image

下载: 全尺寸图片幻灯片

图 5 基于Kmeans算法在不同总聚类下的部位识别结果

Fig. 5 Part recognition results based on Kmeans algorithm under different total clusters

下载: 全尺寸图片幻灯片

图 6 多级随机森林整合算法流程图

Fig. 6 The flowchart of the multi-level random forest integration algorithm

下载: 全尺寸图片幻灯片

图 7 不同特征提取方法的部位分类结果对比

Fig. 7 Results of different feature extraction methods in part classification

下载: 全尺寸图片幻灯片

图 8 本文误分类处理机制处理后的部位分类结果图((a)为随机森林初始识别+膨胀的结果; (b)为分级聚类+膨胀的结果; (c)为多级随机森林整合+膨胀的结果)

Fig. 8 Part classification result based on misclassification processing mechanism ((a)~(c) representing the results of random forest, Kmeans, and multi-level random forest integration algorithm, respectively

下载: 全尺寸图片幻灯片

图 9 本文改进PDA算法和PDA算法的对比识别结果图((a)多级随机森林整合算法的识别结果; (b) PDA算法处理+膨胀的识别结果; (c)改进的PDA算法处理+膨胀的识别结果)

Fig. 9 Contrast recognition results for improved PDA algorithm and PDA algorithm ((a) multi-level random forest integration algorithm, (b)~(c) representing the results of PDA algorithms, and improved PDA algorithms, respectively)

下载: 全尺寸图片幻灯片

图 10 合成数据集上的姿态估计结果((a)深度图像; (b)误分类处理前的结果; (c) Kmeans处理后的结果; (d)多级随机森林整合后的结果; (e) groundtruth)

Fig. 10 Pose estimation on the synthetic dataset ((a) depth image, (b)~(d) representing the results of random forest, Kmeans, multi-level random forest integration algorithm, respectively, (e) ground truth)

下载: 全尺寸图片幻灯片

图 11 ITOP数据集上的姿态估计结果((a)~(d)算法同图 10 (a)~(d))

Fig. 11 Pose estimation on the ITOP dataset ((a)~(d) same as Fig. 10 (a)~(d))

下载: 全尺寸图片幻灯片

图 12 实际拍摄的深度图像上的姿态估计结果((a)~(d)算法同图 10 (a)~(d))

Fig. 12 Pose estimation on the actual captured depth image ((a)~(d) same as Fig. 10 (a)~(d))

下载: 全尺寸图片幻灯片

表 1 不同分类器的部位平均识别准确率结果

Table 1 Average recognition accuracy results for different classifiers

方法	训练时间(s)	平均识别准确率(%)
Ababoost	2 377.93	52.58
KNN	977.46	66.62
RF	187.97	70.29

下载: 导出CSV

表 2 不同特征方法的部位平均分类准确率结果

Table 2 Classification accuracy results for different feature methods

方法	平均识别准确率
深度梯度差分特征	0.7046
文献[19]改进型特征	0.8245
文献[20] FCN方法	0.8417
本文深度数据特征	0.6215
本文自适应深度梯度特征	0.8405
本文融合特征	0.8603

下载: 导出CSV

表 3 合成深度图像上的肘部角度误差结果

Table 3 Elbow angle error results on synthetic depth images

算法	左肘角度误差	右肘角度误差
深度梯度特征+ PDA (文献[6])	14.5575°	13.5241°
自适应(本文) + PDA (文献[6])	12.7654°	13.3342°
自适应+改进的PDA (本文)	12.2893°	13.1284°
融合特征+改进的PDA (本文)	11.8462°	12.0331°
自适应+ Kmeans +改进的PDA (本文)	11.9879°	12.7443°
融合特征+ Kmeans+改进的PDA (本文)	10.2546°	10.6436°
自适应+多级整合+改进的PDA (本文)	9.9637°	9.6216°
融合特征+多级整合+改进的PDA (本文)	8.4581°	8.6824°

下载: 导出CSV

参考文献(24)

[1]	史青宣, 邸慧军, 陆耀, 田学东.基于中粒度模型的视频人体姿态估计.自动化学报, 2018, 44(4): 646-655 doi: 10.16383/j.aas.2018.c160847 Shi Qing-Xuan, Di Hui-Jun, Lu Yao, Tian Xue-Dong. A medium granularity model for human pose estimation in video. Acta Automatica Sinica, 2018, 44(4): 646-655 doi: 10.16383/j.aas.2018.c160847
[2]	李幼蛟, 卓力, 张菁, 李嘉锋, 张辉.行人再识别技术综述.自动化学报, 2018, 44(9): 1554-1568 doi: 10.16383/j.aas.2018.c170505 Li You-Jiao, Zhuo Li, Zhang Jing, Li Jing-Feng, Zhang Hui. A survey of person re-identification. Acta Automatica Sinica, 2018, 44(9): 1554-1568 doi: 10.16383/j.aas.2018.c170505
[3]	朱煜, 赵江坤, 王逸宁, 郑兵兵.基于深度学习的人体行为识别算法综述.自动化学报, 2016, 42(6): 848-857 doi: 10.16383/j.aas.2016.c150710 Zhu Yu, Zhao Jiang-Kun, Wang Yi-Ning, Zheng Bing-Bing. A review of human action recognition based on deep learning. Acta Automatica Sinica, 2016, 42(6): 848-857 doi: 10.16383/j.aas.2016.c150710
[4]	Shotton J, Girshick R, Fitzgibbon A, Sharp T, Cook M, Finocchio M, et al. Efficient human pose estimation from single depth images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12): 2821-2840 doi: 10.1109/TPAMI.2012.241
[5]	杜霄鹏, 郝建平, 李星新, 杨俊.基于单一深度图像的人体姿态实时识别技术研究.计算机与现代化, 2012, 1(4): 192-195 doi: 10.3969/j.issn.1006-2475.2012.04.052 Du Xiao-Peng, Hao Jian-Ping, Li Xing-Xin, Yang Jun. Human pose recognition research based on single depth images. Computer and Modernization, 2012, 1(4): 192-195 doi: 10.3969/j.issn.1006-2475.2012.04.052
[6]	Dinh D L, Han H S, Jeon H J, Lee S, Kim T S. Principal direction analysis-based real-time 3D human pose reconstruction from a single depth image. In: Proceedings of Symposium on Information and Communication Technology. New York, USA: ACM, 2013. 206-212
[7]	殷海艳.基于深度图像的人体姿态识别[硕士学位论文].北京工业大学, 2013 Yin Hai-Yan. Human body pose recognition from the depth image[Master thesis]. Beijing University of Technology, China, 2013
[8]	Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, et al. Real-time human pose recognition in parts from single depth images. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington, D. C., USA: IEEE, 2011. 1297-1304
[9]	Park S, Hwang J, Kwak N. 3D Human pose estimation using convolutional neural networks with 2D pose information. In: Proceedings of the 2016 IEEE Conference on European Conference on Computer Vision (ECCV). Netherlands, Amsterdam: IEEE, 2016. 156-169
[10]	Wei S E, Ramakrishna V, Kanade T, Sheikh Y. Convolutional pose machines. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016. 4724-4732
[11]	Cao Zhe, Simon T, Wei S E, Sheikh Y. Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016. 7291-7299
[12]	Toshev A, Szegedy C. DeepPose: Human pose estimation via deep neural networks. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, OH, USA: IEEE, 2014. 1653-1660
[13]	Wang Ke-Ze, Zhai Sheng-Fu, Cheng Hui, Liang Xiao-Dan, Lin Liang. Human pose estimation from depth images via inference embedded multi-task learning. In: Proceedings of the 2016 ACM on Multimedia Conference. New York, USA: ACM, 2016. 1227-1236
[14]	Haque A, Peng Bo-Ya, Luo Ze-Lun, Alahi A, Yeung S, Li Fei-Fei. Towards viewpoint invariant 3D human pose estimation. In: Proceedings of European Conference on Computer Vision (ECCV). Netherlands, Amsterdam: IEEE, 2016. 160-177
[15]	Han Xu-Feng, Leung T, Jia Yang-Qing, Sukthankar R, Berg A C. MatchNet: Unifying feature and metric learning for patch-based matching. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015. 3279-3286
[16]	Tu Zhuo-Wen. Exemplar-based human action pose correction and tagging. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Washington D. C., USA: IEEE, 2012. 1784-1791
[17]	Shen Wei, Deng Ke, Bai Xiang, Leyvand T. Exemplar-based human action pose correction. IEEE Transactions on Cybernetics, 2014, 44(7): 1053-1066 doi: 10.1109/TCYB.2013.2279071
[18]	Shen Wei, Lei Rui, Zeng Dan, Zhang Zhi-Jiang. Regularity guaranteed human pose correction. In: Proceedings of the 12th Asian Conference on Computer Vision (ACCV). Singapore, 2014. 242-256
[19]	张乐锋, 郑逸, 傅超.用改进的深度差分特征识别人体部位.微型机与应用, 2015, 34(14): 54-57 doi: 10.3969/j.issn.1674-7720.2015.14.017 Zhang Yue-Feng, Zheng Yi, Fu Chao. Improved depth comparison feature for the recognition of human parts. Microcomputer Its Applications, 2015, 34(14): 54-57 doi: 10.3969/j.issn.1674-7720.2015.14.017
[20]	Nishi K, Miura J. Generation of human depth images with body part labels for complex human pose recognition. Pattern Recognition, 2017, 71(6): 402-413 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=1412aca8b9eb938a3b5b629cc146ec94
[21]	吕洁, 刘亚洲, 韩庆龙, 杜晶.基于深度图像的人体关节点定位方法.海军航空工程学院学报, 2016, 31(5): 538-546 http://d.old.wanfangdata.com.cn/Periodical/hjhkgcxyxb201605008 Lv Jie, Liu Ya-Zhou, Han Qing-Long, Du Jing. Method of locationg human body joints based on depth-images. Naval Aeronautical and Astronautical University, 2016, 31(5): 538-546 http://d.old.wanfangdata.com.cn/Periodical/hjhkgcxyxb201605008
[22]	吴敏, 杨源, 张园强, 库涛, 查宇飞, 张胜杰.深度融合特征与梯度特征的红外目标跟踪算法.空军工程大学学报·自然科学版, 2017, 18(6): 76-82 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=kjgcdxxb201706013 Wu Min, Yang Yuan, Zhang Yuan-Qiang, Ku Tao, Zha Yu-Fei, Zhang Sheng-Jie. An infrared target tracking algorithm based on the fusion of deep feature and gradient feature. Air Force Engineering University (Natural Science Edition), 2017, 18(6): 76-82 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=kjgcdxxb201706013
[23]	徐岳峰, 周书仁, 王刚, 佘凯晟.基于深度图像梯度特征的人体姿态估计.计算机工程, 2015, 41(12): 200-205 doi: 10.3969/j.issn.1000-3428.2015.12.038 Xu Yue-Feng, Zhou Shu-Ren, Wang Gang, She Kai-Sheng. Human body attitude estimation based on gradient feature of depth images. Computer Engineering, 2015, 41(12): 200-205 doi: 10.3969/j.issn.1000-3428.2015.12.038
[24]	李红波, 丁林建, 冉光勇.基于Kinect深度图像的人体识别分析.数字通信, 2012, 39(4): 21-26 doi: 10.3969/j.issn.1005-3824.2012.04.004 Li Hong-Bo, Ding Lin-Jian, Ran Guang-Yong. Human body recognition based on Kinect depth image. Digital Communication, 2012, 39(4): 21-26 doi: 10.3969/j.issn.1005-3824.2012.04.004