基于肌电−惯性融合的人体运动估计: 高斯滤波网络方法

杨旭升; 李福祥; 胡佛; 张文安

doi:10.16383/j.aas.c230581

基于肌电−惯性融合的人体运动估计: 高斯滤波网络方法

doi: 10.16383/j.aas.c230581

杨旭升^{1, 2,},
李福祥^{1, 2,},
胡佛^{1, 2,},
张文安^{1, 2,}

1.
浙江工业大学信息工程学院杭州 310023
2.
浙江省嵌入式系统联合重点实验室杭州 310023

基金项目: 浙江省“尖兵”“领雁”研发攻关计划(2022C03114), 浙江省自然科学基金(LY23F030006), 浙江省科技计划项目(2023C04032)资助

详细信息

作者简介:
杨旭升：浙江工业大学信息工程学院副教授. 主要研究方向为信息融合估计, 人体运动估计和目标定位. 本文通信作者. E-mail: xsyang@zjut.edu.cn

李福祥：浙江工业大学信息工程学院硕士研究生. 主要研究方向为多源信息融合估计、人体姿态估计. E-mail: fuxiangli@zjut.edu.cn

胡佛：浙江工业大学信息工程学院助理研究员. 主要研究方向为人机交互、情感计算和人工智能. E-mail: fohu@zjut.edu.cn

张文安：浙江工业大学信息工程学院教授. 主要研究方向为多源信息融合估计及应用. E-mail: wazhang@zjut.edu.cn

计量
- 文章访问数: 133
- HTML全文浏览量: 36
- 被引次数: 0
出版历程
- 收稿日期: 2023-09-18
- 录用日期: 2023-12-21
- 网络出版日期: 2024-02-28

Human Motion Estimation Based on EMG-Inertial Fusion: A Gaussian Filtering Network Approach

YANG Xu-Sheng^{1, 2
,},
LI Fu-Xiang^{1, 2
,},
HU Fo^{1, 2
,},
ZHANG Wen-An^{1, 2
,}

1.
College of Information Engineering, Zhejiang University of Technology, Hangzhou 310023
2.
Zhejiang Provincial United Key Laboratory of Embedded Systems, Hangzhou 310023

Funds: Supported by the “Pioneer” and “Leading Goose” Research and Development Program of Zhejiang Province (2022C03114), Natural Science Foundation of Zhejiang Province (LY23F030006), and the Key Technology Research and Development Program of Zhejiang Province (2023C04032)

More Information

Author Bio:
YANG Xu-Sheng　Associate professor at the College of Information Engineering, Zhejiang University of Technology. His research interest covers information fusion estimation, human motion estimation and target positioning. Corresponding author of this paper

LI Fu-Xiang　Master student at the College of Information Engineering, Zhejiang University of Technology. His research interest covers multi-sensor information fusion estimation and human motion estimation

HU Fo　Assistant researcher at the College of Information Engineering, Zhejiang University of Technology. His research interest covers human machine interaction, emotional computing and artificial intelligence

ZHANG Wen-An　Professor at the College of Information Engineering, Zhejiang University of Technology. His research interest covers multi-sensor information fusion estimation and its applications

摘要

摘要: 本文研究了基于肌电(Electromyography, EMG)−惯性融合的人体运动估计问题, 提出了一种序贯渐进高斯滤波网络(Sequential progressive Gaussian filtering network, SPGF-net)估计方法来形成肌电和惯性的互补性优势, 以提高人体运动估计精度和稳定性. 首先, 利用卷积神经网络对观测数据进行特征提取, 以及利用长短期记忆(Long short-term memory, LSTM)网络模型来学习噪声统计特性和量测模型. 其次, 采用序贯融合的方式融合异构传感器量测特征, 以建立高斯滤波与深度学习相结合的网络模型来实现人体运动估计. 特别地, 引入渐进量测更新对网络量测特征的不确定性进行补偿. 最后, 通过实验结果表明, 相比于现有的卡尔曼滤波网络, 该融合方法在上肢关节角度估计中的均方根误差(Root mean square error, RMSE)下降了13.8%, 相关系数(R²)提高了4.36%.
- 高斯滤波网络 /
- 多传感器融合 /
- 人体运动估计 /
- 非线性卡尔曼滤波
Abstract: This paper investigates the issue of human motion estimation based on the fusion of electromyography (EMG) and inertial data. A sequential progressive Gaussian filtering network (SPGF-net) is proposed to leverage the complementary advantages of EMG and inertial data for enhancing the accuracy and stability of human motion estimation. First, a convolutional neural networks is employed to extract features from the observed data and a long short-term memory (LSTM) network model is utilized to learn the statistical properties of noise and the measurement model. Second, a sequential fusion method is adopted to fuse the measurement features from heterogeneous sensors, thus a combined network model that integrates Gaussian filtering with deep learning techniques is formed for human motion estimation. Moreover, a progressive measurement update is introduced to compensate for the uncertainty in the network＇s measurement features. Finally, experimental results indicate that, compared with existing Kalman networks, the proposed fusion method has a 13.8% reduction in root mean square error (RMSE) and a 4.36% increase in the coefficient of determination (R²) for upper limb joint angle estimation.
- Gaussian filtering network /
- multi-sensor fusion /
- human motion estimation /
- nonlinear Kalman filtering

HTML全文

图 1 多传感器融合的人体肢体估计示意图

Fig. 1 Multi-sensor fusion human body limb estimation schematic diagram

下载: 全尺寸图片幻灯片

图 2 SPGF-net结构

Fig. 2 Structure of SPGF-net

下载: 全尺寸图片幻灯片

图 3 各学习模块

Fig. 3 Various learning modules

下载: 全尺寸图片幻灯片

图 4 序贯渐进量测更新

Fig. 4 Sequential progressive measurement update

下载: 全尺寸图片幻灯片

图 5 数据采集

Fig. 5 Data collection

下载: 全尺寸图片幻灯片

图 6 S1-S4角度估计和误差曲线

Fig. 6 S1~S4 angle estimation and error curves

下载: 全尺寸图片幻灯片

表 1 五种模型性能评价

Table 1 The performance evaluation of five models

测试者	均方根误差 (RMSE)					相关系数($ R^2 $)
测试者	CNN (sEMG+IMU)	PUKF (sEMG)	PUKF (IMU)	PUKF (sEMG+IMU)	SPGF-net	CNN (sEMG+IMU)	PUKF (sEMG)	PUKF (IMU)	PUKF (sEMG+IMU)	SPGF-net
S1	9.75	11.91	12.48	9.56	9.27	0.922	0.884	0.872	0.925	0.930
S2	11.65	12.18	13.25	10.89	9.78	0.917	0.913	0.893	0.923	0.941
S3	16.18	15.90	16.42	15.63	14.15	0.864	0.868	0.859	0.876	0.896
S4	15.66	16.18	16.95	14.57	13.45	0.825	0.822	0.816	0.832	0.847
S5	24.24	23.30	23.79	22.74	18.98	0.594	0.624	0.609	0.651	0.751
S6	10.15	11.43	11.65	9.96	8.91	0.937	0.920	0.917	0.941	0.949
S7	16.31	16.62	17.19	16.13	15.90	0.856	0.851	0.847	0.860	0.869
S8	16.84	16.37	16.53	16.30	16.23	0.807	0.809	0.805	0.813	0.821
S9	9.23	9.95	10.86	8.82	7.73	0.930	0.918	0.903	0.938	0.951
S10	14.97	15.74	16.17	14.53	14.00	0.849	0.831	0.821	0.853	0.866
S11	16.86	17.19	17.66	16.62	15.78	0.852	0.846	0.838	0.857	0.864
S12	12.46	14.09	14.83	12.13	11.74	0.905	0.885	0.870	0.909	0.924
均值	14.52	15.07	15.64	13.99	12.99	0.854	0.847	0.838	0.865	0.884
标准差	4.21	3.54	3.46	3.96	3.51	0.093	0.080	0.080	0.080	0.060

下载: 导出CSV

表 2 五种模型的计算量

Table 2 The computational complexity of five models

	CNN (sEMG+IMU)	PUKF (sEMG)	PUKF (IMU)	PUKF (sEMG+IMU)	SPGF-net
FLOPs	1 237 714	719 448	619 828	1 328 864	1 419 176
Params	442 337	256 511	255 971	473 970	505 614

下载: 导出CSV

参考文献(29)

[1]	丁其川, 熊安斌, 赵新刚, 韩建达. 基于表面肌电的运动意图识别方法研究及应用综述. 自动化学报, 2016, 42(1): 13-25 Ding Qi-Chuan, Xiong An-Bin, Zhao Xin-Gang, Han Jian-Da. A review on researches and applications of sEMG-based motion intent recognition methods. Acta Automatica Sinica, 2016, 42(1): 13-25
[2]	Wen Y, Kim S J, Avrillon S, Levinel J T, Hug F, Pons J L. A deep CNN framework for neural drive estimation from HD-EMG across contraction intensities and joint angles. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30: 2950-2959 doi: 10.1109/TNSRE.2022.3215246
[3]	Huang H, Kuiken T A, Lipschutz R D. A strategy for identifying locomotion modes using surface electromyography. IEEE Transactions on Biomedical Engineering, 2009, 56(1): 65-73 doi: 10.1109/TBME.2008.2003293
[4]	Zhuang Y, Leng Y, Zhou J, Song R, Li L, Su S W. Voluntary control of an ankle joint exoskeleton by able-bodied individuals and stroke survivors using EMG-based admittance control scheme. IEEE Transactions on Biomedical Engineering, 2021, 68(2): 695-705 doi: 10.1109/TBME.2020.3012296
[5]	胡旭晖, 宋爱国, 李会军. 基于表面肌电图像的灵巧假手控制系统. 控制理论与应用,2018, 35(12): 1707-1714 Hu Xu-Hui, Song Ai-Guo, Li Hui-Jun. A dexterous robot hand control system based on surface electromyography. Control Theory & Applications, 2018, 35(12): 1707-1714
[6]	张弼, 姚杰, 赵新刚, 谈笑伟. 一种基于肌电信号的自适应人机交互控制方法. 控制理论与应用, 2020, 37(12): 2560-2570 Zhang Bi, Yao Jie, Zhao Xin-Gang, Tan Xiao-Wei. An adaptive human-robot interaction control method based on electromyography signals. Control Theory & Applications, 2020, 37(12): 2560-2570
[7]	He J Y, Jiang N. Biometric from surface electromyogram (sEMG): Feasibility of user verification and identification based on gesture recognition. Frontiers in Bioengineering and Biotechnology, 2020, 8: 58 doi: 10.3389/fbioe.2020.00058
[8]	赵新刚, 谈晓伟, 张弼. 柔性下肢外骨骼机器人研究进展及关键技术分析. 机器人, 2020, 42(3): 365-384 Zhao Xin-Gang, Tan Xiao-Wei, Zhang Bi. Development of soft lower extremity exoskeleton and its key technologies: A survey. Robot, 2020, 42(3): 365-384
[9]	李自由, 赵新刚, 张弼, 丁其川, 张道辉, 韩建达. 基于表面肌电的意图识别方法在非理想条件下的研究进展. 自动化学报, 2021, 47(5): 955-969 Li Zi-You, Zhao Xin-Gang, Zhang Bi, Ding Qi-Chuan, Zhang Dao-Hui, Han Jian-Da. Review of sEMG-based motion intent recognition methods in non-ideal conditions. Acta Automatica Sinica, 2021, 47(5): 955-969
[10]	Ding Q, Han J, Zhao X, Continuous estimation of human multi-joint angles from sEMG using a state-space model. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017, 25(9): 1518-1528 doi: 10.1109/TNSRE.2016.2639527
[11]	Zhao Y, Zhang Z, Li Z, Yang Z, Dehghani-Sanij A A, Xie S. An EMG-driven musculoskeletal model for estimating continuous wrist motion. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(12): 3113-3120 doi: 10.1109/TNSRE.2020.3038051
[12]	Zhang T, Sun H, Zou Y. An electromyography signals-based human-robot collaboration system for human motion intention recognition and realization. Robot Computer-Integrated Manufacturing, 2022, 77: 102359 doi: 10.1016/j.rcim.2022.102359
[13]	Xu H, Xiong A. Advances and disturbances in sEMG-based intentions and movements recognition: A review. IEEE Sensors Journal, 2021, 21(12): 13019-13028 doi: 10.1109/JSEN.2021.3068521
[14]	Xiong D, Zhang D, Zhao X, Zhao Y. Deep learning for EMG-based human-machine interaction: A review. IEEE/CAA Journal of Automatica Sinica, 2021, 8(3): 512-533 doi: 10.1109/JAS.2021.1003865
[15]	Stival F, Michieletto S, De Agnoi A, Pagello E. Toward a better robotic hand prosthesis control: Using EMG and IMU features for a subject independent multi joint regression model. In: Proceedings of 2018 IEEE 7th International Conference on Biomedical Robotics and Biomechatronics. Enschede, Netherlands: IEEE, 2018. 185−192
[16]	Sakamoto S I, Hutabarat Y, Owaki D, Hayashibe M. Ground reaction force and moment estimation through EMG sensing using long short-term memory network during posture coordination. Cyborg Bionic Syst, 2023, 4: 0016 doi: 10.34133/cbsystems.0016
[17]	Hollinger D, Schall M, Chen H, Bass S, Zabala M. The influence of gait phase on predicting lower-limb joint angles. IEEE Transactions on Medical Robotics and Bionics, 2023, 5(2): 343-352 doi: 10.1109/TMRB.2023.3260261
[18]	Xu L, Chen X, Cao S, Zhang X, Chen X. Feasibility study of advanced neural networks applied to sEMG-based force estimation. Sensors, 2018, 18(10): 3226 doi: 10.3390/s18103226
[19]	Han J, Ding Q, Xiong A, Zhao X. A state-space EMG model for the estimation of continuous joint movements. IEEE Transactions on Industrial Electronics, 2015, 62(7): 4267-4275 doi: 10.1109/TIE.2014.2387337
[20]	Coskun H, Achilles F, DiPietro R, Navab N, Tombari F. Long short-term memory Kalman filters: Recurrent neural estimators for pose regularization. In: Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 5524−5532
[21]	Revach G, Shlezinger N, Ni X, Escoriza A L, Van Sloun R J, Eldar Y C. KalmanNet: neural network aided Kalman filtering for partially known dynamics. IEEE Transactions on Signal Processing, 2022, 70: 1532-1547 doi: 10.1109/TSP.2022.3158588
[22]	Bao T, Zhao Y, Zaidi S A R, Xie S, Yang P, Zhang Z. A deep Kalman filter network for hand kinematics estimation using sEMG. Pattern Recognition Letters, 2021, 143: 88-94 doi: 10.1016/j.patrec.2021.01.001
[23]	杨旭升, 王雪儿, 汪鹏君, 张文安. 基于渐进无迹卡尔曼滤波网络的人体肢体运动估计. 自动化学报, 2023, 49(8): 1723-1731 Yang Xu-Sheng, Wang Xue-Er, Wang Peng-Jun, Zhang Wen-An. Estimation of human limb motion based on progressive unscented Kalman filter network. Acta Automatica Sinica, 2023, 49(8): 1723-1731
[24]	Ke A, Huang J, Chen L, Gao Z, He J. An ultra-sensitive modular hybrid EMG-FMG sensor with floating electrodes. Sensors, 2020, 20(17): 4775 doi: 10.3390/s20174775
[25]	Pasquina P F, Evangelista M, Carvalho A J, Lockhart J, Griffin S, Nanos G, et al. First-in-man demonstration of a fully implanted myoelectric sensors system to control an advanced electromechanical prosthetic hand. Journal of neuroscience methods, 2015, 244: 85-93 doi: 10.1016/j.jneumeth.2014.07.016
[26]	Zheng Z, Wu Z, Zhao R, Ni Y, Jing X, Gao S. A review of EMG-, FMG-, and EIT-based biosensors and relevant human-machine interactivities and biomedical applications. Biosensors, 2022, 12(7): 516 doi: 10.3390/bios12070516
[27]	张鋆豪, 何百岳, 杨旭升, 张文安. 基于可穿戴式惯性传感器的人体运动跟踪方法综述. 自动化学报, 2019, 45(8): 1439-1454 Zhang Jun-Hao, He Bai-Yue, Yang Xu-Sheng, Zhang Wen-An. A review on wearable inertial sensor based human motion tracking. Acta Automatica Sinica, 2019, 45(8): 1439-1454
[28]	周穗华, 张宏欣, 冯士民. 高斯渐进贝叶斯滤波器. 控制理论与应用, 2015, 32(8): 1023-1031 Zhou Sui-Hua, Zhang Hong-Xin, Feng Shi-Min. Gaussian progressive Bayesian filter. Control Theory & Applications, 2015, 32(8): 1023-1031
[29]	郑婷婷, 杨旭升, 张文安, 俞立. 一种高斯渐进滤波框架下的目标跟踪方法. 自动化学报, 2018, 44(12): 2250-2258 Zheng Ting-Ting, Yang Xu-Sheng, Zhang Wen-An, Yu Li. A target tracking method in Gaussian progressive filtering framework. Acta Automatica Sinica, 2018, 44(12): 2250-2258