2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于气压肌动图和改进神经模糊推理系统的手势识别研究

汪雷 黄剑 段涛 伍冬睿 熊蔡华 崔雨琦

汪雷, 黄剑, 段涛, 伍冬睿, 熊蔡华, 崔雨琦. 基于气压肌动图和改进神经模糊推理系统的手势识别研究. 自动化学报, 2021, 47(x): 1−13 doi: 10.16383/j.aas.c200901
引用本文: 汪雷, 黄剑, 段涛, 伍冬睿, 熊蔡华, 崔雨琦. 基于气压肌动图和改进神经模糊推理系统的手势识别研究. 自动化学报, 2021, 47(x): 1−13 doi: 10.16383/j.aas.c200901
Wang Lei, Huang Jian, Duan Tao, Wu Dong-Rui, Xiong Cai-Hua, Cui Yu-Qi. Research on gesture recognition based on pressure-based mechanomyogram and improved neural fuzzy inference system. Acta Automatica Sinica, 2021, 47(x): 1−13 doi: 10.16383/j.aas.c200901
Citation: Wang Lei, Huang Jian, Duan Tao, Wu Dong-Rui, Xiong Cai-Hua, Cui Yu-Qi. Research on gesture recognition based on pressure-based mechanomyogram and improved neural fuzzy inference system. Acta Automatica Sinica, 2021, 47(x): 1−13 doi: 10.16383/j.aas.c200901

基于气压肌动图和改进神经模糊推理系统的手势识别研究

doi: 10.16383/j.aas.c200901
基金项目: 国家自然科学基金联合基金重点支持项目(U1913207), 湖北省技术创新专项(2019AEA171), 科技部政府间国际科技创新合作重点专项(2017YFE0128300)资助
详细信息
    作者简介:

    汪雷:华中科技大学人工智能与自动化学院硕士研究生. 2019年获得华中科技大学自动化学院学士学位. 主要研究方向为机器学习, 手势识别. E-mail: wml0531@hust.edu.cn

    黄剑:华中科技大学人工智能与自动化学院教授. 1997年毕业于中国华中科技大学. 2000年获得华中科技大学工程硕士学位. 2005年获得华中科技大学博士学位. 2006年至2008年, 在日本名古屋大学微纳米系统工程系、机械信息与系统系担任博士后研究员. 主要研究方向为康复机器人、机器人装配、网络控制系统和生物信息学. 本文通信作者. E-mail: huang_jan@mail.hust.edu.cn

    段涛:华中科技大学人工智能与自动化学院硕士研究生. 2020年获得华中科技大学人工智能与自动化学院硕士学位. 主要研究方向为智能机器人, 模式识别. E-mail: tao_duan@hust.edu.cn

    伍冬睿:华中科技大学人工智能与自动化学院教授. 主要研究方向为机器学习, 脑机接口, 计算智能, 情感计算. E-mail: drwu@hust.edu.cn

    熊蔡华:华中科技大学机械科学与工程学院数字制造装备与技术国家重点实验室教授. 1998年毕业于于华中理工大学(现华中科技大学)机械电子工程专业获博士学位. 主要研究方向为为机器人学, 生机电一体化, 康复工程装备. E-mail: chxiong@hust.edu.cn

    崔雨琦:华中科技大学人工智能与自动化学院博士研究生. 2017年获得华中科技大学电子信息工程学士学位. 主要研究方向为模糊系统, 脑机接口, 可穿戴设备. E-mail: yqcui@hust.edu.cn

Research on Gesture Recognition Based on Pressure-based Mechanomyogram and Improved Neural Fuzzy Inference System

Funds: Supported by National Natural Science Foundation of China (61873321, U1913207), Technology Innovation Project of Hubei Province of China (2019AEA171),and International Science and Technology Cooperation Program of China (2017YFE0128300)
More Information
    Author Bio:

    WANG Lei M. St. candidate at the School of Artificial Intelligence and Automation, Huazhong University of Science and Technology. He received his bachelor degree from Huazhong University of Science and Technology in 2019. His research interest covers machine learning and gesture recognition

    HUANG Jian Full professor with the School of Artificial Intelligence and Automation, Huazhong University of Science and Technology (HUST). He graduated from HUST, China in 1997 and received the Master of Engineering degree from HUST in 2000. He received his Ph.D. from HUST in 2005. From 2006 to 2008, he was a postdoctoral researcher in the Department of Micro-Nano System Engineering and Department of Mechano-Informatics and Systems, Nagoya University, Japan. His main research interests include rehabilitation robot, robotic assembly, networked control systems and bioinformatics. Corresponding author of this paper

    DUAN Tao M. St. with the School of Artificial Intelligence and Automation, Huazhong University of Science and Technology. He received his master degree from Huazhong University of Science and Technology in 2020. His research interest covers intelligent robot and pattern recognition

    WU Dong-Rui Professor at the School of Artificial Intelligence and Automation, Huazhong University of Science and Technology. His research interest covers machine learning, brain-computer interfaces, computational intelligence, affective computing

    XIONG Cai-Hua Professor at the State Key Laboratory of Manufacturing Equipment and Technology, School of Mechanical Science and Engineering, Huazhong University of Science and Technology. He received his Ph. D. degree in mechatronics from Huazhong University of Science and Technology in 1998. His research interest covers robotics, biomechatronics and rehabilitation robot

    Cui Yu-Qi Professor at the School of Artificial Intelligence and Automation, Huazhong University of Science and Technology. He received his bachelor degree in electronic information engineering from Huazhong University of Science and Technology in 2017. His research interest covers fuzzy systems, brain-computer interfaces, wearable devices

  • 摘要: 手势识别是人机交互领域的重要研究内容, 为截肢患者控制智能假肢手提供基础. 当前主流方法之一是利用表面肌电图识别手部运动意图, 但肌电信号存在信号弱、易受噪声、汗液、疲劳影响等缺点. 同时肌电图在识别准确率方面, 尤其是截肢患者手势识别方面仍然具有较大的提升空间. 针对这些问题, 本文设计了基于气压肌动图的穿戴式信号采集装置, 为手势识别提供了优质的信号源. 结合深度神经网络中全连接层结构、典型抽样和标准正则化技术, 提出了一种改进多类神经模糊推理系统, 与传统神经模糊推理系统相比泛化能力得到显著提升. 我们招募了7名健康受试者和1名截肢受试者, 并用八种算法开展离线实验. 所提方法在残疾人手势识别实验中取得了97.25%的最高平均准确率, 在健康人手势识别实验中取得了98.18%的最高平均准确率. 与近年公开报道的多种手势识别研究相比, 本文所提方法的综合性能更优.
  • 图  1  pMMG原理图

    Fig.  1  The schematic diagram of pMMG

    图  2  手势识别装置及其佩戴位置

    Fig.  2  The gesture recognition device and it's wearing position

    图  3  每一轮的手势数据采集过程

    Fig.  3  Each round of gesture data acquisition process

    图  4  采集的原始手势数据流

    Fig.  4  The collected raw gesture data stream

    图  5  手势数据处理过程

    Fig.  5  Gesture data processing

    图  6  MC_ANFIS结构图

    Fig.  6  Structure of MC_ANFIS

    图  7  本文研究的六种手势

    Fig.  7  Six gestures studied in this paper

    图  8  MC_TS_UR融合算法在每名受试者数据集上的混淆矩阵

    Fig.  8  The CM of the MC_TS_UR fusion algorithm applied to the datasets of every subject

    图  9  四种基于MC_ANFIS的算法在训练过程中的分类误差随时间变化曲线

    Fig.  9  The classification error changes curve of four MC_ANFIS based algorithms with time during the training process

    表  1  参与手势识别实验的受试者信息

    Table  1  Information of the subjects participating in the gesture recognition experiment

    受试者性别年龄身高 (cm)体重 (kg)腕围 (cm)健康状况
    Subject-125180.472.418.8健康
    Subject-224169.558.516.5健康
    Subject-356164.661.215.8手部截肢
    Subject-425172.362.817.9健康
    Subject-522177.557.016.8健康
    Subject-626166.665.718.4健康
    Subject-723170.173.319.1健康
    Subject-825175.566.917.1健康
    下载: 导出CSV

    表  2  六种手腕手势对应的肌肉信息

    Table  2  Muscles information of the corresponding six gestures

    手势肌肉作用
    Wrist flexion尺侧腕屈肌手腕屈曲和尺侧偏移
    Hand close指浅屈肌手指弯曲
    Ulnar deviation桡侧腕屈肌手腕弯曲和径向偏移
    Wrist extension尺侧腕伸肌手腕伸展和尺侧偏移
    Hand open指伸肌手指伸展
    Radial deviation桡侧腕伸肌手腕伸展和径向偏移
    下载: 导出CSV

    表  3  八种算法在健康人的数据集上离线实验结果

    Table  3  The offline experiment results of seven algorithms on datasets of the normal

    指标SVMGBDTLDATSK_GD_LSEMCMC_TSMC_URMC_TS_UR
    $RER$6.07%7.82%5.15%5.26%3.16%2.52%2.30%1.82%
    $BER$6.18%8.74%5.21%5.35%2.83%2.41%2.33%1.77%
    $\kappa$0.92580.90180.93750.93580.96600.97110.97200.9787
    $T_t$224.64.40.61121.9796.1886.5734.7310.2
    下载: 导出CSV

    表  4  八种算法在残疾人数据集上的离线实验结果

    Table  4  The offline experiment results of seven algorithms on datasets of the disabled

    指标SVMGBDTLDATSK_GD_LSEMCMC_TSMC_URMC_TS_UR
    $RER$5.94%8.13%4.46%5.77%4.64%3.83%3.77%2.75%
    $BER$6.10%8.27%4.48%6.11%4.72%3.98%3.65%2.73%
    $\kappa$0.92680.90080.94620.92670.94340.95220.95620.9672
    $T_t$173.05.30.71006.5766.8942.9768.6313.1
    下载: 导出CSV

    表  5  与近期同类研究工作文献的比较

    Table  5  Comparison with similar research work literature

    作者传感器实验对象是否为公共数据集分类算法手势类别数识别准确率
    Jung et al.[25]6通道pMMG6名健康人Fuzzy Logic695.30%
    Mohammad et al.[26]8通道FMG10名健康人SVM693.00%
    Tavakoli et al.[27]2通道sEMG7名健康人SVM495.00%
    Jiang et al.[28]4通道sEMG+1通道IMU10名健康人Linear Discriminant Analysis892.60%
    Botros et al.[29]8通道sEMG21名健康人Linear Discriminant Analysis694.70%
    Shaabana et al.[30]8通道sEMG8名健康人Hidden Markov Model694.20%
    Proposed6通道pMMG+1通道IMU7名健康人MC_TS_UR698.18%
    Powell et al.[31]8通道sEMG4名残疾人Linear Discriminant Analysis792.00%
    Kartsch et al.[32]7通道sEMG3名残疾人SVM594.02%
    Proposed6通道pMMG+1通道IMU1名残疾人MC_TS_UR697.25%
    下载: 导出CSV
  • [1] Liu H Y, Wang L H. Gesture recognition for human-robot collaboration: A review. International Journal of Industrial Ergonomics, 2018, 68(1): 355−367
    [2] Ding J, Lin R Z, Lin Z Y. Service robot system with integration of wearable Myo armband for specialized hand gesture human–computer interfaces for people with disabilities with mobility problems. Computers Electrical Engineering, 2018, 69(1): 815−827
    [3] 丁其川, 赵新刚, 李自由, 韩建达. 基于自更新混合分类模型的肌电运动识别方法. 自动化学报, 2019, 45(8): 1464−1474

    Ding Q C, Zhao X G, Li Z Y, Han J D. An EMG-motion recognition method with self-update hybrid classification model. Acta Automatica Sinica, 2019, 45(8): 1464−1474
    [4] Zhang X, Chen X, Li Y, Lantz V, Wang K, Yang J. A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 2011, 41(6): 1064−1076 doi: 10.1109/TSMCA.2011.2116004
    [5] Duan T, Huang J, Xie Z, Wang L, Xiong C H. Continuous Control of Wrist-Hand Prosthesis by Extracting Independent sEMG Signals from Cross-talk Muscle Groups. In: 2019 Chinese Control Conference. Guangzhou, China: IEEE 2019. 4537−4542
    [6] 李自由, 王丰焱, 赵新刚, 丁其川, 张道辉, 韩建达. 基于Myo旋转偏移估计与自适应校正的手势识别方法. 自动化学报, 2020, 46(9): 1896−1907

    Li Z Y, Wang F Y, Zhao X G, Ding Q C, Zhang D H, Han J D. The method for gestures recognition based on Myo rotation shifts estimation and adaptive correction. Acta Automatica Sinica, 2020, 46(9): 1896−1907
    [7] Orizio C, Liberati D, Locatelli C, et al. Surface mechanomyogram reflects muscle fibres twitches summation. Journal of Biomechanics, 1996, 29(4): 475−481 doi: 10.1016/0021-9290(95)00063-1
    [8] Akataki K, Mita K, Watakabe M, Itoh K. Mechanomyogram and force relationship during voluntary isometric ramp contractions of the biceps brachii muscle. European Journal of Applied Physiology, 2001, 84(1-2): 19−25 doi: 10.1007/s004210000321
    [9] McIntosh J, Marzo A, Fraser M, Phillips C. Echoflex: Hand gesture recognition using ultrasound imaging. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Denver, USA: ACM 2017. 1923−1934
    [10] Liu M K, Lin Y T, Qiu Z W, Kuo C K, Wu C K. Hand Gesture Recognition by a MMG-Based Wearable Device. Journal of The Neurological Sciences, 2020, 20(24): 14703−14712
    [11] Feng W, Xia C, Zhang Y, Yu J, Jiang W, Research on Chinese Sign Language Recognition Methods Based on Mechanomyogram Signals Analysis. In: 2019 IEEE 4th International Conference on Signal and Image Processing. Wuxi, China: IEEE 2019. 46−50
    [12] Stokes M J, Dalton P A. Acoustic myographic activity increases linearly up to maximal voluntary isometric force in the human quadriceps muscle. Journal of The Neurological Sciences, 1991, 101(2): 163−167 doi: 10.1016/0022-510X(91)90041-5
    [13] Ahsan M R, Ibrahimy M I, Khalifa O O. Electromygraphy (EMG) signal based hand gesture recognition using artificial neural network (ANN). In 2011 4th International Conference on Mechatronics. Kuala Lumpur, Malaysia: IEEE 2011: 1−6
    [14] Zhang Z, Yang K, Qian J, Zhang L. Real-time surface emg pattern recognition for hand gestures based on an artificial neural network. Sensors, 2019, 19(14): 3170−3184 doi: 10.3390/s19143170
    [15] Yao B, Hagras H, Alhaddad M J, Alghazzawi D. A fuzzy logic-based system for the automation of human behavior recognition using machine vision in intelligent environments. Soft Computing, 2015, 19(2): 499−506 doi: 10.1007/s00500-014-1270-4
    [16] Hachaj T, Ogiela M R. Rule-based approach to recognizing human body poses and gestures in real time. Multimedia Systems, 2014, 20(1): 81−99 doi: 10.1007/s00530-013-0332-2
    [17] Mufarroha F A, Utaminingrum F. Hand gesture recognition using adaptive network based fuzzy inference system and k-nearest neighbor. International Journal of Technology, 2017, 8(3): 559−567 doi: 10.14716/ijtech.v8i3.3146
    [18] Khezri M, Jahed M. A neuro–fuzzy inference system for sEMG-based identification of hand motion commands. IEEE Transactions on Industrial Electronics, 2010, 58(5): 1952−60
    [19] Hill A V. The Heat of Shortening and the Dynamic Constants of Muscle. Proceedings of the Royal Society of London, 1938, 126(843): 136−195
    [20] Belyea A, Englehart K, Scheme E. FMG Versus EMG: A comparison of usability for real-time pattern recognition based control. IEEE Transactions on Biomedical Engineering, 2019, 66(11): 3098−3104 doi: 10.1109/TBME.2019.2900415
    [21] Peng X Y, Li L, Wang F Y. Accelerating minibatch stochastic gradient descent using typicality sampling. IEEE Transactions on Neural Networks and Learning Systems, 2019, 31(11): 4649−4659
    [22] Cui Y Q, Wu D R, Huang J. Optimize TSK fuzzy systems for classification problems: Mini-batch gradient descent with uniform regularization and batch normalization. IEEE Transactions on Fuzzy Systems, 2020, 28(12): 3065−3075 doi: 10.1109/TFUZZ.2020.2967282
    [23] You H, Ma Z, Tang Y, Wang Y, Yan J, Ni M, Cen K, Huang Q. Comparison of ANN (MLP), ANFIS, SVM, and RF models for the online classification of heating value of burning municipal solid waste in circulating fluidized bed incinerators. Waste Management, 2017, 68(10): 186−197
    [24] Shariati S, Haghighi M M. Comparison of anfis Neural Network with several other ANNs and Support Vector Machine for diagnosing hepatitis and thyroid diseases. In: 2010 International Conference on Computer Information Systems and Industrial Management Applications. Krackow, Poland: IEEE 2010. 596−599
    [25] Jung P G, Lim G, Kim S, Kong K. A wearable gesture recognition device for detecting muscular activities based on air-pressure sensors. IEEE Transactions on Industrial Informatics, 2015, 11(2): 485−494
    [26] Anvaripour M, Saif M. February. Hand gesture recognition using force myography of the forearm activities and optimized features. In: 2018 IEEE International Conference on Industrial Technology. Lyon, France: IEEE 2018. 187−192
    [27] Tavakoli M, Benussi C, Lopes P A, et al. Robust hand gesture recognition with a double channel surface EMG wearable armband and SVM classifier. Biomedical Signal Processing and Control, 2018, 46(1): 121−130
    [28] Jiang S, Lv B, Guo W, Zhang C, Wang H, Sheng X, Shull P B. Feasibility of wrist-worn, real-time hand, and surface gesture recognition via sEMG and IMU Sensing. IEEE Transactions on Industrial Informatics, 2017, 14(8): 3376−3385
    [29] Botros F, Phinyomark A, Scheme E. EMG-Based Gesture Recognition: Is It Time to Change Focus from the Forearm to the Wrist?. IEEE Transactions on Industrial Informatics, to be published
    [30] Shaabana A, Legere J, Li J, Zheng R, Mohrenschildt MV, Shedden JM. Portable electromyography: A case study on ballistic finger movement recognition. IEEE Sensors Journal, 2019, 19(16): 7043−55 doi: 10.1109/JSEN.2019.2908312
    [31] Powell M A, Kaliki R R, Thakor N V. User training for pattern recognition-based myoelectric prostheses: Improving phantom limb movement consistency and distinguishability. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2014, 22(3): 522−532 doi: 10.1109/TNSRE.2013.2279737
    [32] Kartsch V, Benatti S, Mancini M, et al. Smart wearable wristband for EMG based gesture recognition powered by solar energy harvester. In: 2018 IEEE International Symposium on Circuits and Systems. Florence, Italy: IEEE 2018. 1−5
  • 加载中
计量
  • 文章访问数:  173
  • HTML全文浏览量:  158
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-10-27
  • 修回日期:  2021-03-02
  • 网络出版日期:  2021-05-22

目录

    /

    返回文章
    返回