2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

一种脑肢融合的神经康复训练在线评价与调整方法

舒智林 李思宜 于宁波 朱志中 巫嘉陵 韩建达

舒智林, 李思宜, 于宁波, 朱志中, 巫嘉陵, 韩建达. 一种脑肢融合的神经康复训练在线评价与调整方法. 自动化学报, 2022, 48(5): 1209−1219 doi: 10.16383/j.aas.c200452
引用本文: 舒智林, 李思宜, 于宁波, 朱志中, 巫嘉陵, 韩建达. 一种脑肢融合的神经康复训练在线评价与调整方法. 自动化学报, 2022, 48(5): 1209−1219 doi: 10.16383/j.aas.c200452
Shu Zhi-Lin, Li Si-Yi, Yu Ning-Bo, Zhu Zhi-Zhong, Wu Jia-Ling, Han Jian-Da. A brain-limb fusion approach to online assessment and adjustment of rehabilitation trainings. Acta Automatica Sinica, 2022, 48(5): 1209−1219 doi: 10.16383/j.aas.c200452
Citation: Shu Zhi-Lin, Li Si-Yi, Yu Ning-Bo, Zhu Zhi-Zhong, Wu Jia-Ling, Han Jian-Da. A brain-limb fusion approach to online assessment and adjustment of rehabilitation trainings. Acta Automatica Sinica, 2022, 48(5): 1209−1219 doi: 10.16383/j.aas.c200452

一种脑肢融合的神经康复训练在线评价与调整方法

doi: 10.16383/j.aas.c200452
基金项目: 国家自然科学基金(61720106012, U1913208, 61873135)资助
详细信息
    作者简介:

    舒智林:南开大学人工智能学院博士研究生. 主要研究方向为上肢神经康复, 脑功能网络. E-mail: shuzhilin2017@outlook.com

    李思宜:南开大学人工智能学院硕士研究生. 主要研究方向为康复和辅助机器人. E-mail: lisiyitt@outlook.com

    于宁波:南开大学人工智能学院教授. 主要研究方向为医疗康复机器人, 医疗人工智能. 本文通信作者. E-mail: nyu@nankai.edu.cn

    朱志中:天津市环湖医院康复医学科副主任医师. 主要研究方向为脑血管病和帕金森病的康复治疗及评估. E-mail: zhu36121209@sina.com

    巫嘉陵:天津市环湖医院神经内科和康复医学科主任医师. 主要研究方向为人工智能在神经系统疾病上的应用. E-mail: wywjl2009@hotmail.com

    韩建达:南开大学杰出教授. 主要研究方向为机器人自主行为与人机协作/共融方法, 医疗康复机器人, 地面移动及飞行机器人技术与系统. E-mail: hanjianda@nankai.edu.cn

A Brain-limb Fusion Approach to Online Assessment and Adjustment of Rehabilitation Trainings

Funds: Supported by National Natural Science Foundation of China (61720106012, U1913208, 61873135)
More Information
    Author Bio:

    SHU Zhi-Lin Ph.D. candidate at the College of Artificial Intelligence, Nankai University. His research interest covers upper limb neurorehabilitation and brain functional network

    LI Si-Yi Master student at the College of Artificial Intelligence, Nankai University. Her main research interest is rehabilitation and assistive robots

    YU Ning-Bo Professor at the College of Artificial Intelligence, Nankai University. His research interest covers medical and rehabilitation robotics, and medical artificial intelligence. Corresponding author of this paper

    ZHU Zhi-Zhong Associate senior doctor in the Department of Rehabilitation Medicine, Tianjin Huanhu Hospital. His main research interest is cerebrovascular disease and rehabilitation treatment and evaluation of Parkinson's disease

    WU Jia-Ling Senior doctor in the Department of Neurology and Department of Rehabilitation Medicine, Tianjin Huanhu Hospital. His main research interest is the application of artificial intelligence in nervous system diseases

    HAN Jian-Da Outstanding professor at Nankai University. His research interest covers robot autonomy and human-robot coordination, medical and rehabilitation robotics, mobile and flying robotics

  • 摘要: 在神经康复训练中, 保持患者积极主动参与、提供适配其运动能力的训练难度, 对于取得良好的康复效果至关重要. 针对患者在长期康复训练过程中容易懈怠甚至出现惰性效应、运动能力有波动等挑战, 系统提出了一种脑肢融合的神经康复训练在线评价与调整方法. 首先, 从脑、肢体以及训练任务三个层面, 基于脑电(Electroencephalography, EEG)信号、肢体运动数据和任务评分, 建立了对患者神经参与程度、运动控制能力和任务完成情况的量化评价方法. 进而, 在任务操作难度、辅助或干扰力场以及视觉辅助等方面, 设计了康复训练任务内和任务间的在线调整方法. 通过一个针对手功能康复的灵巧操作任务, 实现了基于所提出的脑肢融合在线评价与调整方法的闭环神经康复训练. 开展实验, 招募16名受试者参加, 对比分析开环训练和闭环训练两种情况下的实验结果, 验证了所提出方法的可行性和有效性. 该工作可推广应用到脑功能障碍患者的运动康复训练, 进一步提高康复效果.
  • 图  1  系统整体设计示意图

    Fig.  1  Illustration of the overall system design

    图  2  操作任务虚拟场景和辅助视窗

    Fig.  2  The manipulation task scenario with visual assistance

    图  3  参与度指标计算流程

    Fig.  3  The calculation process of the engagement index

    图  4  康复训练在线评价与调整流程图

    Fig.  4  The flowchart for online assessment and adjustment of the rehabilitation training

    图  5  任务级调整策略

    Fig.  5  Inter-task adjustment strategy

    图  6  任务内调整策略和任务等级设置

    Fig.  6  Intra-task adjustment strategy and task level setting

    图  7  实验流程. 开环实验中, 操作任务参数确定、任务序列以伪随机方式生成; 闭环实验中, 操作任务的设置根据在线测评结果动态调整

    Fig.  7  The experimental procedures. The task configuration was fixed and the task sequence was pseudo-randomly produced in open-loop experiments, while adapted online based on assessment results in closed-loop experiments

    图  8  参数确定的“开环”实验结果((a) 神经参与程度; (b) 归一化急动度)

    Fig.  8  Results of the open-loop experiment with fixed task configuration ((a) neural engagement; (b) normalized jerk)

    图  9  闭环实验训练任务中虚拟轨迹等级变化

    Fig.  9  Changes of virtual track levels in training trials of closed-loop experiments

    图  10  单次训练任务的完成情况

    Fig.  10  Completion of a single trial

    图  11  开环和闭环实验指标对比((a)神经参与程度; (b)归一化急动度; (c)任务分数)

    Fig.  11  Comparison of indexes in the open-loop and closed-loop experiments ((a) neural engagement; (b) normalized jerk; (c) task scores)

    表  1  大脑皮层位置分区及功能表格[24]

    Table  1  The regions and functions of the cerebral cortex[24]

    字母大脑皮层位置大脑皮层不同空间位置主要功能
    F额叶高级认知功能和自主运动的控制
    T颞叶听觉, 嗅觉, 高级视觉功能, 分辨左右, 长期记忆
    C中部奖励学习和情感处理
    P顶叶躯体感觉, 空间信息处理,
    视觉信息和体感信息的整合
    O枕叶视觉处理
    下载: 导出CSV

    表  2  辅助/干扰力等级设置

    Table  2  Assistance/disturbance force level settings

    辅助/干扰力等级具体情况
    等级 1有辅助力, 无干扰力
    等级 2无辅助力, 无干扰力
    等级 3无辅助力, 有干扰力
    下载: 导出CSV

    表  3  视觉辅助等级设置

    Table  3  Visual assistance level settings

    视觉辅助等级具体情况
    等级 1提供局部放大场景
    等级 2不提供局部放大场景
    下载: 导出CSV

    表  4  虚拟轨迹等级设置

    Table  4  Virtual track level setting

    虚拟轨迹等级角度变化 (rad)
    等级 16.5
    等级 211.0
    等级 319.3
    等级 X13.5
    下载: 导出CSV

    表  5  脑波频段对应的人体状态[28-29]

    Table  5  Brainwave bands and corresponding human states[28-29]

    脑波频段人体状态
    $\delta $0.1 ~ 4 Hz深度睡眠且没有做梦时
    $\theta $4 ~ 8 Hz与深度放松状态相关
    $\alpha $8 ~ 12 Hz与放松、平静和闭眼睛的状态相关
    $\beta $12 ~ 25 Hz与注意力和快速活动相关
    下载: 导出CSV
  • [1] 中华医学会神经病学分会神经康复学组, 中华医学会神经病学分会脑血管病学组, 卫生部脑卒中筛查与防治工程委员会办公室, 中国脑卒中康复治疗指南(2011完全版). 中国康复理论与实践, 2012, 18(04): 301-318. doi: 10.3969/j.issn.1006-9771.2012.04.001

    Neurological rehabilitation group, neurology branch, Chinese Medical Association. Cerebrovascular disease group, neurology branch, Chinese Medical Association. Office of stroke screening and Prevention Engineering Committee, Ministry of Health. Guidelines for rehabilitation of stroke in China (2011 full edition). Chinese Journal of Rehabilitation Theory and Practice, 2012, 18(04): 301-318(in Chinese). doi: 10.3969/j.issn.1006-9771.2012.04.001
    [2] Wang W, Jiang B, Sun H, et al. Prevalence, incidence, and mortality of stroke in China: results from a nationwide population-based survey of 480 687 adults. Circulation, 2017, 135(8): 759-771. doi: 10.1161/CIRCULATIONAHA.116.025250
    [3] Schaechter J D. Motor rehabilitation and brain plasticity after hemiparetic stroke. Progress in Neurobiology, 2004, 73(1): 61-72. doi: 10.1016/j.pneurobio.2004.04.001
    [4] Takahashi C D, Der-Yeghiaian L, Le V, Motiwala R R, Cramer S C. Robot-based hand motor therapy after stroke. Brain, 2008, 131(2): 425-437. doi: 10.1093/brain/awm311
    [5] Oña E D, Garcia-Haro J M, Jardón A, Balaguer C. Robotics in health care: Perspectives of robot-aided interventions in clinical practice for rehabilitation of upper limbs. Applied sciences, 2019, 9(13): Article No. 2586. doi: 10.3390/app9132586
    [6] Maciejasz P, Eschweiler J, Gerlach-Hahn K, Jansen-Troy A, Leonhardt S. A survey on robotic devices for upper limb rehabilitation. Journal of Neuroengineering and Rehabilitation, 2014, 11(1): Article No. 3 doi: 10.1186/1743-0003-11-3
    [7] 侯增广, 赵新刚, 程龙, 王启宁, 王卫群. 康复机器人与智能辅助系统的研究进展. 自动化学报, 2016, 42(12): 1765-1779.

    Hou Zeng-Guang, Zhao Xin-Gang, Cheng Long, Wang Qi-Ning, Wang Wei-Qun. Recent advances in rehabilitation robots and intelligent assistance systems. Acta Automatica Sinica, 2016, 42(12): 1765−1779(in Chinese).
    [8] Marchal-Crespo L, Reinkensmeyer D J. Review of control strategies for robotic movement training after neurologic injury. Journal of neuroengineering and rehabilitation, 2009, 6(1): 1-15. doi: 10.1186/1743-0003-6-1
    [9] Reinkensmeyer D J, Wolbrecht E T, Chan V, Chou C, Cramer S C, Bobrow J E. Comparison of 3D, assist-as-needed robotic arm/hand movement training provided with Pneu-WREX to conventional table top therapy following chronic stroke. American journal of physical medicine & rehabilitation/Association of Academic Physiatrists, 2012, 91(11 0 3): S232.
    [10] Iandolo R, Marini F, Semprini M, et al. Perspectives and challenges in robotic neurorehabilitation. Applied Sciences, 2019, 9(15): Article No. 3183. doi: 10.3390/app9153183
    [11] 彭亮, 侯增广, 王晨, 罗林聪, 王卫群. 康复辅助机器人及其物理人机交互方法. 自动化学报, 2018, 44(11): 2000-2010.

    Peng Liang, Hou Zeng-Guang, Wang Chen, Luo Lin-Cong, Wang Wei-Qun. Physical interaction methods for rehabilitation and assistive robots. Acta Automatica Sinica, 2018, 44(11): 2000−2010(in Chinese).
    [12] Klamroth-Marganska V, Blanco J, Campen K, et al. Three-dimensional, task-specific robot therapy of the arm after stroke: a multicentre, parallel-group randomised trial. The Lancet Neurology, 2014, 13(2): 159-166. doi: 10.1016/S1474-4422(13)70305-3
    [13] Patton J, Small S L, Rymer W Z. Functional restoration for the stroke survivor: informing the efforts of engineers. Topics in Stroke Rehabilitation, 2008, 15(6): 521-541. doi: 10.1310/tsr1506-521
    [14] Blank A A, French J A, Pehlivan A U, O’Malley M K. Current trends in robot-assisted upper-limb stroke rehabilitation: promoting patient engagement in therapy. Current Physical Medicine and Rehabilitation Reports, 2014, 2(3): 184-195. doi: 10.1007/s40141-014-0056-z
    [15] Metzger J C, Lambercy O, Califfi A, et al. Assessment-driven selection and adaptation of exercise difficulty in robot-assisted therapy: A pilot study with a hand rehabilitation robot. Journal of Neuroengineering and Rehabilitation, 2014, 11(1): Article No. 154
    [16] Choi, Y., Gordon, J., Park, H, Schweighofer N. Feasibility of the adaptive and automatic presentation of tasks system for rehabilitation of upper extremity function post-stroke. Journal of NeuroEngineering and Rehabilitation, 2011, 8(1): 1-12. doi: 10.1186/1743-0003-8-1
    [17] Li C, Rusak Z, Horvath I, Kooijman A, Ji L. Implementation and validation of engagement monitoring in an engagement enhancing rehabilitation system. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2016, 25(6): 726-738.
    [18] Wang J X, Wang W Q, Hou Z G, Liang X, Ren S X, Peng L. Towards enhancement of patients' engagement: Online modification of rehabilitation training modes using facial expression and muscle fatigue. In: Proceedings of the 2018 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Hawaii, USA: IEEE, 2018. 2304−2307
    [19] Xu G, Gao X, Pan L, et al. Anxiety detection and training task adaptation in robot-assisted active stroke rehabilitation. International Journal of Advanced Robotic Systems, 2018, 15(6): 1-18.
    [20] Min B K, Marzelli M J, Yoo S S. Neuroimaging-based approaches in the brain–computer interface. Trends in Biotechnology, 2010, 28(11): 552-560. doi: 10.1016/j.tibtech.2010.08.002
    [21] Li S Y, Shu Z L, Sun N, Wang K, Xu C, Yu N B. Online assessment of neural engagement and functional performance for a fine motor control task. In: Proceedings of the 2018 International Conference on Advanced Robotics and Mechatronics. Singapore: IEEE, 2018. 392−396
    [22] Chanel G, Rebetez C, Bétrancourt M, Pun T. Emotion assessment from physiological signals for adaptation of game difficulty. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 2011, 41(6): 1052-1063. doi: 10.1109/TSMCA.2011.2116000
    [23] 于宁波, 李思宜, 赵营泉, 王奎, 徐昌. 基于共享控制的人机灵巧力触觉交互系统设计与实现. 仪器仪表学报, 2017, 38(3): 602-611. doi: 10.3969/j.issn.0254-3087.2017.03.012

    Yu Ning-Bo, Li Si-yi, Zhao Ying-quan, Wang Kui, Xu Chang, Design and implementation of a dexterous human-robot interaction system based on haptic shared control. Chinese Journal of Scientific Instrument, 2017, 38(3): 602-611(in Chinese). doi: 10.3969/j.issn.0254-3087.2017.03.012
    [24] Woolsey T A, Hanaway J, Gado M H. The Brain Atlas: A Visual Guide to the Human Central Nervous System. Hoboken: John Wiley & Sons, 2017.
    [25] Guidali M, Duschau-Wicke A, Broggi S, Klamroth-Marganska V, Nef T, Riener R. A robotic system to train activities of daily living in a virtual environment. Medical & biological engineering & computing, 2011, 49(10): 1213-1223.
    [26] Sigrist R, Rauter G, Marchal-Crespo L, Riener R, Wolf P. Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Experimental brain research, 2015, 233(3): 909-925. doi: 10.1007/s00221-014-4167-7
    [27] Patton J L, Wei Y J, Bajaj P, Scheidt R A. Visuomotor learning enhanced by augmenting instantaneous trajectory error feedback during reaching. PloS One, 2013, 8(1): Article No. e46466
    [28] Abhang P A, Gawali B W, Mehrotra S C. Technological basics of EEG recording and operation of apparatus. Introduction to EEG-and Speech-based Emotion Recognition, 2016: 19-50.
    [29] Newson J J, Thiagarajan T C. EEG frequency bands in psychiatric disorders: A review of resting state studies. Frontiers in Human Neuroscience, 2019, 12: Article No. 521
    [30] Pope A T, Bogart E H, Bartolome D S. Biocybernetic system evaluates indices of operator engagement in automated task. Biological Psychology, 1995, 40(1-2): 187-195. doi: 10.1016/0301-0511(95)05116-3
    [31] Nuamah J K, Seong Y. Support vector machine classification of cognitive tasks based on electroencephalography engagement index. Brain-Computer Interfaces, 2018, 5(1): 1-12. doi: 10.1080/2326263X.2017.1338012
    [32] Al-Hudhud G, Alqahtani L, Albaity H, Alsaeed D, Al-Turaiki I. Analyzing passive BCI signals to control adaptive automation devices. Sensors, 2019, 19(14): Article No. 3042
    [33] Kosmyna N, Maes P. AttentivU: An EEG-based closed-loop biofeedback system for real-time monitoring and improvement of engagement for personalized learning. Sensors, 2019, 19(23): Article No. 5200
    [34] Zollo L, Gallotta E, Guglielmelli E, Sterzi S. Robotic technologies and rehabilitation: new tools for upper-limb therapy and assessment in chronic stroke. European Journal of Physical and Rehabilitation Medicine, 2011, 47(2): 223-236.
    [35] Baron E I, Koop M M, Streicher M C, Rosenfeldt A B, Alberts J L. Altered kinematics of arm swing in Parkinson's disease patients indicates declines in gait under dual-task conditions. Parkinsonism & related disorders, 2018, 48: 61-67.
    [36] Mazzoleni S, Tran V D, Dario P, Posteraro F. Wrist robot-assisted rehabilitation treatment in subacute and chronic stroke patients: from distal-to-proximal motor recovery. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2018, 26(9): 1889-1896. doi: 10.1109/TNSRE.2018.2864935
    [37] Cohen J, Cohen P, West S G, Aiken L S. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. London: Routledge, 2013.
    [38] Zhang J, Fiers P, Witte K A, et al. Human-in-the-loop optimization of exoskeleton assistance during walking. Science, 2017, 356(6344): 1280-1284. doi: 10.1126/science.aal5054
    [39] Maurice P, Huber M E, Hogan N, Sternad D. Velocity-curvature patterns limit human-robot physical interaction. IEEE Robotics & Automation Letters, 2018, 3(1): 249-256.
    [40] Reinkensmeyer D J, Burdet E, Casadio M, et al. Computational neurorehabilitation: modeling plasticity and learning to predict recovery. Journal of neuroengineering and rehabilitation, 2016, 13(1): 1-25. doi: 10.1186/s12984-015-0109-2
  • 加载中
图(11) / 表(5)
计量
  • 文章访问数:  1167
  • HTML全文浏览量:  318
  • PDF下载量:  184
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-06-23
  • 录用日期:  2021-04-02
  • 网络出版日期:  2021-05-26
  • 刊出日期:  2022-05-13

目录

    /

    返回文章
    返回