一种脑肢融合的神经康复训练在线评价与调整方法

舒智林; 李思宜; 于宁波; 朱志中; 巫嘉陵; 韩建达

doi:10.16383/j.aas.c200452

一种脑肢融合的神经康复训练在线评价与调整方法

doi: 10.16383/j.aas.c200452

舒智林^{1, 2,},
李思宜^{1, 2,},
于宁波^{1, 2,},
朱志中^3,,
巫嘉陵^3,,
韩建达^{1, 2,}

1.
南开大学人工智能学院天津 300350
2.
南开大学天津市智能机器人技术重点实验室天津 300350
3.
天津市环湖医院康复医学科天津 300350

基金项目: 国家自然科学基金(61720106012, U1913208, 61873135)资助

详细信息

作者简介:
舒智林：南开大学人工智能学院博士研究生. 主要研究方向为上肢神经康复, 脑功能网络. E-mail: shuzhilin2017@outlook.com

李思宜：南开大学人工智能学院硕士研究生. 主要研究方向为康复和辅助机器人. E-mail: lisiyitt@outlook.com

于宁波：南开大学人工智能学院教授. 主要研究方向为医疗康复机器人, 医疗人工智能. 本文通信作者. E-mail: nyu@nankai.edu.cn

朱志中：天津市环湖医院康复医学科副主任医师. 主要研究方向为脑血管病和帕金森病的康复治疗及评估. E-mail: zhu36121209@sina.com

巫嘉陵：天津市环湖医院神经内科和康复医学科主任医师. 主要研究方向为人工智能在神经系统疾病上的应用. E-mail: wywjl2009@hotmail.com

韩建达：南开大学杰出教授. 主要研究方向为机器人自主行为与人机协作/共融方法, 医疗康复机器人, 地面移动及飞行机器人技术与系统. E-mail: hanjianda@nankai.edu.cn

计量
- 文章访问数: 1301
- HTML全文浏览量: 402
- PDF下载量: 199
- 被引次数: 0
出版历程
- 收稿日期: 2020-06-23
- 录用日期: 2021-04-02
- 网络出版日期: 2021-05-26
- 刊出日期: 2022-05-13

A Brain-limb Fusion Approach to Online Assessment and Adjustment of Rehabilitation Trainings

SHU Zhi-Lin^{1, 2
,},
LI Si-Yi^{1, 2
,},
YU Ning-Bo^{1, 2
,},
ZHU Zhi-Zhong^3
,,
WU Jia-Ling^3
,,
HAN Jian-Da^{1, 2
,}

1.
College of Artificial Intelligence, Nankai University, Tianjin 300350
2.
Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin 300350
3.
Department of Rehabilitation Medicine, Tianjin Huanhu Hospital, Tianjin 300350

Funds: Supported by National Natural Science Foundation of China (61720106012, U1913208, 61873135)

More Information

Author Bio:
SHU Zhi-Lin　Ph.D. candidate at the College of Artificial Intelligence, Nankai University. His research interest covers upper limb neurorehabilitation and brain functional network

LI Si-Yi　Master student at the College of Artificial Intelligence, Nankai University. Her main research interest is rehabilitation and assistive robots

YU Ning-Bo　Professor at the College of Artificial Intelligence, Nankai University. His research interest covers medical and rehabilitation robotics, and medical artificial intelligence. Corresponding author of this paper

ZHU Zhi-Zhong　Associate senior doctor in the Department of Rehabilitation Medicine, Tianjin Huanhu Hospital. His main research interest is cerebrovascular disease and rehabilitation treatment and evaluation of Parkinson＇s disease

WU Jia-Ling　Senior doctor in the Department of Neurology and Department of Rehabilitation Medicine, Tianjin Huanhu Hospital. His main research interest is the application of artificial intelligence in nervous system diseases

HAN Jian-Da　Outstanding professor at Nankai University. His research interest covers robot autonomy and human-robot coordination, medical and rehabilitation robotics, mobile and flying robotics

摘要

摘要: 在神经康复训练中, 保持患者积极主动参与、提供适配其运动能力的训练难度, 对于取得良好的康复效果至关重要. 针对患者在长期康复训练过程中容易懈怠甚至出现惰性效应、运动能力有波动等挑战, 系统提出了一种脑肢融合的神经康复训练在线评价与调整方法. 首先, 从脑、肢体以及训练任务三个层面, 基于脑电(Electroencephalography, EEG)信号、肢体运动数据和任务评分, 建立了对患者神经参与程度、运动控制能力和任务完成情况的量化评价方法. 进而, 在任务操作难度、辅助或干扰力场以及视觉辅助等方面, 设计了康复训练任务内和任务间的在线调整方法. 通过一个针对手功能康复的灵巧操作任务, 实现了基于所提出的脑肢融合在线评价与调整方法的闭环神经康复训练. 开展实验, 招募16名受试者参加, 对比分析开环训练和闭环训练两种情况下的实验结果, 验证了所提出方法的可行性和有效性. 该工作可推广应用到脑功能障碍患者的运动康复训练, 进一步提高康复效果.
- 神经康复训练 /
- 脑肢融合 /
- 量化评价 /
- 在线调整 /
- 神经参与度
Abstract: In neurorehabilitation training, it is crucial to keep the patients actively engaged and ensure the task difficulty compatible with their movement capabilities. Nevertheless, challenges come from that patients often decrease participation due to the slacking effect and their motor function varies in the rehabilitation process. In this paper, we propose a brain-limb fusion approach to online assessment and adjustment of neurorehabilitation training. First of all, a systematic method was established to quantitatively assess the patients＇ neural engagement, motor control capabilities, and task performance based on the measured electroencephalography (EEG) signals, movement data, and task completion scores, from three aspects of brain, limb, and training task. Then, inter and intra-tasks online adjustment methods were designed to change the training, with respect to movement difficulty, assistance or disturbance force fields, and visual feedback, et al. The proposed online assessment and adjustment scheme was implemented on a dexterous manipulation task that was built for hand function rehabilitation, enabling closed-loop training. The pilot study was conducted with 16 subjects. Comparative analysis between open-loop and closed-loop experiments verified the feasibility and efficacy of the proposed scheme. This work can help neurorehabilitation of patients with brain disorder and promise further enhancement of rehabilitation induced recovery.
- Neurorehabilitation training /
- brain-limb fusion /
- quantitative assessment /
- online adjustment /
- neural engagement

HTML全文

图 1 系统整体设计示意图

Fig. 1 Illustration of the overall system design

下载: 全尺寸图片幻灯片

图 2 操作任务虚拟场景和辅助视窗

Fig. 2 The manipulation task scenario with visual assistance

下载: 全尺寸图片幻灯片

图 3 参与度指标计算流程

Fig. 3 The calculation process of the engagement index

下载: 全尺寸图片幻灯片

图 4 康复训练在线评价与调整流程图

Fig. 4 The flowchart for online assessment and adjustment of the rehabilitation training

下载: 全尺寸图片幻灯片

图 5 任务级调整策略

Fig. 5 Inter-task adjustment strategy

下载: 全尺寸图片幻灯片

图 6 任务内调整策略和任务等级设置

Fig. 6 Intra-task adjustment strategy and task level setting

下载: 全尺寸图片幻灯片

图 7 实验流程. 开环实验中, 操作任务参数确定、任务序列以伪随机方式生成; 闭环实验中, 操作任务的设置根据在线测评结果动态调整

Fig. 7 The experimental procedures. The task configuration was fixed and the task sequence was pseudo-randomly produced in open-loop experiments, while adapted online based on assessment results in closed-loop experiments

下载: 全尺寸图片幻灯片

图 8 参数确定的“开环”实验结果((a) 神经参与程度; (b) 归一化急动度)

Fig. 8 Results of the open-loop experiment with fixed task configuration ((a) neural engagement; (b) normalized jerk)

下载: 全尺寸图片幻灯片

图 9 闭环实验训练任务中虚拟轨迹等级变化

Fig. 9 Changes of virtual track levels in training trials of closed-loop experiments

下载: 全尺寸图片幻灯片

图 10 单次训练任务的完成情况

Fig. 10 Completion of a single trial

下载: 全尺寸图片幻灯片

图 11 开环和闭环实验指标对比((a)神经参与程度; (b)归一化急动度; (c)任务分数)

Fig. 11 Comparison of indexes in the open-loop and closed-loop experiments ((a) neural engagement; (b) normalized jerk; (c) task scores)

下载: 全尺寸图片幻灯片

表 1 大脑皮层位置分区及功能表格^[24]

Table 1 The regions and functions of the cerebral cortex^[24]

字母	大脑皮层位置	大脑皮层不同空间位置主要功能
F	额叶	高级认知功能和自主运动的控制
T	颞叶	听觉, 嗅觉, 高级视觉功能, 分辨左右, 长期记忆
C	中部	奖励学习和情感处理
P	顶叶	躯体感觉, 空间信息处理, 视觉信息和体感信息的整合
O	枕叶	视觉处理

下载: 导出CSV

表 2 辅助/干扰力等级设置

Table 2 Assistance/disturbance force level settings

辅助/干扰力等级	具体情况
等级 1	有辅助力, 无干扰力
等级 2	无辅助力, 无干扰力
等级 3	无辅助力, 有干扰力

下载: 导出CSV

表 3 视觉辅助等级设置

Table 3 Visual assistance level settings

视觉辅助等级	具体情况
等级 1	提供局部放大场景
等级 2	不提供局部放大场景

下载: 导出CSV

表 4 虚拟轨迹等级设置

Table 4 Virtual track level setting

虚拟轨迹等级	角度变化 (rad)
等级 1	6.5
等级 2	11.0
等级 3	19.3
等级 X	13.5

下载: 导出CSV

表 5 脑波频段对应的人体状态^[28-29]

Table 5 Brainwave bands and corresponding human states^[28-29]

脑波	频段	人体状态
$\delta $	0.1 ~ 4 Hz	深度睡眠且没有做梦时
$\theta $	4 ~ 8 Hz	与深度放松状态相关
$\alpha $	8 ~ 12 Hz	与放松、平静和闭眼睛的状态相关
$\beta $	12 ~ 25 Hz	与注意力和快速活动相关

下载: 导出CSV

参考文献(40)

[1]	中华医学会神经病学分会神经康复学组, 中华医学会神经病学分会脑血管病学组, 卫生部脑卒中筛查与防治工程委员会办公室, 中国脑卒中康复治疗指南(2011完全版). 中国康复理论与实践, 2012, 18(04): 301-318. doi: 10.3969/j.issn.1006-9771.2012.04.001 Neurological rehabilitation group, neurology branch, Chinese Medical Association. Cerebrovascular disease group, neurology branch, Chinese Medical Association. Office of stroke screening and Prevention Engineering Committee, Ministry of Health. Guidelines for rehabilitation of stroke in China (2011 full edition). Chinese Journal of Rehabilitation Theory and Practice, 2012, 18(04): 301-318(in Chinese). doi: 10.3969/j.issn.1006-9771.2012.04.001
[2]	Wang W, Jiang B, Sun H, et al. Prevalence, incidence, and mortality of stroke in China: results from a nationwide population-based survey of 480 687 adults. Circulation, 2017, 135(8): 759-771. doi: 10.1161/CIRCULATIONAHA.116.025250
[3]	Schaechter J D. Motor rehabilitation and brain plasticity after hemiparetic stroke. Progress in Neurobiology, 2004, 73(1): 61-72. doi: 10.1016/j.pneurobio.2004.04.001
[4]	Takahashi C D, Der-Yeghiaian L, Le V, Motiwala R R, Cramer S C. Robot-based hand motor therapy after stroke. Brain, 2008, 131(2): 425-437. doi: 10.1093/brain/awm311
[5]	Oña E D, Garcia-Haro J M, Jardón A, Balaguer C. Robotics in health care: Perspectives of robot-aided interventions in clinical practice for rehabilitation of upper limbs. Applied sciences, 2019, 9(13): Article No. 2586. doi: 10.3390/app9132586
[6]	Maciejasz P, Eschweiler J, Gerlach-Hahn K, Jansen-Troy A, Leonhardt S. A survey on robotic devices for upper limb rehabilitation. Journal of Neuroengineering and Rehabilitation, 2014, 11(1): Article No. 3 doi: 10.1186/1743-0003-11-3
[7]	侯增广, 赵新刚, 程龙, 王启宁, 王卫群. 康复机器人与智能辅助系统的研究进展. 自动化学报, 2016, 42(12): 1765-1779. Hou Zeng-Guang, Zhao Xin-Gang, Cheng Long, Wang Qi-Ning, Wang Wei-Qun. Recent advances in rehabilitation robots and intelligent assistance systems. Acta Automatica Sinica, 2016, 42(12): 1765−1779(in Chinese).
[8]	Marchal-Crespo L, Reinkensmeyer D J. Review of control strategies for robotic movement training after neurologic injury. Journal of neuroengineering and rehabilitation, 2009, 6(1): 1-15. doi: 10.1186/1743-0003-6-1
[9]	Reinkensmeyer D J, Wolbrecht E T, Chan V, Chou C, Cramer S C, Bobrow J E. Comparison of 3D, assist-as-needed robotic arm/hand movement training provided with Pneu-WREX to conventional table top therapy following chronic stroke. American journal of physical medicine & rehabilitation/Association of Academic Physiatrists, 2012, 91(11 0 3): S232.
[10]	Iandolo R, Marini F, Semprini M, et al. Perspectives and challenges in robotic neurorehabilitation. Applied Sciences, 2019, 9(15): Article No. 3183. doi: 10.3390/app9153183
[11]	彭亮, 侯增广, 王晨, 罗林聪, 王卫群. 康复辅助机器人及其物理人机交互方法. 自动化学报, 2018, 44(11): 2000-2010. Peng Liang, Hou Zeng-Guang, Wang Chen, Luo Lin-Cong, Wang Wei-Qun. Physical interaction methods for rehabilitation and assistive robots. Acta Automatica Sinica, 2018, 44(11): 2000−2010(in Chinese).
[12]	Klamroth-Marganska V, Blanco J, Campen K, et al. Three-dimensional, task-specific robot therapy of the arm after stroke: a multicentre, parallel-group randomised trial. The Lancet Neurology, 2014, 13(2): 159-166. doi: 10.1016/S1474-4422(13)70305-3
[13]	Patton J, Small S L, Rymer W Z. Functional restoration for the stroke survivor: informing the efforts of engineers. Topics in Stroke Rehabilitation, 2008, 15(6): 521-541. doi: 10.1310/tsr1506-521
[14]	Blank A A, French J A, Pehlivan A U, O’Malley M K. Current trends in robot-assisted upper-limb stroke rehabilitation: promoting patient engagement in therapy. Current Physical Medicine and Rehabilitation Reports, 2014, 2(3): 184-195. doi: 10.1007/s40141-014-0056-z
[15]	Metzger J C, Lambercy O, Califfi A, et al. Assessment-driven selection and adaptation of exercise difficulty in robot-assisted therapy: A pilot study with a hand rehabilitation robot. Journal of Neuroengineering and Rehabilitation, 2014, 11(1): Article No. 154
[16]	Choi, Y., Gordon, J., Park, H, Schweighofer N. Feasibility of the adaptive and automatic presentation of tasks system for rehabilitation of upper extremity function post-stroke. Journal of NeuroEngineering and Rehabilitation, 2011, 8(1): 1-12. doi: 10.1186/1743-0003-8-1
[17]	Li C, Rusak Z, Horvath I, Kooijman A, Ji L. Implementation and validation of engagement monitoring in an engagement enhancing rehabilitation system. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2016, 25(6): 726-738.
[18]	Wang J X, Wang W Q, Hou Z G, Liang X, Ren S X, Peng L. Towards enhancement of patients＇ engagement: Online modification of rehabilitation training modes using facial expression and muscle fatigue. In: Proceedings of the 2018 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Hawaii, USA: IEEE, 2018. 2304−2307
[19]	Xu G, Gao X, Pan L, et al. Anxiety detection and training task adaptation in robot-assisted active stroke rehabilitation. International Journal of Advanced Robotic Systems, 2018, 15(6): 1-18.
[20]	Min B K, Marzelli M J, Yoo S S. Neuroimaging-based approaches in the brain–computer interface. Trends in Biotechnology, 2010, 28(11): 552-560. doi: 10.1016/j.tibtech.2010.08.002
[21]	Li S Y, Shu Z L, Sun N, Wang K, Xu C, Yu N B. Online assessment of neural engagement and functional performance for a fine motor control task. In: Proceedings of the 2018 International Conference on Advanced Robotics and Mechatronics. Singapore: IEEE, 2018. 392−396
[22]	Chanel G, Rebetez C, Bétrancourt M, Pun T. Emotion assessment from physiological signals for adaptation of game difficulty. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 2011, 41(6): 1052-1063. doi: 10.1109/TSMCA.2011.2116000
[23]	于宁波, 李思宜, 赵营泉, 王奎, 徐昌. 基于共享控制的人机灵巧力触觉交互系统设计与实现. 仪器仪表学报, 2017, 38(3): 602-611. doi: 10.3969/j.issn.0254-3087.2017.03.012 Yu Ning-Bo, Li Si-yi, Zhao Ying-quan, Wang Kui, Xu Chang, Design and implementation of a dexterous human-robot interaction system based on haptic shared control. Chinese Journal of Scientific Instrument, 2017, 38(3): 602-611(in Chinese). doi: 10.3969/j.issn.0254-3087.2017.03.012
[24]	Woolsey T A, Hanaway J, Gado M H. The Brain Atlas: A Visual Guide to the Human Central Nervous System. Hoboken: John Wiley & Sons, 2017.
[25]	Guidali M, Duschau-Wicke A, Broggi S, Klamroth-Marganska V, Nef T, Riener R. A robotic system to train activities of daily living in a virtual environment. Medical & biological engineering & computing, 2011, 49(10): 1213-1223.
[26]	Sigrist R, Rauter G, Marchal-Crespo L, Riener R, Wolf P. Sonification and haptic feedback in addition to visual feedback enhances complex motor task learning. Experimental brain research, 2015, 233(3): 909-925. doi: 10.1007/s00221-014-4167-7
[27]	Patton J L, Wei Y J, Bajaj P, Scheidt R A. Visuomotor learning enhanced by augmenting instantaneous trajectory error feedback during reaching. PloS One, 2013, 8(1): Article No. e46466
[28]	Abhang P A, Gawali B W, Mehrotra S C. Technological basics of EEG recording and operation of apparatus. Introduction to EEG-and Speech-based Emotion Recognition, 2016: 19-50.
[29]	Newson J J, Thiagarajan T C. EEG frequency bands in psychiatric disorders: A review of resting state studies. Frontiers in Human Neuroscience, 2019, 12: Article No. 521
[30]	Pope A T, Bogart E H, Bartolome D S. Biocybernetic system evaluates indices of operator engagement in automated task. Biological Psychology, 1995, 40(1-2): 187-195. doi: 10.1016/0301-0511(95)05116-3
[31]	Nuamah J K, Seong Y. Support vector machine classification of cognitive tasks based on electroencephalography engagement index. Brain-Computer Interfaces, 2018, 5(1): 1-12. doi: 10.1080/2326263X.2017.1338012
[32]	Al-Hudhud G, Alqahtani L, Albaity H, Alsaeed D, Al-Turaiki I. Analyzing passive BCI signals to control adaptive automation devices. Sensors, 2019, 19(14): Article No. 3042
[33]	Kosmyna N, Maes P. AttentivU: An EEG-based closed-loop biofeedback system for real-time monitoring and improvement of engagement for personalized learning. Sensors, 2019, 19(23): Article No. 5200
[34]	Zollo L, Gallotta E, Guglielmelli E, Sterzi S. Robotic technologies and rehabilitation: new tools for upper-limb therapy and assessment in chronic stroke. European Journal of Physical and Rehabilitation Medicine, 2011, 47(2): 223-236.
[35]	Baron E I, Koop M M, Streicher M C, Rosenfeldt A B, Alberts J L. Altered kinematics of arm swing in Parkinson's disease patients indicates declines in gait under dual-task conditions. Parkinsonism & related disorders, 2018, 48: 61-67.
[36]	Mazzoleni S, Tran V D, Dario P, Posteraro F. Wrist robot-assisted rehabilitation treatment in subacute and chronic stroke patients: from distal-to-proximal motor recovery. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2018, 26(9): 1889-1896. doi: 10.1109/TNSRE.2018.2864935
[37]	Cohen J, Cohen P, West S G, Aiken L S. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. London: Routledge, 2013.
[38]	Zhang J, Fiers P, Witte K A, et al. Human-in-the-loop optimization of exoskeleton assistance during walking. Science, 2017, 356(6344): 1280-1284. doi: 10.1126/science.aal5054
[39]	Maurice P, Huber M E, Hogan N, Sternad D. Velocity-curvature patterns limit human-robot physical interaction. IEEE Robotics & Automation Letters, 2018, 3(1): 249-256.
[40]	Reinkensmeyer D J, Burdet E, Casadio M, et al. Computational neurorehabilitation: modeling plasticity and learning to predict recovery. Journal of neuroengineering and rehabilitation, 2016, 13(1): 1-25. doi: 10.1186/s12984-015-0109-2