Direct-brain-controlled Robot Direction and Speed Based on SSVEP Brain Computer Interaction
-
摘要: 直接用思维意图来控制机器人而没有大脑外周神经和肌肉的参与是人类的一个梦想，目前这一研究已成为国际前沿热点和突破点.传统的脑控机器人（Brain-controlled robot，BCR）主要控制其方向，而本文旨在探讨能够同时脑控机器人方向和速度的有效方法.采用可分类目标数多、单次识别率高且训练时间短的稳态视觉诱发电位（Steady state visual evoked potentials，SSVEP）脑机交互（Brain-computer/machine interaction，BCI/BMI）方法，为脑控机器人运动规划了向左、向右、前进和后退4个方向，设计了低速、中速和高速3级运动速度并组合了9个脑控指令；进而比较并优化了SSVEP刺激目标布局间距以及刺激目标闪烁时间，采用典型相关分析（Canonical correlation analysis，CCA）进行识别.结果表明恰当设置SSVEP刺激目标数及其布局间距和刺激目标闪烁时间，可以有效提高被试/用户直接脑控机器人的性能；优化的SSVEP刺激范式三结合适应SSVEP解码的典型相关分析，8名被试脑控机器人到达终点平均用时为2分40秒，最少用时1分29秒；同时，在脑控机器人运动过程中触碰障碍平均次数为0.88，最少碰触次数为0.本研究显示基于SSVEP的脑机交互可以作为直接脑控机器人灵活运动的一种可选方法，能够实现对机器人多个运动方向和多级速度的控制；也证实了适当增加刺激目标间距可以有效提高SSVEP-BCI脑控指令识别的正确率，说明了该脑控方法的性能与刺激被试的范式有关；再次验证了CCA算法在基于SSVEP的脑机交互中具有优良的效果.最后，为克服单一SSVEP范式存在的局限，本研究也尝试把该范式与运动想象相结合的混合范式用于脑控机器人方向和速度，并进行了初步的研究，表明可以进一步改善控制速度和提高被试舒适度.本文可望为基于SSVEP或与运动想象混合的脑机交互应用于分级或精细控制机器人方向和速度提供思路，并为直接脑控机器人技术推向实际应用打下一定的基础.Abstract: Direct use of thinking to control a robot without peripheral nerves and muscles is a dream of human beings, and this research has become a hot spot and a breakthrough point in the world. The traditional brain-controlled robot (BCR) mainly controls the direction, and this paper aims to explore the effective method to control the direction and speed of robot through brain. Using the brain-computer/interaction (BCI/BMI) method based on steady state visual evoked potential (SSVEP), 9 instructions are designed to control the robot. And the canonical correlation analysis (CCA) method is used to identify EGG patterns. The results show that appropriate setting of the target number and its layout spacing and the flashing time can effectively improve the performance of the direct-brain-controlled robot. The optimization of the SSVEP stimulus paradigm three, the average time of 8 subjects controlled robot to reach destination using 2 minutes and 40 seconds, with a minimum of 1 minutes and 29 seconds. Meanwhile, the average number of touching obstacles is 0.88, the least touch number is 0, and the maximum number of touch is 3. This study shows that the BCI based on SSVEP can be used for the direct-brain-controlled robot. In order to overcome the limitations in the single SSVEP paradigm, this study also combines motor (Hybrid-BCI) imagery to control the robot. This paper is expected to provide ideas for the direction and speed control for the brain-computer interaction based on SSVEP or combined with motor imagery, and to provide a certain basis for practical direct-brain-controlled robot technology.
-
图 2 脑控机器人测试平台及脑控机器人系统客户/服务器结构
Fig. 2 The test platform and client/server architecture \\for brain-controlled robot system
图 4 SSVEP脑机交互刺激范式二和范式三
Fig. 4 The second and the third SSVEP-based BCI stimulation paradigm
图 5 基于SSVEP脑机交互直接脑控机器人系统结构
Fig. 5 The structure of SSVEP-based BCI directly brain-controlled robot
图 6 三种SSVEP刺激范式下被试脑控机器人达到终点平均用时及触碰障碍物次数及在1 $\sim$ 4秒刺激时间下的平均正确识别率
Fig. 6 The averaged consuming time,number of touching obstacles and classification accuracies across 8 subjects when their controlling robot by their brains to reach the destination under three SSVEP stimulation paradigms
表 1 一种脑控机器人策略: SSVEP 脑机交互刺激范式刺激目标对应的脑控制指令
Table 1 A strategy for brain-controlled robot: control commands corresponding to the stimulus targets of SSVEP-based BCI stimulation paradigm
F(Forward) BCIID01CA1000 低速前进 F+ BCIID01CA2000 中速前进 F++ BCIID01CA3000 高速前进 B(Backward) BCIID01CA0100 低速后退 B+ BCIID01CA0200 中速后退 L(Left) BCIID01CA0010 低速左转 L+ BCIID01CA0020 中速左转 R(Right) BCIID01CA0001 低速右转 R+ BCIID01CA0002 中速右转 
下载: 导出CSV
表 2 三种SSVEP 刺激范式下被试脑控机器人达到终点用时及触碰障碍物次数
Table 2 The consuming time and the number of touching obstacles when subjects controlling robot by their brains to reach the destination under three SSVEP stimulation paradigms
S SSVEP刺激范式一 SSVEP刺激范式二 SSVEP刺激范式三 用时(min) 触碰次数 用时(min) 触碰次数 用时(min) 触碰次数 S1 6018 5 4036 3 3059 3 6026 3 3054 2 3036 0 5056 4 4012 2 3041 1 6032 6 4057 3 2046 1 S2 5048 3 4020 1 3039 2 5026 4 4029 3 3001 0 5050 3 4021 2 2019 0 S3 5003 2 3039 1 3002 0 5043 4 3054 2 2043 1 6038 5 3004 0 2037 0 S4 5055 4 4020 3 3056 3 5052 4 3039 1 3017 2 5018 4 3035 1 1059 0 S5 4048 3 3026 2 2032 1 4044 3 3005 1 2009 0 5046 4 3022 2 2047 0 S6 6009 6 3051 2 3014 2 5054 4 3025 1 2025 0 5027 5 2050 2 1029 0 S7 4059 5 3009 2 2027 1 4055 6 2059 2 1047 1 7016 5 3053 1 3042 2 S8 6011 4 3054 2 3025 0 6049 4 3013 2 3024 1 Aver 5035 4.17 3032 1.75 2040 1.88 Min 4044 2 2050 0 1029 0 Var 0.39 1.06 0.39 0.60 0.46 0.64
下载: 导出CSV
表 3 SSVEP脑机交互刺激范式一下刺激目标不同闪烁时间被试的正确识别率(%)
Table 3 The correct recognition rate (%) at different flickering durations for 8 subjects under the first SSVEP-based BCI stimulation paradigm
S Td=1s Td=2s Td=3s Td=4s S1 33.33 66.67 83.33 86.67 S2 36.67 70.00 86.67 93.33 S3 30.00 63.33 80 86.67 S4 30.00 60.00 76.67 80 S5 46.67 80.00 93.33 100 S6 43.33 80.00 90 96.67 S7 36.67 70.00 86.67 90 S8 40.00 66.67 83.33 86.67 Aver 37.08 69.58 85 90 Max 46.67 80.00 93.33 100 Var 31.78 45.66 24.99 36.11
下载: 导出CSV
表 4 SSVEP脑机交互刺激范式二下刺激目标不同闪烁时间被试的正确识别率(%)
Table 4 The correct recognition rate (%) at different flickering durations for 8 subjects under the second SSVEP-based BCI stimulation paradigm stimulation paradigm
S Td=1s Td=2s Td=3s Td=4s S1 40.00 73.33 86.67 93.33 S2 43.33 80.00 90 96.67 S3 36.67 73.33 86.67 93.33 S4 40.00 76.67 90 93.33 S5 56.67 86.67 96.67 100 S6 53.33 83.33 93.33 100 S7 43.33 76.67 90 93.33 S8 46.67 80.00 93.33 96.67 Aver 45.00 78.75 90.83 95.83 Max 56.67 86.67 96.67 100 Var 41.67 19.28 10.41 7.65
下载: 导出CSV
表 5 脑机交互刺激范式三下刺激目标不同闪烁时间被试的正确识别率(%)
Table 5 The correct recognition rate (%) at different flickering durations for 8 subjects under the third SSVEP-based BCI stimulation paradigm stimulation paradigm
S Td=1 s Td=2 s Td=3 s Td=4 s 4 5 S1 43.33 80.00 90.00 93.33 4 5 S2 46.67 83.33 93.33 96.67 4 5 S3 43.33 80.00 90.00 93.33 4 5 S4 40.00 76.67 90.00 96.67 4 5 S5 60.00 86.67 96.67 100 4 5 S6 53.33 86.67 96.67 100 4 5 S7 46.67 83.33 93.33 93.33 4 5 S8 46.67 83.33 96.67 100 4 5 Aver 47.50 82.50 93.33 96.67 4 5 Max 60.00 86.67 96.67 100 4 5 Var 35.42 10.42 8.34 8.34 4 5
下载: 导出CSV
-
[1] Brain-computer interface from Wikipedia, the free encyclopedia [Online], available: http://en.wikipedia.org/wiki/ Brain-computer_interface, April 29, 2016. [2] Hochberg L R, Bacher D, Jarosiewicz B, Masse N Y, Simeral J D, Vogel J, Haddadin S, Liu J, Cash S S, van der Smagt P, Donoghue J P. Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature, 2012, 485(7398): 372-375 doi: 10.1038/nature11076 [3] 伏云发, 王越超, 李洪谊, 徐保磊, 李永程. 直接脑控机器人接口技术. 自动化学报, 2012, 38(8): 1229-1246 doi: 10.3724/SP.J.1004.2012.01229Fu Yun-Fa, Wang Yue-Chao, Li Hong-Yi, Xu Bao-Lei, Li Yong-Cheng. Direct brain-controlled robot interface technology. Acta Automatica Sinica, 2012, 38(8): 1229-1246 doi: 10.3724/SP.J.1004.2012.01229 [4] McFarland D J, Wolpaw J R. Brain-computer interface operation of robotic and prosthetic devices. Computer, 2008, 41(10): 52-56 doi: 10.1109/MC.2008.409 [5] Bell C J, Shenoy P, Chalodhorn R, Rao R P N. Control of a humanoid robot by a noninvasive brain-computer interface in humans. Journal of Neural Engineering, 2008, 5(2): 214-220 doi: 10.1088/1741-2560/5/2/012 [6] Millan J R, Renkens F, Mourino J, Gerstner W. Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE Transactions on Biomedical Engineering, 2004, 51(6): 1026-1033 doi: 10.1109/TBME.2004.827086 [7] Comparison of consumer brain-computer interfaces from Wikipedia, the free encyclopedia [Online], available: http://en.wikipedia.org/wiki/Comparison_of_consumer_ brain-computer_interface_devices, December 20, 2015. [8] 伏云发, 徐保磊, 李永程, 李洪谊, 王越超, 余正涛. 基于运动相关皮层电位握力运动模式识别研究. 自动化学报, 2014, 40(6): 1045-1057 http://www.aas.net.cn/CN/abstract/abstract18374.shtmlFu Yun-Fa, Xu Bao-Lei, Li Yong-Cheng, Li Hong-Yi, Wang Yue-Chao, Yu Zheng-Tao. Recognition of actual grip force movement modes based on movement-related cortical potentials. Acta Automatica Sinica, 2014, 40(6): 1045-1057 http://www.aas.net.cn/CN/abstract/abstract18374.shtml [9] SchlÖgl A, Lee F, Bischof H, Pfurtscheller G. Characterization of four-class motor imagery EEG data for the BCI-competition 2005. Journal of Neural Engineering, 2005, 2(4): L14-L22 http://cn.bing.com/academic/profile?id=2057219090&encoded=0&v=paper_preview&mkt=zh-cn [10] 孙会文, 伏云发, 熊馨, 杨俊, 刘传伟, 余正涛. 基于HHT运动想象脑电模式识别研究. 自动化学报, 2015, 41(9): 1686-1692 http://www.aas.net.cn/CN/abstract/abstract18742.shtmlSun Hui-Wen, Fu Yun-Fa, Xiong Xin, Yang Jun, Liu Chuan-Wei, Yu Zheng-Tao. Identification of EEG induced by motor imagery based on Hilbert-Huang transform. Acta Automatica Sinica, 2015, 41(9): 1686-1692 http://www.aas.net.cn/CN/abstract/abstract18742.shtml [11] Fu Y F, Xu B L, Li Y C, Wang Y C, Yu Z T, Li H Y. Single-trial decoding of imagined grip force parameters involving the right or left hand based on movement-related cortical potentials. Chinese Science Bulletin, 2014, 59(16): 1907-1916 doi: 10.1007/s11434-014-0234-5 [12] Lv J, Li Y Q, Gu Z H. Decoding hand movement velocity from electroencephalogram signals during a drawing task. Biomedical Engineering Online, 2010, 9: 64 doi: 10.1186/1475-925X-9-64 [13] Yin X X, Xu B L, Jiang C H, Fu Y F, Wang Z D, Li H Y, Shi G. A hybrid BCI based on EEG and fNIRS signals improves the performance of decoding motor imagery of both force and speed of hand clenching. Journal of Neural Engineering, 2015, 12(3): 036004, DOI: 10.1088/1741-2560/12/3/036004 [14] 尧德中, 刘铁军, 雷旭, 杨平, 徐鹏, 张杨松. 基于脑电的脑—机接口: 关键技术和应用前景. 电子科技大学学报, 2009, 38(5): 550-554 http://www.cnki.com.cn/Article/CJFDTOTAL-DKDX200905010.htmYao De-Zhong, Liu Tie-Jun, Lei Xu, Yang Ping, Xu Peng, Zhang Yang-Song. Electroencephalogram based brain-computer interface: key techniques and application prospect. Journal of University of Electronic Science and Technology of China, 2009, 38(5): 550-554 http://www.cnki.com.cn/Article/CJFDTOTAL-DKDX200905010.htm [15] 王行愚, 金晶, 张宇, 王蓓. 脑控: 基于脑—机接口的人机融合控制. 自动化学报, 2013, 39(3): 208-221 doi: 10.1016/S1874-1029(13)60023-3Wang Xing-Yu, Jin Jing, Zhang Yu, Wang Bei. Brain control: human-computer integration control based on brain-computer interface. Acta Automatica Sinica, 2013, 39(3): 208-221 doi: 10.1016/S1874-1029(13)60023-3 [16] Zhang R, Yao D, Valdés-Sosa P A, Li F L, Li P Y, Zhang T, Ma T, Li Y J, Xu P. Efficient resting-state EEG network facilitates motor imagery performance. Journal of Neural Engineering, 2015, 12(6): 066024, DOI: 10.1088/1741-2560/12/6/066024 [17] Fazel-Rezai R, Allison B Z, Guger C, Sellers E W, Kleih S C, Kübler A. P300 brain computer interface: current challenges and emerging trends. Frontiers in Neuroengineering, 2012, 5: 14 http://cn.bing.com/academic/profile?id=2105438329&encoded=0&v=paper_preview&mkt=zh-cn [18] Gao S K, Wang Y J, Gao X R, Hong B. Visual and auditory brain-computer interfaces. IEEE Transactions on Biomedical Engineering, 2014, 61(5): 1436-1447 doi: 10.1109/TBME.2014.2300164 [19] Chen X G, Wang Y J, Nakanishi M, Gao X R, Jung T P, Gao S K. High-speed spelling with a noninvasive brain-computer interface. Proceedings of the National Academy of Sciences of the United States of America, 2015, 112(44): E6058-E6067 http://cn.bing.com/academic/profile?id=2132876794&encoded=0&v=paper_preview&mkt=zh-cn [20] Bin G Y, Gao X R, Yan Z, Hong B, Gao S K. An online multi-channel SSVEP-based brain-computer interface using a canonical correlation analysis method. Journal of Neural Engineering, 2009, 6(4): 1771-1779 http://cn.bing.com/academic/profile?id=2105478324&encoded=0&v=paper_preview&mkt=zh-cn [21] The 2th competition of China brain-computer interface [Online], available: http://www.bcicompetition.cn, December 22, 2015. [22] Lin Z L, Zhang C S, Wu W, Gao X R. Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Transactions on Biomedical Engineering, 2006, 53(12): 2610-2614 doi: 10.1109/TBME.2006.886577 [23] Long J Y, Li Y Q, Wang H T, Yu T Y, Pan J H, Li F. A hybrid brain computer interface to control the direction and speed of a simulated or real wheelchair. IEEE Transactions on Neural Systems & Rehabilitation Engineering, 2012, 20(5): 720-729 http://cn.bing.com/academic/profile?id=2001471870&encoded=0&v=paper_preview&mkt=zh-cn [24] Middendorf M, McMillan G, Calhoun G, Jones K S. Brain-computer interfaces based on the steady-state visual-evoked response. IEEE Transactions on Rehabilitation Engineering, 2000, 8(2): 211-214 doi: 10.1109/86.847819 [25] Mak J N, Arbel Y, Minett J W, McCane L M, Yuksel B, Ryan D, Thompson D, Bianchi L, Erdogmus D. Optimizing the P300-based brain-computer interface: current status, limitations and future directions. Journal of Neural Engineering, 2011, 8(2): 025003, DOI: 10.1088/1741-2560/8/2/025003 [26] Guger C, Edlinger G, Harkam W, Niedermayer I, Pfurtscheller G. How many people are able to operate an EEG-based brain-computer interface (BCI)? IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2003, 11(2): 145-147 doi: 10.1109/TNSRE.2003.814481 [27] Guger C, Allison B Z, GroÜwindhager B, PrÜckl R, HintermÜller C, Kapeller C, Bruckner M, Krausz G, Edlinger G. How many people could use an SSVEP BCI? Frontiers in Neuroscience, 2012, 6: 169 http://cn.bing.com/academic/profile?id=2023666981&encoded=0&v=paper_preview&mkt=zh-cn [28] Jin J, Daly I, Zhang Y, Wang X Y, Cichocki A. An optimized ERP brain-computer interface based on facial expression changes. Journal of Neural Engineering, 2014, 11(3): 1082-1088 http://cn.bing.com/academic/profile?id=2089448644&encoded=0&v=paper_preview&mkt=zh-cn -
图(6) / 表(5)
计量
- 文章访问数: 2468
- HTML全文浏览量: 302
- PDF下载量: 1757
- 被引次数: 0
