基于单试次脑电解码的类自举法谎言预测研究

白帅帅; 陈超; 魏玮; 代璐瑶; 刘烨; 邱爽; 何晖光

doi:10.16383/j.aas.c220341

基于单试次脑电解码的类自举法谎言预测研究

doi: 10.16383/j.aas.c220341

白帅帅^1,,
陈超^{1, 2,},
魏玮^3,,
代璐瑶^{4, 5,},
刘烨^{4, 5,},
邱爽^{3, 6,},
何晖光^{3, 6,}

1.
天津理工大学复杂系统控制理论与应用天津市重点实验室天津 300384
2.
天津大学医学工程与转化医学研究院天津 300072
3.
中国科学院自动化研究所模式识别国家重点实验室类脑智能研究中心北京 100190
4.
中国科学院心理研究所脑与认知科学国家重点实验室北京 100101
5.
中国科学院大学心理学系北京 100049
6.
中国科学院大学人工智能学院北京 100049

基金项目: 国家重点研发计划(2022YFF1202500, 2022YFF1202501), 国家自然科学基金(62206285, 61806146), 中国博士后科学基金(2021M703490)资助

详细信息

作者简介:
白帅帅：天津理工大学硕士研究生. 主要研究方向为ERP分析与解码. E-mail: baishuai@stud.tjut.edu.cn

陈超：天津理工大学教授. 主要研究方向为脑机接口, 人机交互技术. E-mail: cccovb@hotmail.com

魏玮：中国科学院自动化研究所助理研究员. 主要研究方向为脑机接口, 脑电处理分析, 模式识别方法. 本文通信作者. E-mail: weiwei2018@ia.ac.cn

代璐瑶：中国科学院心理研究所硕士研究生. 主要研究方向为欺骗检测, 谎言识别. E-mail: smxdly5@163.com

刘烨：中国科学院心理研究所副研究员. 主要研究方向为认知心理学, 谎言识别. E-mail: liuye@psych.ac.cn

邱爽：中国科学院自动化研究所副研究员. 主要研究方向为精细运动想象脑机接口, 神经调控. E-mail: shuang.qiu@ia.ac.cn

何晖光：中国科学院自动化研究所研究员, 中国科学院大学岗位教授. 主要研究方向为脑机接口, 人工智能, 医学影像分析. E-mail: huiguang.he@ia.ac.cn

计量
- 文章访问数: 639
- HTML全文浏览量: 142
- PDF下载量: 129
- 被引次数: 0
出版历程
- 收稿日期: 2022-04-27
- 录用日期: 2022-09-26
- 网络出版日期: 2022-11-21
- 刊出日期: 2023-10-24

Research on Single-trial EEG Decoding-based Class Bootstrap Method for Lie Prediction

BAI Shuai-Shuai^1
,,
CHEN Chao^{1, 2
,},
WEI Wei^3
,,
DAI Lu-Yao^{4, 5
,},
LIU Ye^{4, 5
,},
QIU Shuang^{3, 6
,},
HE Hui-Guang^{3, 6
,}

1.
Tianjin Key Laboratory of Complex System Control Theory and Application, Tianjin University of Technology, Tianjin 300384
2.
Academy of Medical Engineering and Translational Medicine, Tianjin University, Tianjin 300072
3.
Research Center for Brain-Inspired Intelligence, National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190
4.
State Key Laboratory of Brain and Cognitive Science, Institute of Psychology, Chinese Academy of Sciences, Beijing 100101
5.
Department of Psychology, University of Chinese Academy of Sciences, Beijing 100049
6.
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049

Funds: Supported by National Key Research and Development Program of China (2022YFF1202500, 2022YFF1202501), National Natural Science Foundation of China (62206285, 61806146), and China Postdoctoral Science Foundation (2021M703490)

More Information

Author Bio:
BAI Shuai-Shuai　Master student at Tianjin University of Technology. His research interest covers event-related potential analysis and decoding

CHEN Chao　Professor at Tianjin University of Technology. His research interest covers brain-computer interface and human-computer interaction technology

WEI Wei　Assistant research fellow at the Institute of Automation, Chinese Academy of Sciences. His research interest covers brain-computer interface, EEG processing and analysis, and pattern recognition. Corresponding author of this paper

DAI Lu-Yao　Master student at the Institute of Psychology, Chinese Academy of Sciences. His research interest covers deception detection and lie detection

LIU Ye　Associate research fellow at the Institute of Psychology, Chinese Academy of Sciences. Her research interest covers cognitive psychology and lie detection

QIU Shuang　Associate research fellow at the Institute of Automation, Chinese Academy of Sciences. Her research interest covers fine motor imagination brain-computer interface and neural regulation

HE Hui-Guang　Researcher at the Institute of Automation, Chinese Academy of Sciences, professor at University of Chinese Academy of Sciences. His research interest covers brain-computer interface, artificial intelligence, and medical image analysis

摘要

摘要: 基于脑电(Electroencephalogram, EEG)的谎言检测技术依赖于对事件相关电位(Event-related potential, ERP)的有效解码, 当前主要采用手工设计特征进行脑电分析. 近年来, 单试次脑电分类方法取得了长足进步, 其中端到端的脑电分类方法能够实现对脑电的自动特征提取和分类, 但在谎言检测中缺乏研究和应用, 同时存在无法在测谎场景下直接应用的问题. 本研究设计基于复合反应范式(Complex trial protocol, CTP)进行自我面孔信息识别任务的实验, 采集了18 名被试的脑电数据. 研究了不同端到端的单试次ERP分类方法在谎言检测中的应用, 同时针对单试次脑电解码方法无法直接实际应用的问题, 提出了一种类自举算法. 算法基于数据分布假设, 通过对比各类刺激图像被视为探针刺激时所训练模型的性能, 来推断真正的探针刺激. 实验结果表明, 在基于自我面孔信息的CTP的谎言预测中, 所提出的类自举法性能优于传统探针预测方法, 在仅使用少量脑电数据情况下, 可实现准确的谎言预测.
- 脑电 /
- 谎言预测 /
- 事件相关电位 /
- 复合反应范式 /
- 类自举法
Abstract: Lie detection techniques based on electroencephalogram (EEG) rely on the effective decoding of event-related potential (ERP). At present, manual design features are mainly used for EEG analysis. In recent years, the single-trial EEG classification method has made progress. End-to-end EEG classification methods can realize automatically extract features from EEG and classify, which lacks research and application in lie detection, also those methods cannot be directly applied in lie detection. In this study, we designed the autobiographical-based face recognition task based on a complex trial protocol (CTP) and the EEG of 18 subjects was collected. The application of different single-trial ERP classification methods in lie detection are studied. A class bootstrap method is proposed to solve the problem that the single-trial EEG decoding method cannot be applied to practice directly. The class bootstrap method was based on the assumption of data distribution, the probe stimulus was deduced by comparing the classification performance of classifiers that were trained when each category of stimulus images was set as probe stimuli. The experimental results show that the proposed class bootstrap method outperforms the traditional lie detection method and can accurately predict lies when only a small amount of EEG data is used.
- Electroencephalogram (EEG) /
- lie prediction /
- event-related potential (ERP) /
- complex trial protocol (CTP) /
- class bootstrap method

HTML全文

图 1 单试次实验流程图

Fig. 1 Flow chart of a single-trial experiment

下载: 全尺寸图片幻灯片

图 2 类自举法的分布假设示意图

Fig. 2 Schematic diagram of distribution hypothesis of the class bootstrap method

下载: 全尺寸图片幻灯片

图 3 事件相关电位波形图

Fig. 3 Event-related potential waveform

下载: 全尺寸图片幻灯片

图 4 类自举法中不同脑电标签训练解码模型的特征可视化

Fig. 4 Feature visualization of decoding models trained with different EEG labels in class bootstrap method

下载: 全尺寸图片幻灯片

表 1 单试次脑电解码实验主要参数

Table 1 Main parameters of single-trial EEG decoding experiment

方法	主要参数	代码来源
时域 + SVM	时域特征: Pz导联数据 SVM分类器: C = 100; kernel = “rbf”	利用python工具包sklearn实现
小波 + SVM	连续小波变换; wavelet = “morl”; SVM分类器: C = 100; kernel = “rbf”	利用python工具包pywt、sklearn实现
CSP + LDA	空域特征: 全脑64 通道	利用python工具包pyRiemann、sklearn实现
HDCA	时间窗: 100 ms	根据文献[22]复现
MDRM	xDAWN-Covariances + MDM	python工具包pyRiemann复现
OCLNN	Batch size = 8; optimizer = “adam” lr = 0.0002; betas = (0.9, 0.999) weight_decay = 0.001	根据文献[24]利用Pytorch包复现
EEGNet	Batch size = 64; lr = 0.001 optimizer = “adam”; betas = (0.9, 0.99)	https://github.com/vlawhern/arleegmodels
PLNet	Batch size = 16; lr = 0.001 optimizer = “adam”; betas = (0.9, 0.99) early stop	根据文献[26]利用tensorflow.keras库复现

下载: 导出CSV

表 2 不同方法在不同训练数据量下的分类均衡精度(均值±标准差) (%)

Table 2 Balanced accuracy of different methods under different training data (mean±standard deviation) (%)

方法	模型训练中采用的训练数据量(组)
方法	1	2	3	4	5
时域 + SVM	57.31±5.21**	59.42±5.25***	60.51±6.03***	61.12±6.66***	61.49±7.08***
小波 + SVM	58.06±5.79***	58.80±4.97***	59.40±5.77***	59.12±5.53***	59.31±6.10***
CSP + LDA	59.16±7.04*	63.06±9.14**	65.56±9.11***	67.66±10.91***	68.73±10.77***
HDCA	56.80±5.41**	60.89±5.82**	62.69±7.09***	64.68±7.34***	67.42±7.88***
MDRM	52.52±2.60***	57.99±6.44***	62.47±9.04***	64.91±8.34***	67.69±8.93***
OCLNN	55.25±4.88***	61.32±5.64***	65.83±7.16***	68.28±7.40***	70.88±8.55***
EEGNet	61.79±7.57	64.75±6.79*	68.72±7.39**	69.59±7.73**	71.71±7.80**
PLNet	60.57±6.54	68.38±7.22	73.42±8.44	74.88±8.69	76.73±8.94

下载: 导出CSV

表 3 不同方法在不同数据量下的探针预测正确率 (%)

Table 3 Probe prediction accuracy of different methods under different data volume (%)

方法		使用的总数据量（组）
方法		2	3	4	5	6
BAD	B-P	61.11	66.67	72.22	72.22	77.78
BAD	P-P	61.11	72.22	77.78	77.78	77.78
类自举法	HDCA	27.78	38.89	61.11	77.78	83.33
	MDRM	55.56	50.00	66.67	88.89	88.89
	OCLNN	88.89	72.22	83.33	88.89	88.89
	EEGNet	83.33	72.22	83.33	94.44	100.00
	PLNet	88.89	72.22	83.33	94.44	100.00

下载: 导出CSV

参考文献(29)

[1]	李贺, 傅元, 梁静, 傅小兰. 欺骗识别的准确率:影响因素与提高途径. 心理科学, 2020, 43(5): 1204-1210 doi: 10.16719/j.cnki.1671-6981.20200525 Li He, Fu Yuan, Liang Jing, Fu Xiao-Lan. The accuracy of deceptin detection: Influence factors and improving approaches. Psychological Science, 2020, 43(5): 1204-1210 doi: 10.16719/j.cnki.1671-6981.20200525
[2]	Santhiya P, Chitrakala S. A survey on emotion recognition from EEG signals: Approaches, techniques and challenges. In: Proceedings of the International Conference on Vision Towards Emerging Trends in Communication and Networking (ViTECoN). Vellore, India: IEEE, 2019. 1−6
[3]	Honts C R, Thurber S D, Handler M D. A comprehensive meta-analysis of the comparison question polygraph test. Applied Cognitive Psychology, 2021, 35: 411-427 doi: 10.1002/acp.3779
[4]	Ramele R, Villar A J, Santos J M. EEG waveform analysis of P300 ERP with applications to brain computer interfaces. Brain Sciences, 2018, 8(11):199 doi: 10.3390/brainsci8110199
[5]	Steiner G Z, Barry R J, Gonsalvez C J. Can working memory predict target-to-target interval effects in the P300? International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology, 2013, 89(3): 399−408
[6]	Rosenfeld J P, Labkovsky E, Winograd M, Lui M, Vandenboom C, Chedid E. The Complex Trial Protocol (CTP): a new, countermeasure-resistant, accurate, P300-based method for detection of concealed information. Psychophysiology, 2008, 45(6): 906-919 doi: 10.1111/j.1469-8986.2008.00708.x
[7]	Chang W W, Wang H, Lu Z G, Liu C. A concealed information test system based on functional brain connectivity and signal entropy of audio–visual ERP. IEEE Transactions on Cognitive and Developmental Systems, 2020, 12: 361-370 doi: 10.1109/TCDS.2020.2991359
[8]	Wang H, Qi Y, Yu H, Wang Y, Liu C, Hu G, et al. RCIT: An RSVP-based concealed information test framework using EEG signals. IEEE Transactions on Cognitive and Developmental Systems, 2022, 14: 541-551 doi: 10.1109/TCDS.2021.3053455
[9]	彭丝雨, 周到, 张家琦, 王宇, 高军峰.基于互信息的脑网络及测谎研究.电子学报, 2019, 47(7): 1551-1556 doi: 10.3969/j.issn.0372-2112.2019.07.021 Peng Si-Yu, Zhou Dao, Zhang Jia-Qi, Wang Yu, Gao Jun-Feng. Research on mutual information-based brain network and lie detection. Acta Electronica Sinica, 2019, 47(7): 1551-1556. doi: 10.3969/j.issn.0372-2112.2019.07.021
[10]	Dodia S, Edla D R, Bablani A, Cheruku R. Lie detection using extreme learning machine: A concealed information test based on short-time Fourier transform and binary bat optimization using a novel fitness function. Computational Intelligence, 2020, 36: 637-658, DOI: 10.1111/coin.12256
[11]	Baghel N, Singh D, Dutta M K, Burget R, Myska V. Truth identification from EEG signal by using convolution neural network: Lie detection. In: Proceedings of the 43rd International Conference on Telecommunications and Signal Processing (TSP). Milan, Italy: IEEE, 2020. 550−553
[12]	Bablani A, Edla D R, Kupilli V, Dharavath R. Lie detection using fuzzy ensemble approach with novel defuzzification method for classification of EEG signals. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-13
[13]	Javaid H, Dilawari A, Khan U G, Wajid B. EEG guided multimodal lie detection with audio-visual cues. In: Proceedings of the 2nd International Conference on Artificial Intelligence (ICAI). Islamabad, Pakistan: IEEE, 2022. 71−78
[14]	Zou Z X, Shi Z W, Guo Y H, Ye J P. Object detection in 20 years: A survey [Online], available: https: //doi.org/10.48550/arXiv.1905.05055, November 15, 2022
[15]	Qi F, Wu W, Yu Z L, Gu Z, Wen Z, Yu T, et al. Spatiotemporal-filtering-based channel selection for single-trial EEG classification. IEEE Transactions on Cybernetics, 2021, 51: 558-567 doi: 10.1109/TCYB.2019.2963709
[16]	柳长源, 李文强, 毕晓君.基于RCNN-LSTM的脑电情感识别研究.自动化学报, 2022, 48(3): 917-925 doi: 10.16383/j.aas.c190357 Liu C Y, Li W Q, Bi X J. Research on EEG emotion recognition based on RCNN-LSTM. Acta Automatica Sinica, 2022, 48(3): 917-925 doi: 10.16383/j.aas.c190357
[17]	Santamaría V E, Martínez C V, Vaquerizo V F, Hornero R. EEG-inception: A novel deep convolutional neural network for assistive ERP-based brain-computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28: 2773-2782 doi: 10.1109/TNSRE.2020.3048106
[18]	Luo J, Shi W, Lu N, Wang J, Chen H, Wang Y, et al. Improving the performance of multisubject motor imagery-based BCIs using twin cascaded softmax CNNs. Journal of Neural Engineering, 2021, 18(3), DOI: 10.1088/1741-2552/abe357
[19]	Fu L, Chong W, Yang L, Hao W, Boxun Fu, Ji Y S, et al. Phase preservation neural network for electroencephalography classification in rapid serial visual presentation task. IEEE Transactions on Biomedical Engineering, 2022, 69(6):1931-1942, DOI: 10.1109/TBME.2021.3130917
[20]	Martínez C R, Delorme A, Truong D, Artoni F, Kreutz D K, Sivagnanam S, et al. The open EEGLAB portal interface: High-performance computing with EEGLAB. NeuroImage, 2021, 224(6): Article No. 116778
[21]	Zhang C, Eskandarian A. A survey and tutorial of EEG-based brain monitoring for driver state analysis. IEEE/CAA Journal of Automatica Sinica, 2021, 8: 1222-1242 doi: 10.1109/JAS.2020.1003450
[22]	Gerson A D, Parra L C, Sajda P. Cortically coupled computer vision for rapid image search. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2006, 14(2): 174-179 doi: 10.1109/TNSRE.2006.875550
[23]	Barachant A, Congedo M. A plug and play P300 BCI using information geometry [Online], available: https://arxiv.org/abs/1409.0107, October 10, 2022
[24]	Lawhern V J, Solon A J, Waytowich N R. EEGNet: A compact convolutional neural network for EEG-based brain-computer interfaces. Journal of Neural Engineering, 2018, 15(5): Article No. 056013
[25]	Shan H, Liu Y, Stefanov T P. A simple convolutional neural network for accurate P300 detection and character spelling in brain computer interface. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI). Stockholm, Sweden: 2018. 1604−1610
[26]	Zang B, Lin Y, Liu Z. A deep learning method for single-trial EEG classification in RSVP task based on spatiotemporal features of ERPs. Journal of Neural Engineering, 2021, 18(4).DOI: 10.1088/1741-2552/ac1610
[27]	Haider S K, Jiang A, Jamshed M A, Pervaiz H B, Mumtaz S. Performance enhancement in P300 ERP single trial by machine learning adaptive denoising mechanism. IEEE Networking Letters, 2019, 1: 26-29 doi: 10.1109/LNET.2018.2883859
[28]	何章锦, 陈晨, 张金, 黎峰, 张家琦, 蓝文威等.多通道EEG信号小波相干分析的测谎研究.电子测量与仪器学报, 2020, 34(10): 156-162 doi: 10.13382/j.jemi.B2003017 He Zhang-Jin, Chen Chen, Zhang Jin, Li Feng, Zhang Jia-Qi, Lan Wen-Wei, et al. Lie detection study based on wavelet coherence analysis on multi-channel EEG signals. Journal of Electronic Measurement and Instrument, 2020, 34(10): 156-162. doi: 10.13382/j.jemi.B2003017
[29]	邓晓红, 李婷, 薛超, Rosenfeld J P, 卢洋, 王莹等.基于自我参照编码的复合反应范式:有效鉴别有罪者和无罪知情者. 心理学报, 2021, 53(10): 1105-1119 doi: 10.3724/SP.J.1041.2021.01105 Deng Xiao-Hong, Li Ting, Xue Chao, Rosenfeld J P, Lu Yang, Wang Ying, et al. The complex trial protocol based on self-referential encoding: Discriminating the guilty from the knowledgeable innocent. Acta Psychologica Sinica, 2021, 53(10): 1105-1119 doi: 10.3724/SP.J.1041.2021.01105