2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于生理信号的情感计算研究综述

权学良 曾志刚 蒋建华 张亚倩 吕宝粮 伍冬睿

权学良, 曾志刚, 蒋建华, 张亚倩, 吕宝粮, 伍冬睿. 基于生理信号的情感计算研究综述. 自动化学报, 2021, 47(x): 1−16 doi: 10.16383/j.aas.c200783
引用本文: 权学良, 曾志刚, 蒋建华, 张亚倩, 吕宝粮, 伍冬睿. 基于生理信号的情感计算研究综述. 自动化学报, 2021, 47(x): 1−16 doi: 10.16383/j.aas.c200783
Quan Xue-Liang, Zeng Zhi-Gang, Jiang Jian-Hua, Zhang Ya-Qian, Lv Bao-Liang, Wu Dong-Rui. Physiological signals based affective computing: a systematic review. Acta Automatica Sinica, 2021, 47(x): 1−16 doi: 10.16383/j.aas.c200783
Citation: Quan Xue-Liang, Zeng Zhi-Gang, Jiang Jian-Hua, Zhang Ya-Qian, Lv Bao-Liang, Wu Dong-Rui. Physiological signals based affective computing: a systematic review. Acta Automatica Sinica, 2021, 47(x): 1−16 doi: 10.16383/j.aas.c200783

基于生理信号的情感计算研究综述

doi: 10.16383/j.aas.c200783
基金项目: 湖北省技术创新专项资助项目(2019AEA171), 湖北省杰出青年基金(2020CFA050), 武汉市应用基础前沿项目(2020020601012240), 国家自然科学基金(61673266, 61976135)资助
详细信息
    作者简介:

    权学良:华中科技大学人工智能与自动化学院硕士研究生. 主要研究方向为机器学习、脑机接口、情感计算. E-mail: quanxl@hust.edu.cn

    曾志刚:华中科技大学人工智能与自动化学院院长,教授,博士生导师. 主要研究方向为: 神经网络理论与应用,动力系统稳定性,联想记忆. E-mail: zgzeng@mail.hust.edu.cn

    蒋建华:华中科技大学人工智能与自动化学院副教授,硕士生导师. 主要研究方向为燃料电池系统集成与控制、动力电池系统管理、系统优化. E-mail: drwu@hust.edu.cn

    张亚倩:上海交通大学计算机科学与工程系研究助理教授. 主要研究方向为强化学习、机器学习、人机交互. E-mail: zhangyaqian@sjtu.edu.cn

    吕宝粮:上海交通大学计算机科学与工程系教授,博士生导师. 主要研究方向为仿脑计算理论与模型、神经网络、机器学习、脑-机交互、情感计算. E-mail: blu@cs.sjtu.edu.cn

    伍冬睿:华中科技大学人工智能与自动化学院教授,博士生导师,图像信息处理与智能控制教育部重点实验室副主任. 主要研究方向为机器学习、脑机接口、计算智能、情感计算. 本文通信作者. E-mail: drwu@hust.edu.cn

Physiological Signals Based Affective Computing: A Systematic Review

Funds: Supported by Technology Innovation Project of Hubei Province of China (2019AEA171), Hubei Province Distinguished Young Scholar Fund (2020CFA050), Wuhan Science and Technology Bureau (2020020601012240), and the National Natural Science Foundation of China~(61673266, 61976135)
  • 摘要: 情感计算是现代人机交互中的一个重要研究方向, 旨在研究与开发能够识别、解释、处理和模拟人类情感的理论、方法与系统. 脑电、心电、皮肤电等生理信号是情感计算中重要的输入信号. 本文总结了近年来基于脑电等生理信号的情感计算研究所取得的进展. 首先介绍情感计算的相关基础理论, 不同生理信号与情感变化之间的联系, 以及基于生理信号的情感计算工作流程和相关公开数据集. 接下来介绍生理信号的特征工程和情感计算中的机器学习算法, 重点介绍适合处理个体差异的迁移学习、降低数据标注量的主动学习和融合特征工程与学习器的深度学习算法. 最后, 指出基于生理信号的情感计算研究中面临的一些挑战.
  • 图  1  情感计算研究发展简史.

    Fig.  1  A brief history of affective computing research.

    图  2  情绪的连续型维度空间表示.

    Fig.  2  Continuous dimensional representations of emotions.

    表  1  情感计算中常用的生理信号.

    Table  1  Common physiological signals in affective computing.

    生理信号类别 英文名称 英文缩写
    脑电图 Electroencephalogram EEG
    肌电图 Electromyogram EMG
    心电图 Electrocardiogram ECG
    眼电图 Electrooculogram EOG
    心率变异性 Heart Rate Variability HRV
    皮肤电反应 Galvanic Skin Response GSR
    皮肤电应答 Electrodermal Response EDR
    皮肤电活动 Electrodermal Activity EDA
    血压信号 Blood Pressure BP
    皮肤温度 Skin Temperature ST
    呼吸模式 Respiration Pattern RSP
    光电容积脉搏波 Photoplethysmogram PPG
    眼动信号 Eye Movement EM
    脉搏信号 Pulse Rate PR
    血氧饱和度 Oxygen Saturation SpO2
    下载: 导出CSV

    表  2  脑电频率划分.

    Table  2  Frequency bands of EEG.

    脑波 频率 人体状态
    $ \delta $ 0.1−3 Hz 深度睡眠且没有做梦时
    $ \theta $ 4−7 Hz 成人情绪受到压力、失望或挫折时
    $ \alpha $ 8−12 Hz 放松、平静、闭眼但清醒时
    $ \beta $ 12.5−28 Hz 放松但精神集中、激动或焦虑
    $ \gamma $ 29−50 Hz 提高意识、幸福感、放松、冥想
    下载: 导出CSV

    表  3  谷歌学术中2010年以来基于生理信号的情感计算工作统计.

    Table  3  Statistics of physiological signal based affective computing Google Scholar publications since 2010.

    生理信号类型 对应检索关键词 文献数量
    脑电图 EEG OR Electroencephalogram 913
    心电图 ECG OR Electrocardiogram 70
    心率变异性 HRV OR (Heart Rate Variability) 38
    皮肤电 GSR OR EDA OR EDR OR Electrodermal 27
    肌电图 EMG OR Electromyogram 25
    光电容积脉搏波 PPG OR Photoplethysmogram 13
    血压 Blood Pressure 7
    脉搏 Pulse Rate 4
    皮肤温度 Skin Temperature 3
    眼电图 EOG OR Electrooculogram 2
    血氧 SpO2 OR (Oxygen Saturation) OR (Blood Oxygen) 2
    总计 1104
    注: “情感计算”对应的检索关键词为: (emotion OR emotional OR affect OR affective) + (recognize OR recognition OR classify OR classification OR detect OR detection OR predict OR prediction OR estimate OR estimation OR model OR state OR computing).
    下载: 导出CSV

    表  4  部分最近的基于生理信号的情感计算工作.

    Table  4  Some recent studies on physiological signals based affective computing.

    参考文献 生理信号
    [26] GSR、PPG
    [27] EMG、GSR、PPG
    [28] EMG、GSR、BP
    [29] EEG、EMG、EOG、GSR、BP、ST、PR、EDA、RSP
    [30] EEG、ECG、GSR
    [31] EMG、ECG、EDR、BP、ST、RSP
    [32] ECG、EDA、ST
    [33] EEG、EM
    下载: 导出CSV

    表  5  情感计算常用公开数据集.

    Table  5  Popular public affective computing datasets.

    数据集 内容说明 任务模型
    MAHNOB-HCI[34] 27名被试的EEG及多种生理信号和图片、视频信息 VAD模型
    RECOLA[35] 46名被试的ECG、EOG及音频、视频信息 VA模型
    DECAF[36] 30名被试的EOG、ECG、EMG和视频信息 VA模型
    ASCERTAIN[37] 58名被试的EEG、ECG、GSR和图片信息 5种情绪类别, VA模型
    AMIGOS[38] 40名被试的EEG、ECG、GSR及图片、视频信息 5种情绪类别, VA模型
    DEAMER[39] 23名被试的EEG、ECG VAD模型
    RCLS[40] 14名被试的EEG 3种情绪类别
    MPED[41] 23名被试的EEG、ECG、RSP、GSR 7种情绪类别
    HR-EEG4EMO[42] 27名被试的EEG 高兴、悲伤两种情绪类别
    SEED[21, 33] 15名被试, 每名被试3次实验的EEG 3种情绪类别
    SEED-IV[33] 15名被试, 每名被试3次实验的EEG、EM 4种情绪类别
    DEAP[43] 32名被试的EEG、EOG、EMG、GSR、RSP、BP、ST VAD模型
    下载: 导出CSV

    表  6  不同深度特征提取方式及效果.

    Table  6  Different deep learning methods of feature extract and their effects.

    作者及参考文献 神经网络模型 数据集 准确率
    Yin等[109] 堆叠式自编码器 DEAP 83.0%(Valence/2)、84.1%(Arousal/2)
    Fourati等[110] 回声状态网络 DEAP 71.0%(Valence/2)、68.3%(Arousal/2)
    Ren等[111] 融合大脑不对称特性的回声状态网络 DEAP 78.2%(Average/4)
    Liu等[112] 多层次特征引导胶囊网络 DEAP 98.0%(Valence/2)、98.3%(Arousal/2)、98.3%(Dominance/2)
    Wu等[113] 关键子网络选择 SEED 81.5%(Average/3)
    Yang等[114] 具有子网节点的分层网络模型 SEED 85.7%(Average/3)
    Wang等[115] 双向长短期记忆网络 SEED 95.0%(Average/3)
    Zhang等[116] 变分路径推理 SEED 94.3%(Average/3)
    Cimtay等[117] 卷积神经网络 SEED 73.7%(Average/3)、82.9%(Average/2)
    注: (Valence/2)表示Valence维度2分类准确率, (Average/4)表示情绪4分类准确率.
    下载: 导出CSV
  • [1] 林崇德, 杨治良, 黄希庭. 心理学大辞典. 上海: 上海教育出版社, 2003

    Lin Chong-De, Yang Zhi-Liang, Huang Xi-Ting. The Comprehensive Dictionary of Psychology. Shanghai: Shanghai Educational Publishing House, 2003
    [2] 陶建华, 李雅. 情感计算研究进展. 中国计算机学会通讯, 2016, 12(10): 62−70

    Tao Jian-Hua, Li Ya. Advances in affective computing. Communications of China Computer Federation, 2016, 12(10): 62−70
    [3] Affective computing.[Online], available: https://en.wikipedia.org/wiki/Affective_computing, January, 12, 2021
    [4] Minsky M. The Society of Mind. USA: Simon and Schuster, 1986
    [5] Picard R W. Affective Computing. Boston: MIT Press, 1997
    [6] Liu Y, Sourina O, Nguyen M K. Real-time EEG-based emotion recognition and its applications. Transactions on Computational Science, 2011, 12: 256−277
    [7] 林文倩. 生理信号驱动的情绪识别及交互应用研究[Ph.D. dissertation], 浙江大学, 2019

    Lin Wen-Qian. Emotion Recongnition and Application Based on Physiological Signals[Ph.D. dissertation], Zhejiang University, 2019
    [8] Ortony A, Clore G L, Collins A. The Cognitive Structure of Emotions. UK: Cambridge University Press, 1990
    [9] 林传鼎. 心理学词典. 江西: 江西科学技术出版社, 1986

    Lin Chuan-Ding. Dictionary of Psychology. Jiangxi: Jiangxi Science and Technology Press, 1986
    [10] Ekman P, Friesen W. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 1971, 17: 124−129 doi: 10.1037/h0030377
    [11] Lazarus R S. From psychological stress to the emotions: A history of changing outlooks. Annual Review of Psychology, 1993, 44(1): 1−21 doi: 10.1146/annurev.ps.44.020193.000245
    [12] Plutchik R. Emotions and Life: Perspectives from Psychology, Biology, and Evolution. USA: American Psychological Association, 2003
    [13] Russell J A. A circumplex model of affect. Journal of Personality and Social Psychology, 1980, 39(6): 1161 doi: 10.1037/h0077714
    [14] MMehrabian A. Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology, 1996, 14(4): 261−292 doi: 10.1007/BF02686918
    [15] Electroencephalography.[Online], available: https://en.wikipedia.org/wiki/Electroencephalography, January, 12, 2021
    [16] Zheng W L, Zhu J Y, Lu B L. Identifying stable patterns over time for emotion recognition from EEG. IEEE Trans. on Affective Computing, 2019, 10(3): 417−429 doi: 10.1109/TAFFC.2017.2712143
    [17] 张冠华, 余旻婧, 陈果, 韩义恒, 张丹, 赵国朕, 刘永进. 面向情绪识别的脑电特征研究综述. 中国科学: 信息科学, 2019, 49(9): 1097−1118 doi: 10.1360/N112018-00337

    Zhang Guang-Hua, Yu Min-Jing, Chen Guo, Han Yi-Heng, Zhang Dan, Zhao Guo-Zhen, Liu Yong-Jin. A review of EEG features for emotion recognition. Science Sinica Information, 2019, 49(9): 1097−1118 doi: 10.1360/N112018-00337
    [18] Mehdizadehfar V, Ghassemi F, Fallah A, Pouretemad H. EEG study of facial emotion recognition in the fathers of autistic children. Biomedical Signal Processing and Control, 2020, 56: 101721 doi: 10.1016/j.bspc.2019.101721
    [19] Cacioppo J T, Tassinary L G. Principles of Psychophysiology: Physical, Social, and Inferential Elements. UK: Cambridge University Press, 1990
    [20] Heart rate variability.[Online], available: https://en.wikipedia.org/wiki/Heart_rate_variability, January, 12, 2021
    [21] Zheng W L, Lu B L. Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans. on Autonomous Mental Development, 2015, 7(3): 162−175 doi: 10.1109/TAMD.2015.2431497
    [22] Sohaib A T, Qureshi S, Hagelback J, Hilborn O, Jercic P. Evaluating classifiers for emotion recognition using EEG. In: Proc. 7th Int'l Conf. on Augmented Cognition. Las Vegas, NV, Jul 2013. 492−501
    [23] Nie D, Wang X, Shi L, Lu B. EEG-based emotion recognition during watching movies. In: Proc. 5th Int'l IEEE/EMBS Conf. on Neural Engineering. Cancun, Mexico, Apr 2011. 667−670
    [24] Lin Y, Wang C, Jung T, Wu T, Jeng S, Duann J, Chen J. EEG-based emotion recognition in music listening. IEEE Trans. on Biomedical Engineering, 2010, 57(7): 1798−1806 doi: 10.1109/TBME.2010.2048568
    [25] Alarcao S M, Fonseca M J. Emotions recognition using EEG signals: A survey. IEEE Trans. on Affective Computing, 2019, 10(3): 374−393 doi: 10.1109/TAFFC.2017.2714671
    [26] Dominguez J A, Campo K C, Martinez J C, Delahoz E J, Ortiz S H C. A machine learning model for emotion recognition from physiological signals. Biomedical Signal Processing and Control, 2020, 55: 101646 doi: 10.1016/j.bspc.2019.101646
    [27] Joesph C, Rajeswari A, Premalatha B, Balapriya C. Implementation of physiological signal based emotion recognition algorithm. In: Proc. 36th IEEE Int'l Conf. on Data Engineering. Dallas, TX, Apr 2020. 2075−2079
    [28] Zhu Q, Lu G, Yan J. Valence-arousal model based emotion recognition using EEG, peripheral physiological signals and facial expression. In: Proc. 4th Int'l Conf. on Machine Learning and Soft Computing. Haiphong City, Viet Nam, Jan 2020. 81−85
    [29] Fabiano D, Canavan S J. Emotion recognition using fused physiological signals. In: Proc. 8th Int'l Conf. on Affective Computing and Intelligent Interaction. Cambridge, UK, Sep 2019. 42−48
    [30] Chang E J, Rahimi A, Benini L, Wu A. Hyperdimensional computing-based multimodality emotion recognition with physiological signals. In: Proc. IEEE Int'l Conf. on Artificial Intelligence Circuits and Systems. Hsinchu, Taiwan, China, Mar 2019. 137−141
    [31] Zhu J, Zhao X, Hu H, Gao Y. Emotion recognition from physiological signals using multi-hypergraph neural networks. In: Proc. IEEE Int'l Conf. on Multimedia and Expo. Shanghai, China, Jul 2019. 610−615
    [32] Ali M, Machot F A, Mosa A H, Jdeed M, Machot E A, Kyamakya K. A globally generalized emotion recognition system involving different physiological signals. Sensors, 2018, 18(6): 1905 doi: 10.3390/s18061905
    [33] Zheng W L, Liu W, Lu Y, Lu B L, Cichocki A. EmotionMeter: A multimodal framework for recognizing human emotions. IEEE Trans. on Cybernetics, 2019, 49(3): 1110−1122 doi: 10.1109/TCYB.2018.2797176
    [34] Soleymani M, Lichtenauer J, Pun T, Pantic M. A multimodal database for affect recognition and implicit tagging. IEEE Trans. on Affective Computing, 2012, 3(1): 42−55 doi: 10.1109/T-AFFC.2011.25
    [35] Ringeval F, Sonderegger A, Sauer J S, Lalanne D. Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions. In: Proc. 10th IEEE Int'l Conf. and Workshops on Automatic Face and Gesture Recognition. Shanghai, China, Apr 2013. 1−8
    [36] Abadi M K, Subramanian R, Kia S M, Avesani P, Patras I, Sebe N. DECAF: MEG-based multimodal database for decoding affective physiological responses. IEEE Trans. on Affective Computing, 2015, 6(3): 209−222 doi: 10.1109/TAFFC.2015.2392932
    [37] Subramanian R, Wache J, Abadi M K, Vieriu R L, Winkler S, Sebe N. ASCERTAIN: Emotion and personality recognition using commercial sensors. IEEE Trans. on Affective Computing, 2016, 9(2): 147−160
    [38] Correa J A, Abadi M K, Sebe N, Patras I. AMIGOS: A dataset for affect, personality and mood research on individuals and groups. IEEE Trans. on Affective Computing, 2018, 1: 1−15
    [39] Katsigiannis S, Ramzan N. DREAMER: A database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices. IEEE Journal of Biomedical and Health Informatics, 2017, 22(1): 98−107
    [40] Li Y, Zheng W, Cui Z, Zong Y, Ge S. EEG emotion recognition based on graph regularized sparse linear regression. Neural Processing Letters, 2019, 49(2): 555−571 doi: 10.1007/s11063-018-9829-1
    [41] Song T, Zheng W, Lu C, Zong Y, Zhang X, Cui Z. MPED: A multi-modal physiological emotion database for discrete emotion recognition. IEEE Access, 2019, 7(12): 177−191
    [42] Becker H, Fleureau J, Guillotel P, Wendling F, Merlet I, Albera L. Emotion recognition based on high-resolution EEG recordings and reconstructed brain sources. IEEE Trans. on Affective Computing, 2020, 11(2): 244−257 doi: 10.1109/TAFFC.2017.2768030
    [43] Koelstra S, Muhl C, Soleymani M, Lee J, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I. DEAP: A database for emotion analysis; using physiological signals. IEEE Trans. on Affective Computing, 2012, 3(1): 18−31 doi: 10.1109/T-AFFC.2011.15
    [44] Jenke R, Peer A, Buss M. Feature extraction and selection for emotion recognition from EEG. IEEE Trans. on Affective Computing, 2014, 5(3): 327−339 doi: 10.1109/TAFFC.2014.2339834
    [45] Wang X, Nie D, Lu B. EEG-based emotion recognition using frequency domain features and support vector machines. In: Proc. 18th Int'l Conf. on Neural Information Processing. Shanghai, China, Nov 2011. 734−743
    [46] Mohammadi Z, Frounchi J, Amiri M. Wavelet-based emotion recognition system using EEG signal. Neural Computing and Applications, 2017, 28(8): 1985−1990 doi: 10.1007/s00521-015-2149-8
    [47] Duan R N, Zhu J Y, Lu B L. Differential entropy feature for EEG-based emotion classification. In: Proc. 6th Int'l IEEE/EMBS Conf. on Neural Engineering. San Diego, CA, 2013. 81−84
    [48] Moon S E, Jang S, Lee J S. Convolutional neural network approach for EEG-based emotion recognition using brain connectivity and its spatial information. In: Proc. IEEE Int'l Conf. on Acoustics, Speech and Signal Processing. Calgary, Canada, Apr 2018. 2556−2560
    [49] Yan X, Zheng W L, Liu W, Lu B L. Identifying gender differences in multimodal emotion recognition using bimodal deep autoencoder. In: Proc. 24th Int'l Conf. on Neural Information Processing, Guangzhou, China, Nov 2017. 533−542
    [50] Yan X, Zheng W L, Liu W, Lu B L. Investigating gender differences of brain areas in emotion recognition using LSTM neural network. In: Proc. 24th Int'l Conf. on Neural Information Processing. Guangzhou, China, Nov 2017. 820−829
    [51] Soroush M Z, Maghooli K, Setarehdan S K, Nasrabadi A M. Emotion recognition using EEG phase space dynamics and poincare intersections. Biomedical Signal Processing and Control, 2020, 59: 101918 doi: 10.1016/j.bspc.2020.101918
    [52] Shi L C, Lu B L. Off-line and on-line vigilance estimation based on linear dynamical system and manifold learning. In: Proc. 32nd Annual Int'l Conf. of the IEEE Engineering in Medicine and Biology Society. Buenos Aires, Argentina, Aug 2010. 6587−6590
    [53] Pham T D, Tran D, Ma W, Tran N T. Enhancing performance of EEG-based emotion recognition systems using feature smoothing. In: Proc. 22nd Int'l Conf. on Neural Information Processing. Istanbul, Turkey, Nov 2015. 95−102
    [54] Hu B, Li X, Sun S, Ratcliffe M. Attention recognition in EEG-based affective learning research using CFS+KNN algorithm. IEEE/ACM Trans. on Computational Biology and Bioinformatics, 2018, 15(1): 38−45 doi: 10.1109/TCBB.2016.2616395
    [55] Zheng W. Multi channel EEG-based emotion recognition via group sparse canonical correlation analysis. IEEE Trans. on Cognitive and Developmental Systems, 2017, 9(3): 281−290 doi: 10.1109/TCDS.2016.2587290
    [56] Ozerdem M S, Polat H. Emotion recognition based on EEG features in movie clips with channel selection. Brain Informatics, 2017, 4(4): 241−252 doi: 10.1007/s40708-017-0069-3
    [57] Picard R W, Vyzas E, Healey J. Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2001, 23(10): 1175−1191 doi: 10.1109/34.954607
    [58] Agrafioti F, Hatzinakos D, and Anderson A K. ECG pattern analysis for emotion detection. IEEE Trans. on Affective Computing, 2011, 3(1): 102−115
    [59] Goshvarpour A, Abbasi A, Goshvarpour A. An accurate emotion recognition system using ECG and GSR signals and matching pursuit method. Biomedical Journal, 2017, 40(6): 355−368 doi: 10.1016/j.bj.2017.11.001
    [60] Wu G, Liu G, Hao M. The analysis of emotion recognition from GSR based on PSO. In: Proc. Int'l Symposium on Intelligence Information Processing and Trusted Computing. Huanggang, China, Oct 2010. 360−363
    [61] Udovicic G, Derek J, Russo M, Sikora M. Wearable emotion recognition system based on GSR and PPG signals. In: Proc. of the 2nd Int'l Workshop on Multimedia for Personal Health and Health Care. Mountain View, CA, Oct 2017. 53−59
    [62] Lanata A, Valenza G, Scilingo E P. A novel EDA glove based on textile-integrated electrodes for affective computing. Medical and Biological Engineering and Computing, 2012, 50(11): 1163−1172 doi: 10.1007/s11517-012-0921-9
    [63] Jerritta S, Murugappan M, Wan K, Yaacob S. Emotion recognition from facial EMG signals using higher order statistics and principal component analysis. Journal of the Chinese Institute of Engineers, 2014, 37(3): 385−394 doi: 10.1080/02533839.2013.799946
    [64] Yang S, Yang G. Emotion recognition of EMG based on improved LM BP neural network and SVM. Journal of Software, 2011, 6(8): 1529−1536
    [65] Murugappan M. Electromyogram signal based human emotion classification using KNN and LDA. In: Proc. IEEE Int'l Conf. on System Engineering and Technology, Shah Alam, Malaysia, Jun 2011. 106−110
    [66] Goshvarpour A, Goshvarpour A. Poincares section analysis for PPG-based automatic emotion recognition. Chaos, Solitons and Fractals, 2018, 114: 400−407 doi: 10.1016/j.chaos.2018.07.035
    [67] Lee Y K, Kwon O W, Shin H S, Jo J, Lee Y. Noise reduction of PPG signals using a particle filter for robust emotion recognition. In: Proc. IEEE Int'l Conf. on Consumer Electronics-Berlin. Berlin, Germany, Sep 2011. 202−205
    [68] Kunecke J, Hildebrandt A, Recio G, Sommer W, Wilhelm O. Facial EMG responses to emotional expressions are related to emotion perception ability. PLOS ONE, 2014, 9(1): 84053 doi: 10.1371/journal.pone.0084053
    [69] McCubbin J A, Merritt M M, Sollers J J, Evans M K, Zonderman A B, Lane R D, Thayer J F. Cardiovascular emotional dampening: The relationship between blood pressure and recognition of emotion. Psychosomatic Medicine, 2011, 73(9): 743 doi: 10.1097/PSY.0b013e318235ed55
    [70] Quintana D S, Guastella A J, Outhred T, Hickie I B, Kemp A H. Heart rate variability is associated with emotion recognition: direct evidence for a relationship between the autonomic nervous system and social cognition. International Journal of Psychophysiology, 2012, 86(2): 168−172 doi: 10.1016/j.ijpsycho.2012.08.012
    [71] Williams D P, Cash C, Rankin C, Bernardi A, Koenig J, Thayer J F. Resting heart rate variability predicts self-reported difficulties in emotion regulation: a focus on different facets of emotion regulation. Frontiers in Psychology, 2015, 6: 261
    [72] Geisler F C, Vennewald N, Kubiak T, Weber H. The impact of heart rate variability on subjective well-being is mediated by emotion regulation. Personality and Individual Differences, 2010, 49(7): 723−728 doi: 10.1016/j.paid.2010.06.015
    [73] Guo H W, Huang Y S, Lin C H, Chien J C, Haraikawa K, Shieh J S. Heart rate variability signal features for emotion recognition by using principal component analysis and support vectors machine. In: Proc. IEEE 16th Int'l Conf. on Bioinformatics and Bioengineering. Taichung, China, Oct 2016. 274−277
    [74] Goshvarpour A, Abbasi A, Goshvarpour A. Fusion of heart rate variability and pulse rate variability for emotion recognition using lagged poincare plots. Australasian Physical and Engineering Sciences in Medicine, 2017, 40(3): 617−629 doi: 10.1007/s13246-017-0571-1
    [75] Zheng W L, Dong B N, Lu B L. Multimodal emotion recognition using EEG and eye tracking data. In: Proc. Int'l Conf. IEEE Engineering in Medicine and Biology Society. Chicago, IL, Aug 2014
    [76] Guo J J, Zhou R, Zhao L M, Lu B L. Multimodal emotion recognition from eye image, eye movement and EEG using deep neural networks. In: Proc. 41st Annual Int'l Conf. of the IEEE Engineering in Medicine and Biology Society. Berlin, Germany, Jul 2019. 3071−3074
    [77] Lu Y F, Zheng W L, Li B B, Lu B L. Combining eye movements and EEG to enhance emotion recognition. In: Proc. Int'l Joint Conf. on Artificial Intelligence. Buenos Aires, Argentina, Jul 2015
    [78] Wu D, Han X, Wang H, Wang R. Robust EEG-based emotion recognition using multi-feature joint sparse representation. In: Proc. Int'l Conf. on Computing, Networking and Communications. Big Island, HI, Feb 2020. 802−807
    [79] Thammasan N, Fukui K, Numao M. Multimodal fusion of EEG and musical features in music-emotion recognition. In: Proc. 21st AAAI Conf. on Artificial Intelligence. San Francisco, CA, Feb 2017. 4991−4992
    [80] Doma V, Pirouz M. A comparative analysis of machine learning methods for emotion recognition using EEG and peripheral physiological signals. Journal of Big Data, 2020, 7(1): 18 doi: 10.1186/s40537-020-00289-7
    [81] Pan S J, Yang Q. A survey on transfer learning. IEEE Trans. on Knowledge and Data Engineering, 2010, 22(10): 1345−1359 doi: 10.1109/TKDE.2009.191
    [82] Wu D, Xu Y, Lu B L. Transfer learning for EEG-based brain-computer interfaces: A review of progress made since 2016. IEEE Trans. on Cognitive and Developmental Systems, 2020, to be published
    [83] Pan S J, Tsang I W, Kwok J T, Yang Q. Domain adaptation via transfer component analysis. IEEE Trans. on Neural Networks, 2011, 22(2): 199−210 doi: 10.1109/TNN.2010.2091281
    [84] Sangineto E, Zen G, Ricci E, Sebe N. We are not all equal: Personalizing models for facial expression analysis with transductive parameter transfer. In: Proc. ACM Int'l Conf. on Multimedia. Orlando, FL, Nov 2014. 357−366
    [85] Zhang X, Liang W, Ding T, Pan J, Shen J, Huang X, Gao J. Individual similarity guided transfer modeling for EEG-based emotion recognition. In: Proc. IEEE Int'l Conf. on Bioinformatics and Biomedicine. San Diego, CA, Nov 2019. 1156−1161
    [86] Zhang X Y, Liu C L. Style transfer matrix learning for writer adaptation. In: Proc. 24th IEEE Conf. on Computer Vision and Pattern Recognition. Colorado Springs, CO, Jun 2011. 393−400
    [87] Zhang X Y, Liu C L. Writer adaptation with style transfer mapping. IEEE Trans. on Pattern Analysis and Machine Intelligence, 2012, 35(7): 1773−1787
    [88] Zheng W L, Lu B L. Personalizing EEG-based affective models with transfer learning. In: Proc. 25th Int'l Joint Conf. on Artificial Intelligence, New York, NY, Jul 2016. 2732−2739
    [89] Scholkopf B, Smola A J, Muller K. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 1998, 10(5): 1299−1319 doi: 10.1162/089976698300017467
    [90] Borgwardt K M, Gretton A, Rasch M J, Kriegel H, Scholkopf B, Smola A J. Integrating structured biological data by kernel maximum mean discrepancy. In: Proc. 14th Int'l Conf. on Intelligent Systems for Molecular Biology. Fortaleza, Brazil, Aug 2006. 49−57
    [91] Li J, Qiu S, Shen Y Y, Liu C L, He H. Multisource transfer learning for cross-subject EEG emotion recognition. IEEE Trans. on Cybernetics, 2019, 50(7): 3281−3293
    [92] Lan Z, Sourina O, Wang L, Scherer R, Muller G R. Domain adaptation techniques for EEG-based emotion recognition: A comparative study on two public datasets. IEEE Trans. on Cognitive and Developmental Systems, 2019, 11(1): 85−94 doi: 10.1109/TCDS.2018.2826840
    [93] 郑伟龙, 石振锋, 吕宝粮. 用异质迁移学习构建跨被试脑电情感模型. 计算机学报, 2020, 43(2): 177−189 doi: 10.11897/SP.J.1016.2020.00177

    Zheng Wei-Long, Shi Zhen-Feng, Lu Bao-Liang. Building cross-subject EEG-based affective models using heterogeneous transfer learning. Chuinese Journa of Computers, 2020, 43(2): 177−189 doi: 10.11897/SP.J.1016.2020.00177
    [94] Mehrabian A. Basic Dimensions for a General Psychological Theory: Implications for Personality, Social, Environmental, and Developmental Studies. Cambridge, MA: Oelgeschlager, Gunn & Hain, 1980
    [95] Grimm M, Kroschel K, Narayanan S S. The Vera Am Mittag German audio-visual emotional speech database. In: Proc. Int'l Conf. on Multimedia and Expo. Hannover, German, Jun 2008. 865−868
    [96] Bradley M M, Lang P J. The International Affective Digitized Sounds (2nd Edition; IADS-2): Affective Ratings of Sounds and Instruction Manual. Technical Report B-3, University of Florida, FL, 2007
    [97] Settles B. Active Learning Literature Survey. Technical Report, Department of Computer Sciences, University of Wisconsin−Madison, WI, 2009
    [98] Wu D, Parsons T D. Active class selection for arousal classification. In: Proc. 4th Int'l Conf. on Affective Computing and Intelligent Interaction. Memphis, TN, Oct 2011. 132−141
    [99] Wu D, Lance B J, Parsons T D. Collaborative filtering for brain-computer interaction using transfer learning and active class selection. PLOS ONE, 2013
    [100] Wu D, Lawhern V J, Gordon S, Lance B J, Lin C T. Offline EEG-based driver drowsiness estimation using enhanced batch-mode active learning (EBMAL) for regression. In: Proc. IEEE Int'l Conf. on Systems, Man and Cybernetics. Budapest, Hungary, Oct 2016. 730−736
    [101] Liu Z, Jiang X, Wu D. Unsupervised pool-based active learning for linear regression. Acta Automatica Sinica, 2020, to be published
    [102] Wu D. Pool-based sequential active learning for regression. IEEE Trans. on Neural Networks and Learning Systems, 2019, 30(5): 1348−1359 doi: 10.1109/TNNLS.2018.2868649
    [103] Wu D, Lin C T, Huang J. Active learning for regression using greedy sampling. Information Sciences, 2019, 474: 90−105 doi: 10.1016/j.ins.2018.09.060
    [104] Wu D, Huang J. Affect estimation in 3D space using multi-task active learning for regression. IEEE Trans. on Affective Computing, 2020, to be published
    [105] Zheng W, Zhu J, Peng Y, Lu B. EEG-based emotion classification using deep belief networks. In: Proc. IEEE Int'l Conf. on Multimedia and Expo. Chengdu, China, Jul 2014. 1−6
    [106] Thammasan N, Fukui K, Numao M. Application of deep belief networks in EEG-based dynamic music-emotion recognition. In: Proc. Int'l Joint Conf. on Neural Networks. Vancouver, Canada, Jul 2016. 881−888
    [107] Li J, Zhang Z, He H. Hierarchical convolutional neural networks for EEG-based emotion recognition. Cognitive Computation, 2018, 10(2): 368−380 doi: 10.1007/s12559-017-9533-x
    [108] Wei C, Chen L, Song Z, Lou X, Li D. EEG-based emotion recognition using simple recurrent units network and ensemble learning. Biomedical Signal Processing and Control, 2020, 58: 101756 doi: 10.1016/j.bspc.2019.101756
    [109] Yin Z, Zhao M, Wang Y, Yang J, Zhang J. Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Computer Methods and Mrograms in Biomedicine, 2017, 140: 93−110 doi: 10.1016/j.cmpb.2016.12.005
    [110] Fourati R, Ammar B, Aouiti C, Medina J S, Alimi A M. Optimized echo state network with intrinsic plasticity for EEG-based emotion recognition. In: Proc. Int'l Conf. on Neural Information Processing. Guangzhou, China, Nov 2017. 718−727
    [111] Ren F, Dong Y, Wang W. Emotion recognition based on physiological signals using brain asymmetry index and echo state network. Neural Computing and Applications, 2019, 31(9): 4491−4501 doi: 10.1007/s00521-018-3664-1
    [112] Liu Y, Ding Y, Li C, Cheng J, Song R, Wan F, Chen X. Multi-channel EEG-based emotion recognition via a multi-level features guided capsule network. Computers in Biology and Medicine, 2020, 123: 103927 doi: 10.1016/j.compbiomed.2020.103927
    [113] Wu X, Zheng W L, Lu B L. Identifying functional brain connectivity patterns for EEG-based emotion recognition. In: Proc. 9th Int'l IEEE/EMBS Conf. on Neural Engineering. San Francisco, CA, Mar 2019. 235−238
    [114] Yang Y, Wu Q M, Zheng W L, Lu B L. EEG-based emotion recognition using hierarchical network with subnetwork nodes. IEEE Trans. on Cognitive and Developmental Systems, 2018, 10(2): 408−419 doi: 10.1109/TCDS.2017.2685338
    [115] Wang Y, Qiu S, Li J, Ma X, Liang Z, Li H, He H. EEG-based emotion recognition with similarity learning network. In: Proc. 41st Annual Int'l Conf. of the IEEE Engineering in Medicine and Biology Society. Berlin, Germany, Jul 2019. 1209−1212
    [116] Zhang T, Cui Z, Xu C, Zheng W, Yang J. Variational pathway reasoning for EEG emotion recognition. In: Proc. 34th AAAI Conf. on Artificial Intelligence. New York, NY, Feb 2020. 2709−2716
    [117] Cimtay Y, Ekmekcioglu E. Investigating the use of pretrained convolutional neural network on cross-subject and cross-dataset EEG emotion recognition. Sensors, 2020, 20(7): 2034 doi: 10.3390/s20072034
    [118] Ma J, Tang H, Zheng W L, Lu B L. Emotion recognition using multimodal residual LSTM network. In: Proc. 27th ACM Int'l Conf. on Multimedia. New York, NY, Oct 2019. 176−183
    [119] Liu W, Qiu J L, Zheng W L, Lu B L. Multimodal emotion recognition using deep canonical correlation analysis. arXiv preprint arXiv: 1908.05349, 2019
    [120] Rayatdoost S, Rudrauf D, Soleymani M. Expression-guided EEG representation learning for emotion recognition. In: Proc. IEEE Int'l Conf. on Acoustics, Speech and Signal Processing. Barcelona, Spain, May 2020. 3222−3226
    [121] Du C, Wang H, Li J, Zheng W L, Lu B L, He H. Semi-supervised deep generative modelling of incomplete multi-modality emotional data. In: Proc. ACM Int'l Conf. on Multimedia. Seoul, Korea, Oct 2018. 108−116
    [122] Long M, Cao Y, Wang J, Jordan M. Learning transferable features with deep adaptation networks. In: Proc. Int'l Conf. on Machine Learning. Lille, France, Jul 2015. 97−105
    [123] Gretton A, Sriperumbudur B K, Sejdinovic D, Strathmann H, Balakrishnan S, Pontil M, Fukumizu K. Optimal kernel choice for large-scale two-sample tests. In: Proc. Advances in Neural Information Processing Systems. Lake Tahoe, NV, Dec 2012. 1214−1222
    [124] Li H, Jin Y M, Zheng W L, Lu B L. Cross-subject emotion recognition using deep adaptation networks. In: Proc. Int'l Conf. on Neural Information Processing. Siem Reap, Cambodia, Dec 2018. 403−413
    [125] Long M, Wang J, Ding G, Sun J, Yu P S. Transfer feature learning with joint distribution adaptation. In: Proc. IEEE Int'l Conf. on Computer Vision. Sydney, Australia, Dec 2013. 2200−2207
    [126] Li J, Qiu S, Du C, Wang Y, He H. Domain adaptation for EEG emotion recognition based on latent representation similarity. IEEE Trans. on Cognitive and Developmental Systems, 2020, 12(2): 344−353 doi: 10.1109/TCDS.2019.2949306
    [127] Tzeng E, Hoffman J, Saenko K, Darrell T. Adversarial discriminative domain adaptation. In: Proc IEEE Conf. on Computer Vision and Pattern Recognition. Honolulu, HI, Jul 2017. 2962−2971
    [128] Luo Y, Zhang S Y, Zheng W L, Lu B L. WGAN domain adaptation for EEG-based emotion recognition. In: Proc. Int'l Conf. on Neural Information Processing. Siem Reap, Cambodia, Dec 2018. 275−286
    [129] Arjovsky M, Chintala S, Bottou L. Wasserstein GAN. arXiv preprint arXiv: 1701.07875, 2017
    [130] Ma B Q, Li H, Zheng W L, Lu B L. Reducing the subject variability of EEG signals with adversarial domain generalization. In: Proc. Int'l Conf. on Neural Information Processing. Sydney, Australia, Dec 2019. 30−42
    [131] Ganin Y, Ustinova E, Ajakan H, Germain P, Larochelle H, Laviolette F, Marchand M, Lempitsky V. Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 2016, 17: 2096−2030
    [132] Li Y, Zheng W, Cui Z, Zhang T, Zong Y. A novel neural network model based on cerebral hemispheric asymmetry for EEG emotion recognition. In: Proc. 27th Int'l Joint Conf. on Artificial Intelligence. Stockholm, Sweden, Jul 2018. 1561−1567
    [133] Soleymani M, Asghari S, Fu Y, Pantic M. Analysis of EEG signals and facial expressions for continuous emotion detection. IEEE Trans. on Affective Computing, 2016, 7(1): 17−28 doi: 10.1109/TAFFC.2015.2436926
    [134] Nath D, Anubhav, Singh M, Sethia D, Kalra D, Indu S. A comparative study of subject-dependent and subject-independent strategies for EEG-based emotion recognition using LSTM network. In: Proc. 4th Int'l Conf. on Compute and Data Analysis. Silicon Valley, CA, Mar 2020. 142−147
    [135] Zhao S, Ding G, Han J, Gao Y. Personality-aware personalized emotion recognition from physiological signals. In: Proc. Proc. 27th Int'l Joint Conf. on Artificial Intelligence. Stockholm, Sweden, Jul 2018. 1660−1667
    [136] Zhao S, Gholaminejad A, Ding G, Gao Y, Han J, Keutzer K. Personalized emotion recognition by personality-aware high-order learning of physiological signals. ACM Trans. on Multimedia Computing, Communications, and Applications, 2019, 15(1): 14:1−14:18
    [137] Song T, Liu S, Zheng W, Zong Y, Cui Z. Instance-adaptive graph for EEG emotion recognition. In: Proc. 34th AAAI Conf. on Artificial Intelligence. New York, NY, Feb 2020. 2701−2708
    [138] Song T, Zheng W, Song P, Cui Z. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans. on Affective Computing, 2020, 11(3): 532−541 doi: 10.1109/TAFFC.2018.2817622
    [139] Agarwal A, Dowsley R, McKinney N, Wu D, Lin C T, Cock M D, Nascimento A. Protecting privacy of users in brain-computer interface applications. IEEE Trans. on Neural Systems and Rehabilitation Engineering, 2019, 27(8): 1546−1555 doi: 10.1109/TNSRE.2019.2926965
    [140] Cock M D, Dowsley R, McKinney N, Nascimento A C, Wu D. Privacy preserving machine learning with EEG data. In: Proc. 34th Int'l Conf. on Machine Learning. Sydney, Australia, Aug 2017
  • 加载中
计量
  • 文章访问数:  92
  • HTML全文浏览量:  70
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-09-22
  • 录用日期:  2020-12-31
  • 网络出版日期:  2021-02-01

目录

    /

    返回文章
    返回