2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于流形正则化框架和MMD的域自适应BLS模型

赵慧敏 郑建杰 郭晨 邓武

赵慧敏, 郑建杰, 郭晨, 邓武. 基于流形正则化框架和MMD的域自适应BLS模型. 自动化学报, 2021, x(x): 1−14 doi: 10.16383/j.aas.c210009
引用本文: 赵慧敏, 郑建杰, 郭晨, 邓武. 基于流形正则化框架和MMD的域自适应BLS模型. 自动化学报, 2021, x(x): 1−14 doi: 10.16383/j.aas.c210009
Zhao Hui-Min, Heng Jian-Jie, Guo Chen, Deng Wu. Domain adaptive BLS model based on manifold regularization framework and MMD. Acta Automatica Sinica, 2021, x(x): 1−14 doi: 10.16383/j.aas.c210009
Citation: Zhao Hui-Min, Heng Jian-Jie, Guo Chen, Deng Wu. Domain adaptive BLS model based on manifold regularization framework and MMD. Acta Automatica Sinica, 2021, x(x): 1−14 doi: 10.16383/j.aas.c210009

基于流形正则化框架和MMD的域自适应BLS模型

doi: 10.16383/j.aas.c210009
基金项目: 国家自然科学基金(61771087, 51879027), 中国民航大学科研启动基金(2020KYQD123)
详细信息
    作者简介:

    赵慧敏:中国民航大学电子信息与自动化学院教授, 硕士生导师. 主要研究方向为智能控制与信息处理、深度学习与智能优化、智能诊断与性能评估. E-mail: hm_zhao1977@126.com

    郑建杰:首都师范大学心理学院博士生. 主要研究方向为宽度学习系统与图像处理、人类脑图谱的构建和应用. E-mail: zheng853796151@126.com

    郭晨:大连海事大学船舶电气工程学院教授, 博士生导师. 主要研究方向为船舶自动控制系统、智能控制理论与应用、虚拟现实技术及应用. E-mail: dmuguoc@126.com

    邓武:中国民航大学电子信息与自动化学院教授, 博士生导师. 主要研究方向为智能优化与资源调度、深度学习与智能诊断, 本文通信作者. E-mail: dw7689@163.com

Domain Adaptive BLS Model Based on Manifold Regularization Framework and MMD

Funds: Supported by National Natural Science Foundation of P. R. China (61771087, 51879027), and the Research Foundation for Civil Aviation University of China (2020KYQD123).
More Information
    Author Bio:

    ZHAO Hui-Min Professor at the College of Electronic Information and Automation, Civil Aviation University of China. Her research interest includes Intelligent control and information processing, deep learning and intelligent optimization, intelligent diagnosis and performance evaluation

    ZHENG Jian-Jie Doctor at the College of School of Psychology, Capital Normal University. His research interest includes broad learning system and image processing, construction and application of human brain atlas

    GUO Chen Professor at the School of Marine Electrical Engineering, Dalian Maritime University. His research interest includes ship automatic control system, intelligent control theory and application, virtual reality technology and application

    DENG Wu Professor at the College of Electronic Information and Automation, Civil Aviation University of China. His research interest includes intelligent optimization and resource scheduling, deep learning and intelligent diagnosis. Corresponding author of this paper

  • 摘要: 宽度学习系统(Broad learning system, BLS)作为一种基于随机向量函数型网络(Random vector functional link network, RVFLN)的高效增量学习系统, 具有快速自适应模型结构选择能力和高精度的特点. 但针对目标分类任务中有标签数据匮乏问题, 传统的BLS难以借助相关领域知识来提升目标域的分类效果, 为此本文提出一种基于流形正则化框架和最大均值差异(Maximum mean discrepancy, MMD)的域适应BLS(DABLS)模型, 实现目标域无标签条件下的跨域图像分类. DABLS模型首先构造BLS的特征节点和增强节点, 从源域和目标域数据中有效提取特征; 再利用流形正则化框架构造拉普拉斯矩阵, 以探索目标域数据中的流形特性, 挖掘目标域数据的潜在信息. 接着基于迁移学习方法构建源域数据与目标域数据之间的MMD惩罚项, 以匹配源域和目标域之间的投影均值; 将特征节点、增强节点、MMD惩罚项和目标域拉普拉斯矩阵相结合, 构造目标函数, 并采用岭回归分析法对其求解, 获得输出系数, 从而提高模型的跨域分类性能. 最后在不同图像数据集上进行大量的验证与对比实验, 结果表明DABLS在不同图像数据集上均能获得较好的跨域分类性能, 具有较强的泛化能力和较好的稳定性.
  • 图  1  BLS的结构示意图

    Fig.  1  Structure diagram of BLS

    图  2  DABLS模型的算法流程

    Fig.  2  Algorithm flow of DABLS model

    图  3  第一行显示Office和Caltech256数据集, 第二行显示MNIST, USPS和COIL20数据集(从左到右)

    Fig.  3  The first row shows office and caltech256 datasets, and the second row shows MNIST, USPS and COIL20 datasets (from left to right)

    图  4  显示ImageNet和 VOC2007数据集

    Fig.  4  Display ImageNet and VOC207 datasets

    表  1  数据集的详细描述

    Table  1  Detailed description of dataset

    数据集样本数目特征维数类别子集
    USPS180025610U
    MNIST200025610M
    COIL201440102420CO1, CO2
    Office141080010A, W, D
    Caltech256112380010C
    ImageNet734140965I
    VOC2007337640965V
    下载: 导出CSV

    表  2  不同节点数下DABLS的实验结果

    Table  2  Experimental results of DABLS with different number of nodes

    特征
    节点
    增强
    节点
    M→UU→M
    精度(%)时间(s)STD精度(%)时间(s)STD
    10050060.4119.011.2545.2520.501.84
    100100063.9419.591.6147.5722.462.14
    100150066.9821.561.4948.0526.162.27
    100200068.5423.981.6750.1327.271.53
    100250068.7727.151.7250.2529.271.64
    100300069.2233.12.1.8150.3732.831.78
    100350069.3137.261.8949.9136.251.92
    100400068.9540.961.7549.6341.171.82
    150200068.3625.031.8749.4927.921.72
    200200067.5325.912.0150.0828.721.79
    300200066.5726.412.0349.9229.621.87
    400200064.3526.973.3748.5930.581.95
    500200062.9627.494.6149.3831.411.64
    下载: 导出CSV

    表  3  从源域到目标域的平均分类精度(%)

    Table  3  Average classification accuracy from source domain to target domain (%)

    任务BLSSS-BLSTCACDELMCD-CDBNDABLS
    M→U25.3459.6754.2853.2750.2768.54
    U→ M20.2529.8452.0039.8041.6550.13
    平均值22.7944.7553.1446.5345.9659.33
    下载: 导出CSV

    表  4  从源域到目标域的平均训练时间(s)

    Table  4  Average training time from source domain to target domain (s)

    任务/方法BLSSS-BLSTCACDELMCD-CDBNDABLS
    M→U0.6912.7527.1424.77548.4723.98
    U→M0.5813.1822.3623.63536.9522.27
    平均值0.6412.9724.7524.20542.7123.13
    下载: 导出CSV

    表  5  不同节点数下DABLS的实验结果

    Table  5  Experimental results of DABLS with different number of nodes

    特征
    节点
    增强
    节点
    CO1→CO2CO2→CO1
    精度(%)时间(s)STD精度(%)时间(s)STD
    10050085.432.591.8282.702.831.61
    100100086.403.171.8083.293.351.71
    100150086.833.651.3283.833.761.52
    100200087.334.501.2784.044.021.46
    100250087.694.931.3084.414.851.32
    100300088.185.851.1284.725.021.28
    100350088.266.541.1384.165.811.21
    100400087.557.611.1783.766.101.06
    150300087.556.021.1583.616.211.21
    200300087.096.151.1682.956.381.63
    300300085.926.641.1581.526.621.18
    400300084.087.241.2080.486.791.20
    500300082.297.421.0279.517.041.11
    下载: 导出CSV

    表  6  从源域到目标域的平均分类精度(%)

    Table  6  Average classification accuracy from source domain to target domain (%)

    任务/方法BLSSS-BLSTCACDELMCD-CDBNDABLS
    CO1→CO282.2583.7588.6181.6684.6788.12
    CO2→CO180.6381.1786.3380.1980.7484.72
    平均值81.4482.4687.4780.9382.7086.42
    下载: 导出CSV

    表  7  从源域到目标域的平均训练时间(s)

    Table  7  Average training time from source domain to target domain (s)

    任务/方法BLSSS-BLSTCACDELMCD-CDBNDABLS
    CO1→CO21.383.2219.985.87125.335.35
    CO2→CO10.852.9514.675.48136.785.33
    平均值1.123.0917.325.68131.055.34
    下载: 导出CSV

    表  8  不同节点数下DABLS的实验结果

    Table  8  Experimental results of DABLS with different number of nodes

    特征
    节点
    增强
    节点
    A→CW→C
    精度(%)时间(s)STD精度(%)时间(s)STD
    5005043.638.810.7534.696.010.35
    50010043.738.860.5335.246.240.91
    50015043.828.920.7735.536.500.65
    50020043.929.010.6235.616.780.71
    50030043.989.180.8535.936.930.92
    50040044.029.530.8736.277.210.77
    50050044.299.770.4436.507.430.70
    50060043.509.980.6136.527.720.93
    50080043.3610.520.5936.218.010.64
    500100043.1810.900.8236.048.390.33
    10050042.018.720.7632.396.480.91
    20050042.369.020.8333.956.660.99
    30050042.839.300.6534.276.870.92
    40050043.599.530.6936.117.120.77
    60050044.2510.440.7836.177.850.82
    80050043.4310.750.6535.778.030.69
    下载: 导出CSV

    表  9  从源域到目标域的平均分类精度(%)

    Table  9  Average classification accuracy from source domain to target domain (%)

    任务/方法BLSSS-BLSTCACDELMCD-CDBNDABLS
    A$ \to $C20.8242.1640.7831.6735.5644.29
    A$ \to $D17.8339.4031.8532.4833.7942.06
    A$ \to $W19.6140.6137.6331.47)27.4642.09
    C$ \to $A29.1649.6744.8944.9938.7851.68
    C$ \to $D24.8444.2045.8435.3736.9445.85
    C$ \to $W20.4645.7436.6138.9235.5447.79
    D$ \to $A32.4235.5731.5230.6128.3436.73
    D$ \to $C30.0330.1932.5028.9626.7932.47
    D$ \to $W79.9879.11)87.1276.9550.7880.06
    W$ \to $A34.6137.5130.6935.5530.8940.01
    W$ \to $C31.7335.2927.1632.0327.2636.50
    W$ \to $D80.8180.8990.4578.9950.4282.73
    平均值35.1946.6945.3841.5035.2148.52
    下载: 导出CSV

    表  10  从源域到目标域的平均训练时间(s)

    Table  10  Average training time from source domain to target domain (s)

    任务/方法BLSSS-BLSTCACDELMCD-CDBNDABLS
    A$ \to $C0.785.3936.696.8346.369.77
    A$ \to $D0.670.8311.731.2933.282.76
    A$ \to $W0.640.9913.651.5234.232.91
    C$ \to $A0.683.8635.715.7441.448.18
    C$ \to $D0.710.9315.051.5835.863.71
    C$ \to $W0.671.0916.751.5938.373.63
    D$ \to $A0.593.5811.824.1420.125.50
    D$ \to $C0.545.1916.216.1426.637.45
    D$ \to $W0.550.793.530.7320.481.17
    W$ \to $A0.583.4314.294.3345.535.93
    W$ \to $C0.575.1719.056.1256.287.43
    W$ \to $D0.520.693.220.6612.401.02
    平均值0.622.6616.483.3930.084.96
    下载: 导出CSV

    表  11  不同节点数下DABLS的实验结果

    Table  11  Experimental results of DABLS with different number of nodes

    特征
    节点
    增强
    节点
    V→II→V
    精度(%)时间(s)STD精度(%)时间(s)STD
    10060074.1335.251.0566.6116.621.31
    15060075.9437.990.7566.8319.240.77
    20060076.5342.290.5967.5122.420.62
    25060077.8744.510.3367.8326.030.41
    30060077.6547.640.2967.8128.960.43
    40060076.8753.890.3667.2431.620.45
    50060075.6757.130.1967.0235.370.39
    60060075.2061.290.2766.8638.690.37
    25020074.2926.560.7366.3214.730.87
    25040077.2131.590.6767.1317.330.54
    25080077.2346.210.5667.6924.090.42
    250100076.8352.680.6567.5227.550.34
    250150074.9365.710.4667.4236.430.31
    250200073.8683.610.3667.3343.530.33
    250250071.77103.380.3166.7350.920.26
    下载: 导出CSV

    表  12  从源域到目标域的平均分类精度(%)

    Table  12  Average classification accuracy from source domain to target domain (%)

    任务/方法BLSSS-BLSTCACDELMCD-CDBNDABLS
    V → I76.2277.0373.7976.8276.0277.87
    I → V66.3267.1364.3466.8567.1967.83
    平均值71.2772.0869.0671.8371.6072.85
    下载: 导出CSV

    表  13  从源域到目标域的平均训练时间(s)

    Table  13  Average training time from source domain to target domain(s)

    任务/方法BLSSS-BLSTCACDELMCD-CDBNDABLS
    V → I6.3737.6952.1555.041039.0344.51
    I → V3.6217.3230.4233.39823.3526.03
    平均值4.9627.5041.2844.25931.1935.27
    下载: 导出CSV
  • [1] Salakhutdinov R, Hinton G. An efficient learning procedure for deep boltzmann machine. Neural Computation, 2014, 24(8): 1967−2006
    [2] 林景栋, 吴欣怡, 柴毅, 尹宏鹏. 卷积神经网络结构优化综述. 自动化学报, 2020, 46(1): 24−37

    Lin Jing-Dong, Wu Xin-Yi, Chai Yi, Yin Hong-Peng. Structure optimization of convolutional neural networks: A survey. Acta Automatica Sinica, 2020, 46(1): 24−37
    [3] Hinton G E, Osinderos T YW. A fast learning algorithm for deep belief nets. Neural Computation, 2006, 18(7): 1527−1554 doi: 10.1162/neco.2006.18.7.1527
    [4] 刘建伟, 谢浩杰, 罗雄麟. 生成对抗网络在各领域应用研究进展. 自动化学报, 2020, 46(12): 2500−2536

    Liu Jian-Wei, Xie Hao-Jie, Luo Xiong-Lin. Research progress on application of generative adversarial networks in various fields. Acta Automatica Sinica, 2020, 46(12): 2500−2536
    [5] Feng L, Chen Z, Jie W. Video image target monitoring based on RNN-LSTM. Multimedia Tools Applications, 2018, 78(2): 1−18
    [6] Chen C L P, Liu Z. Broad learning system: An effective and efficient incremental learning system without the need for deep architecture. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(1): 10−24 doi: 10.1109/TNNLS.2017.2716952
    [7] Chen C L P, Liu Z, Feng S. Universal approximation capability of broad learning system and its structural variations. IEEE Transactions on Neural Networksand Learning Systems, 2019, 30(4): 1191−1204 doi: 10.1109/TNNLS.2018.2866622
    [8] Jin J W, Philip Chen C L. Regularized robust broad learning system for uncertain data modeling. Neurocomputing, 2018, 322(1): 58−69
    [9] 杨刚, 陈鹏, 戴丽珍, 杨辉. 一种基于池计算的宽度学习系统. 控制与决策, 2020, Doi: 10.13195/j.kzyjc.2019.1729.

    Yang Gang, Chen Peng, Dai Li-Zhen, Yang Hui. A broad learning system based on reservoir computing. Control and Decision, 2020, doi: 10.13195/j.kzyjc.2019.1729.
    [10] 邹伟东, 夏元清. 基于压缩因子的宽度学习系统的虚拟机性能预测. 自动化学报, 2019, Doi: 10.16383/j.aas.c190307.

    Zou Wei-Dong, Xia Yuan-Qing. Virtual machine performance prediction using broad learning system based on compression factor. Acta Automatica Sinica, 2019, Doi: 10.16383/j.aas.c190307.
    [11] Kong Y, Wang X, Cheng Y, Chen C. Hyperspectral imagery classification based on semi-supervised broad learning system. Remote Sensing, 2018, 10(5): 685 doi: 10.3390/rs10050685
    [12] 郑云飞, 陈霸东. 基于最小p-范数的宽度学习系统. 模式识别与人工智能, 2019, 32(1): 51−57

    Zheng Yun-Fei, Chen Ba-Dong. Least p-Norm based broad learning system. Pattern Recognition and Artificial Intelligence, 2019, 32(1): 51−57
    [13] Lin J, Liu Z, Chen C L P, Zhang Y. Quaternion broad learning system: a novel multi-dimensional filter for estimation and elimination tremor in teleoperation. Neurocomputing, 2020, 380: 78−86 doi: 10.1016/j.neucom.2019.10.059
    [14] Han M, Feng S B, Chen C L P, et al. Structured manifold broad learning system: A manifold perspective for large-scale chaotic time series analysis and prediction. IEEE Transactions on Knowledge and Data Engineering, 2019, 31(9): 1809−1821 doi: 10.1109/TKDE.2018.2866149
    [15] Fan J H, Wang X, Wang X X, et al. Incremental wishart broad learning system for fast polsar image classification. IEEE Geoscience and Remote Sensing Letters, 2019, 16(12): 1854−1858 doi: 10.1109/LGRS.2019.2913999
    [16] Feng S, ChenN C L P. Fuzzy Broad Learning System: A novel neuro-fuzzy model for regression and classification. IEEE Transactions on Cybernetics, 2020, 50(2): 414−424 doi: 10.1109/TCYB.2018.2857815
    [17] Xu M L, Han M, Chen C L P, et al. Recurrent broad learning systems for time series prediction. IEEE Transactions on Cybernetics, 2020, 50(4): 1405−1417 doi: 10.1109/TCYB.2018.2863020
    [18] Chu F, Liang T, Chen C L P, et al. Weighted broad learning system and its application in nonlinear industrial process modeling. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(8): 3017−3031 doi: 10.1109/TNNLS.2019.2935033
    [19] Liu Y, Wang Y F, Chen L, et al. Incremental Bayesian broad learning system and its industrial application. Artificial Intelligence Review, 2020, Doi: 10.1007/s10462-020-09929-z.
    [20] Shi Z H, Chen X M, Zhao C M, et al. Multi-view broad learning system for primate oculomotor decision decoding. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(9): 1908−1920 doi: 10.1109/TNSRE.2020.3003342
    [21] Zhao H M, Zheng J J, Deng W, et al. Semi-supervised broad learning system based on manifold regularization and broad network. IEEE Transactions on Circuits and Systems--I: Regular Papers, 2020, 67(3): 983−994 doi: 10.1109/TCSI.2019.2959886
    [22] Li S, Song S J, Huang G, et al. Cross-domain extreme learning machines for domain adaptation. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018, 49(6): 2168−2216
    [23] Pan S J, Tsang I W, Kwok J T, et al. Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks, 2021, 22(2): 199−210
    [24] Huang G B. Learning hierarchical representations for face verification with convolutional deep belief networks. 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), . 2012: 2518−2525.
  • 加载中
计量
  • 文章访问数:  264
  • HTML全文浏览量:  119
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-01-04
  • 录用日期:  2021-05-12
  • 网络出版日期:  2021-06-28

目录

    /

    返回文章
    返回