2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

一种随机配置网络的模型与数据混合并行学习方法

代伟 李德鹏 杨春雨 马小平

代伟, 李德鹏, 杨春雨, 马小平. 一种随机配置网络的模型与数据混合并行学习方法. 自动化学报, 2021, 47(10): 2427−2437 doi: 10.16383/j.aas.c190411
引用本文: 代伟, 李德鹏, 杨春雨, 马小平. 一种随机配置网络的模型与数据混合并行学习方法. 自动化学报, 2021, 47(10): 2427−2437 doi: 10.16383/j.aas.c190411
Dai Wei, Li De-Peng, Yang Chun-Yu, Ma Xiao-Ping. A model and data hybrid parallel learning method for stochastic configuration networks. Acta Automatica Sinica, 2021, 47(10): 2427−2437 doi: 10.16383/j.aas.c190411
Citation: Dai Wei, Li De-Peng, Yang Chun-Yu, Ma Xiao-Ping. A model and data hybrid parallel learning method for stochastic configuration networks. Acta Automatica Sinica, 2021, 47(10): 2427−2437 doi: 10.16383/j.aas.c190411

一种随机配置网络的模型与数据混合并行学习方法

doi: 10.16383/j.aas.c190411
基金项目: 国家自然科学基金(61603393, 61973306), 江苏省自然科学基金(BK20160275), 中国博士后科学基金(2018T110571), 流程工业综合自动化国家重点实验室开放基金资助(PAL-N201706)
详细信息
    作者简介:

    代伟:中国矿业大学信息与控制工程学院副教授. 主要研究方向为复杂工业过程建模, 运行优化与控制. 本文通信作者. E-mail: weidai@cumt.edu.cn

    李德鹏:中国矿业大学信息与控制工程学院硕士研究生. 主要研究方向为数据驱动建模,机器学习算法. E-mail: dpli@cumt.edu.cn

    杨春雨:中国矿业大学信息与控制工程学院教授. 于2009年获得东北大学博士学位. 主要研究方向为广义系统,鲁棒控制. E-mail: chunyuyang@cumt.edu.cn

    马小平:中国矿业大学信息与控制工程学院教授. 主要研究方向为过程控制,网络控制,故障诊断. E-mail: xpma@cumt.edu.cn

A Model and Data Hybrid Parallel Learning Method for Stochastic Configuration Networks

Funds: Supported by National Natural Science Foundation of China (61603393, 61973306), Natural Science Foundation of Jiangsu Provinces (BK20160275), the Postdoctoral Science Foundation of China (2018T110571), State Key Laboratory of Synthetical Automation for Process Industries (PAL-N201706)
More Information
    Author Bio:

    DAI Wei Associate professor at the School of Information and control Engineering, China University of Mining and Technology. His research interest covers modeling, operational optimization, and control for complex industrial process. Corresponding author of this paper

    LI De-Peng Master student at the School of Information and control Engineering, China University of Mining and Technology. His research interest covers data-driving modeling and machine learning algorithms

    YANG Chun-Yu Professor at the School of Information and Control Engineering, China University of Mining and Technology. He received his Ph.D. degree from Northeastern University in 2009. His research interest covers descriptor systems and robust control

    MA Xiao-Ping Professor at the School of Information and Control Engineering, China University of Mining and Technology. His research interest covers process control, networked control, and fault detection

  • 摘要: 随机配置网络(Stochastic configuration networks, SCNs)在增量构建过程引入监督机制来分配隐含层参数以确保其无限逼近特性, 具有易于实现、收敛速度快、泛化性能好等优点. 然而, 随着数据量的不断扩大, SCNs的建模任务面临一定的挑战性. 为了提高神经网络算法在大数据建模中的综合性能, 本文提出了一种混合并行随机配置网络(Hybrid parallel stochastic configuration networks, HPSCNs)架构, 即: 模型与数据混合并行的增量学习方法. 所提方法由不同构建方式的左右两个SCNs模型组成, 以快速准确地确定最佳隐含层节点, 其中左侧采用点增量网络(PSCN), 右侧采用块增量网络(BSCN); 同时每个模型建立样本数据的动态分块方法, 从而加快候选“节点池”的建立、降低计算量. 所提方法首先通过大规模基准数据集进行了对比实验, 然后应用在一个实际工业案例上, 表明其有效性.
  • 图  1  模型并行结构图

    Fig.  1  The structure diagram of model parallelism

    图  2  数据并行策略

    Fig.  2  Strategy of data parallelism

    图  3  不同算法综合性能比较

    Fig.  3  Comparison of comprehensive performance of different algorithms

    图  4  模型的收敛曲线

    Fig.  4  Convergence curve of HPSCNs

    图  5  模型的逼近特性

    Fig.  5  Approximation performance of HPSCNs

    表  1  基准数据集说明

    Table  1  Specification of benchmark data sets

    数据集属性样本数
    输入变量输出变量
    DB1144241 600
    DB212110 000
    DB310140 768
    DB426114 998
    下载: 导出CSV

    表  2  分块数递增区间长度及其上下界

    Table  2  Incremental interval length of block number and its upper and lower bounds

    $L_{en}^k$$L_{\max }^k$$L_{\min }^k$
    50500
    10015050
    150300150
    ·········
    下载: 导出CSV

    表  3  不同算法性能比较

    Table  3  Performance comparison of different algorithms

    数据集算法t(s)kL
    DB1SC-III24.35$\pm $1.69164.40$\pm $7.76164.40$\pm $7.76
    ${\rm{BSC - }}{{\rm{I}}_3}$12.60$\pm $1.2169.20$\pm $3.03207.60$\pm $9.09
    ${\rm{BSC - }}{{\rm{I}}_5}$9.41$\pm $1.3344.00$\pm $3.24220.00$\pm $16.20
    ${\rm{HPSCN}}_1^1$3.48$\pm $0.38122.40$\pm $8.02122.40$\pm $8.02
    ${\rm{HPSCN}}_3^1$3.03$\pm $0.2863.40$\pm $4.16162.80$\pm $7.90
    ${\rm{HPSCN}}_5^1$2.96$\pm $0.1945.00$\pm $2.83215.00$\pm $9.71
    DB2SC-III26.97$\pm $2.54300.00$\pm $14.18300.00$\pm $14.18
    ${\rm{BSC - }}{{\rm{I}}_3}$14.66$\pm $1.33120.40$\pm $3.98361.20$\pm $11.93
    ${\rm{BSC - }}{{\rm{I}}_5}$11.01$\pm $1.0778.80$\pm $2.91394.00$\pm $14.87
    ${\rm{HPSCN}}_1^1$7.22$\pm $0.95239.30$\pm $14.55239.3$\pm $14.55
    ${\rm{HPSCN}}_3^1$5.47$\pm $0.33123.50$\pm $3.34301.90$\pm $10.99
    ${\rm{HPSCN}}_5^1$4.39$\pm $0.4281.80$\pm $3.74378.60$\pm $16.54
    DB3SC-III18.04$\pm $2.15106.60$\pm $3.36106.60$\pm $3.36
    ${\rm{BSC - }}{{\rm{I}}_3}$8.96$\pm $1.2139.80$\pm $2.28119.40$\pm $6.84
    ${\rm{BSC - }}{{\rm{I}}_5}$6.81$\pm $0.5525.20$\pm $1.10126.00$\pm $5.48
    ${\rm{HPSCN}}_1^1$3.45$\pm $0.2497.00$\pm $2.6597.00$\pm $2.65
    ${\rm{HPSCN}}_3^1$2.05$\pm $0.1341.20$\pm $2.17106.40$\pm $4.39
    ${\rm{HPSCN}}_5^1$1.88$\pm $0.1225.00$\pm $1.22121.00$\pm $6.44
    DB4SC-III9.16$\pm $0.34161.20$\pm $2.56161.20$\pm $2.56
    ${\rm{BSC - }}{{\rm{I}}_3}$3.79$\pm $0.6854.20$\pm $0.84162.60$\pm $2.51
    ${\rm{BSC - }}{{\rm{I}}_5}$2.59$\pm $0.1333.40$\pm $0.89167.00$\pm $4.47
    ${\rm{HPSCN}}_1^1$4.23$\pm $0.13154.80$\pm $2.59154.80$\pm $2.59
    ${\rm{HPSCN}}_3^1$2.01$\pm $0.1359.00$\pm $2.00162.60$\pm $2.41
    ${\rm{HPSCN}}_5^1$1.36$\pm $0.1134.20$\pm $1.09166.20$\pm $3.03
    下载: 导出CSV

    表  4  不同块宽的算法性能比较

    Table  4  Performance comparison of algorithms with different block sizes

    数据集算法nRnLEff (%)
    DB1${\rm{HPSCN}}_1^1$61.361.149.9
    ${\rm{HPSCN}}_2^1$63.822.426.0
    ${\rm{HPSCN}}_3^1$52.812.619.3
    ${\rm{HPSCN}}_5^1$42.52.55.6
    ${\rm{HPSCN}}_{10}^1$24.20.62.4
    DB2${\rm{HPSCN}}_1^1$119.2120.150.2
    ${\rm{HPSCN}}_2^1$115.056.432.9
    ${\rm{HPSCN}}_3^1$99.224.319.7
    ${\rm{HPSCN}}_5^1$74.27.69.3
    ${\rm{HPSCN}}_{10}^1$44.60.40.9
    DB3${\rm{HPSCN}}_1^1$48.448.650.1
    ${\rm{HPSCN}}_2^1$40.823.436.4
    ${\rm{HPSCN}}_3^1$33.67.618.4
    ${\rm{HPSCN}}_5^1$24.01.04.0
    ${\rm{HPSCN}}_{10}^1$13.60.21.4
    DB4${\rm{HPSCN}}_1^1$77.377.550.0
    ${\rm{HPSCN}}_2^1$64.229.431.4
    ${\rm{HPSCN}}_3^1$51.87.212.2
    ${\rm{HPSCN}}_5^1$33.01.23.5
    ${\rm{HPSCN}}_{10}^1$17.00.21.1
    下载: 导出CSV
  • [1] Chen Xing, Niu Ya-Wei, Wang Guang-Hui, Yan Gui-Ying. MKRMDA: Multiple Kernel Learning-based Kronecker Regularized Least Squares for MiRNA-disease Association Prediction. Journal of Translational Medicine, 2013, 15(1): 251-264
    [2] 周平, 刘记平. 基于数据驱动多输出ARMAX建模的高炉十字测温中心温度在线估计. 自动化学报, 2018, 44(3): 552-561

    Zhou Ping, Liu Ji-ping. Data-driven Multi-output ARMAX Modeling for Online Estimation of Central Temperatures for Cross Temperature Measuring in Blast Furnace Ironmaking. Acta Automatica Sinica, 2018, 44(9): 1569-1589
    [3] 汤健, 乔俊飞, 柴天佑, 刘卓, 吴志伟. 基于虚拟样本生成技术的多组分机械信号建模. 自动化学报, 2018, 44(9): 1569-1589

    Tang Jian, Qiao Jun-Fei, Chai Tian-You, Liu Zhuo, Wu ZhiWei. Modeling Multiple Components Mechanical Signals by Means of Virtual Sample Generation Technique. Acta Automatica Sinica, 2018, 44(9): 1569-1589
    [4] Witten Ian H, Frank E, Hall M A. Data Mining: Practical Machine Learning Tools and Technique, 3rd Edition. Amsterdam: Morgan Kaufmann, 2011
    [5] Pao Y H, Takefji Y, Functional-link Net Computing: Theory, System Architecture, and Functionalities. Computer, 1992, 25(5): 76-79 doi: 10.1109/2.144401
    [6] Schmidt W F, Kraaijveld M A, Duin R P W. Feedforward neural networks with random weights. International Conference on Pattern Recognition IEEE Computer Society, 1992
    [7] Cao Wei-Peng, Wang Xi-Zhao, Ming Zhong, Gao Jin-Zhu. A Review on Neural Networks with Random Weights. Neurocomputing, 2018, 275: 278-287 doi: 10.1016/j.neucom.2017.08.040
    [8] Scardapane S, Wang Dian-Hui. Randomness in Neural Networks: An Overview. Wiley Interdisciplinary Reviews-Data Mining and Knowledge Discovery, 2017, 7(2): e1200 doi: 10.1002/widm.1200
    [9] Lu Jing, Zhao Jian-Wei, Cao Fei-Long. Extended Feed Forward Neural Networks with Random Weights for Face Recognition. Neurocomputing, 2014, 136: 96-102 doi: 10.1016/j.neucom.2014.01.022
    [10] Dai Wei, Liu Qiang, Chai Tian-You. Particle Size Estimate of Grinding Processes Using Random Vector Functional Link Networks with Improved Robustness. Neurocomputing, 2015, 169: 361-372 doi: 10.1016/j.neucom.2014.08.098
    [11] Dai Weo, Chen Qi-Xin, Chu Fei, Chai Tian-You. Robust Regularized Random Vector Functional Link Network and Its Industrial Application. IEEE Access, 2017, 5: 16162-16172 doi: 10.1109/ACCESS.2017.2737459
    [12] Li Ming, Wang Dian-Hui. Insights into Randomized Algorithms for Neural Networks: Practical Issues and Common Pitfalls. Information Sciences, 2017, 382: 170-178
    [13] Gorban A N, Tyukin I Y, Prokhorov D V, Sofeikov K I. Approximation with Random Bases: Pro et contra. Information Sciences, 2016, 364: 129-145
    [14] Wang Dian-Hui, Li Ming. Stochastic Configuration Networks: Fundamentals and Algorithms. IEEE Transactions on Cybernetics, 2017, 47(10): 3466-3479 doi: 10.1109/TCYB.2017.2734043
    [15] Zhu Xiao-Long, Feng Xiang-Chu, Wang Wei-Wei, Jia XiXi, He Rui-Qiang. A Further Study on the Inequality Constraints in Stochastic Configuration Networks. Information Sciences, 2019, 487: 77-83 doi: 10.1016/j.ins.2019.02.066
    [16] Wang Dian-Hui, Cui Cai-Hao. Stochastic Configuration Networks Ensemble for Large-Scale Data Analytics. biochemical treatment process. Information Sciences, 2017, 417: 55-71
    [17] Sheng Zhui-Yong, Zeng Zhi-Qiang, Qu Hong-Quan, Zhang Yuan. Optical fiber intrusion signal recognition method based on TSVD-SCN. Optical Fiber Technology, 2019, 48: 270-277 doi: 10.1016/j.yofte.2019.01.023
    [18] 王前进, 杨春雨, 马小平, 张春富, 彭思敏. 基于随机配置网络的井下供给风量建模. 自动化学报, DOI: 10.16383/j.aas.c190602

    Wang Qian-Jin, Yang Chun-Yu, Ma Xiao-Ping, Zhang Chun-Fu, Peng Si-Min. Underground Airflow Quantity Modeling Based on SCN. Acta Automatica Sinica, DOI: 10.16383/j.aas.c190602
    [19] Wang Dian-Hui, Li Ming. Deep stochastic configuration networks with universal approximation property. In: Proceedings of the 2018 International Joint Conference on Neural Networks, 2018.
    [20] Wang Dian-Hui, Li Ming. Robust Stochastic Configuration Networks with Kernel Density Estimation for Uncertain Data Regression. Information Sciences, 2017, 412-413: 210-222 doi: 10.1016/j.ins.2017.05.047
    [21] Li Ming, Huang Chang-Qin, Wang Dian-Hui. Robust Stochastic Configuration Networks with Maximum Correntropy Criterion for Uncertain Data Regression. Information Sciences, 2019, 473: 73-86 doi: 10.1016/j.ins.2018.09.026
    [22] He Qing, Shang Tang-Feng, Zhuang Fu-Zhen, Shi Zhon-Zhi. Parallel Extreme Learning Machine for Regression Based on MapReduce. Neurocomputing, 2013, 102: 52-58 doi: 10.1016/j.neucom.2012.01.040
    [23] Duan Ming-Xing, Li Ken-Li, Liao Xiang-Ke, Li Ke-Qin. A Parallel Multiclassification Algorithm for Big Data Using an Extreme Learning Machine. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(6): 2337-2351 doi: 10.1109/TNNLS.2017.2654357
    [24] Wang Yue-Qing, Dou Yong, Liu Xin-Wang, Lei Yuan-Wu. PR-ELM: Parallel Regularized Extreme Learning Machine Based on Cluster. Neurocomputing, 2016, 173: 1073-1081 doi: 10.1016/j.neucom.2015.08.066
    [25] Lancaster P, Tismenetsky M. The Theory of Matrices: With Applications. Elsevier, 1985
    [26] Dai Wei, Li De-Peng, Zhou Ping, Chai Tian-You. Stochastic Configuration Networks with Block Increments for Data Modeling in Process Industries. Information Sciences, 2019, 484: 367-386 doi: 10.1016/j.ins.2019.01.062
    [27] UCI Machine Learning Repository. [Online], available: https://archive.ics.uci.edu, 2013
    [28] KEEL data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework. [Online], available: http://www.keel.es/, 2011
    [29] Dai Wei, Zhou Ping, Zhao Da-Yong, Lu Shao-Wen, Chai Tian-You. Hardware-in-the-loop Simulation Platform for Supervisory Control of Mineral Grinding Process. Powder technology, 2016, 288: 422-434 doi: 10.1016/j.powtec.2015.11.032
  • 加载中
图(5) / 表(4)
计量
  • 文章访问数:  6813
  • HTML全文浏览量:  1666
  • PDF下载量:  385
  • 被引次数: 0
出版历程
  • 收稿日期:  2019-05-27
  • 录用日期:  2019-12-02
  • 网络出版日期:  2019-12-24
  • 刊出日期:  2021-10-20

目录

    /

    返回文章
    返回