基于替代函数及贝叶斯框架的1范数ELM算法

韩敏; 李德才

doi:10.3724/SP.J.1004.2011.01344

基于替代函数及贝叶斯框架的1范数ELM算法

doi: 10.3724/SP.J.1004.2011.01344

韩敏^,,
李德才

1.
大连理工大学电子信息与电气工程学部大连 116023

详细信息

通讯作者:
韩敏大连理工大学教授. 主要研究方向为神经网络理论及其应用, 复杂系统建模及自适应控制. E-mail: minhan@dlut.edu.cn

计量
- 文章访问数: 2257
- HTML全文浏览量: 43
- PDF下载量: 1402
- 被引次数: 0
出版历程
- 收稿日期: 2010-08-30
- 修回日期: 2010-12-27
- 刊出日期: 2011-11-20

An Norm 1 Regularization Term ELM Algorithm Based on Surrogate Function and Bayesian Framework

HAN Min^,,
LI De-Cai

1.
Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian 116023

摘要

摘要: 针对极端学习机 (Extreme learning machine, ELM)算法的不适定问题和模型规模控制问题,本文提出基于1范数正则项的改进型ELM算法. 通过在二次损失函数基础上引入1范数正则项以控制模型规模,改善ELM的泛化能力.此外,为简化1范数正则化方法的求解过程,利用边际优化方法,构建适当的替代函数,以便于采用贝叶斯方法代替计算复杂的交叉检验方法,并实现正则化参数的自适应估计.仿真结果表明,本文所提算法能够有效简化模型结构,并保持较高的预测精度.
- 1范数正则化 /
- 极端学习机 /
- 替代函数 /
- 贝叶斯方法
Abstract: Focusing on the ill-posed problem and the model scale control of ELM (Extreme learning machine), this paper proposes an improved ELM algorithm based on 1-norm regularization term. This is achieved by involving an 1-norm regularization term into the original square cost function, and it can be used to control the model scale and enhance the generalization capability. Furthermore, to simplify the solving process of the 1-norm regularization method, the bound optimization algorithm is employed and a suitable surrogate function is established. Based on the surrogate function, the Bayesian algorithm can be used to substitute the complicated cross validation method and estimate the regularization parameter adaptively. Simulation results illustrate that the proposed method can effectively simplify the model structure, while remaining acceptable prediction accurate.
- Norm 1 regularization /
- extreme learning machine (ELM) /
- surrogate function /
- Bayesian method

HTML全文

参考文献(1)

[1]

Meng Zu-Qiang, Cai Zi-Xing. Identification method of nonlinear systems based on parallel genetic algorithm. Control and Decision, 2003, 18(3): 367-370 (蒙祖强, 蔡自兴. 一种基于并行遗传算法的非线性系统辨识方法. 控制与决策, 2003, 18(3): 367-370)[2] Qiao Jun-Fei, Han Hong-Gui. Optimal structure design for RBFNN structure. Acta Automatica Sinica, 2010, 36(6): 865-872 (乔俊飞, 韩红桂. RBF神经网络的结构动态优化设计. 自动化学报, 2010, 36(6): 865-872)[3] Ye Jian, Ge Lin-Dong, Wu Yue-Xian. An application of improved RBF neural network in modulation recognition. Acta Automatica Sinica, 2007, 33(6): 652-654 (叶健, 葛临东, 吴月娴. 一种优化的RBF神经网络在调制识别中的应用. 自动化学报, 2007, 33(6): 652-654)[4] Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: theory and applications. Neurocomputing, 2006, 70(1-3): 489-501[5] Malathi V, Marimuthu N S, Baskar S. Intelligent approaches using support vector machine and extreme learning machine for transmission line protection. Neurocomputing, 2010, 73(10-12): 2160-2167[6] Minhas R, Baradarani A, Seifzadeh S, Wu Q M J. Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing, 2010, 73(10-12): 1906-1917[7] Tang X L, Han M. Partial Lanczos extreme learning machine for single output regression problems. Neurocomputing, 2009, 72(13-15): 3066-3076[8] Huang G B, Chen L. Enhanced random search based incremental extreme learning machine. Neurocomputing, 2008, 71(16-18): 3460-3468[9] Huang G B, Chen L, Siew C K. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Network, 2006, 17(4): 879-892[10] Hansen P C. Rank-deficient and Discrete Ill--posed Problems: Numerical Aspects of Linear Inversion. Philadelphia: SIAM, 1998. 45-68[11] Kaban A. On Bayesian classification with Laplace priors. Pattern Recognition Letters, 2007, 28(10): 1271-1282[12] Krishnapuram B, Carin L, Figueiredo M A T, Hartemink A J. Sparse multinomial logistic regression: fast algorithms and generalization bounds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(6): 957-968

施引文献

资源附件(0)

访问统计