一种基于<i>L</i><sub>1</sub>范数正则化的回声状态网络

韩敏; 任伟杰; 许美玲

doi:10.3724/SP.J.1004.2014.02428

一种基于L₁范数正则化的回声状态网络

doi: 10.3724/SP.J.1004.2014.02428

1.
大连理工大学电子信息与电气工程学部大连 116024

基金项目:

国家重点基础研究发展计划(973计划) (2013CB430403),国家自然科学基金(61374154)资助

详细信息

作者简介:
任伟杰大连理工大学电子信息与电气工程学部博士研究生. 主要研究方向为变量选择, 多元时间序列预测.E-mail: renweijie@mail.dlut.edu.cn

通讯作者:
韩敏, 大连理工大学电子信息与电气工程学部教授. 主要研究方向为神经网络, 模式识别, 混沌时间序列预测. 本文通信作者.E-mail: minhan@dlut.edu.cn

计量
- 文章访问数: 1935
- HTML全文浏览量: 84
- PDF下载量: 1028
- 被引次数: 0
出版历程
- 收稿日期: 2013-11-06
- 修回日期: 2014-05-02
- 刊出日期: 2014-11-20

An Improved Echo State Network via L₁-Norm Regularization

1.
Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian 116024

Funds:

Supported by National Basic Research Program of China (973 Program) (2013CB430403) and National Natural Science Foundation of China (61374154)

摘要

摘要: 针对回声状态网络存在的病态解以及模型规模控制问题,本文提出一种基于L1范数正则化的改进回声状态网络.该方法通过在目标函数中添加L1范数惩罚项,提高模型求解的数值稳定性,同时借助于L1范数正则化的特征选择能力,控制网络的复杂程度,防止出现过拟合.对于L1范数正则化的求解,采用最小角回归算法计算正则化路径,通过贝叶斯信息准则进行模型选择,避免估计正则化参数.将模型应用于人造数据和实际数据的时间序列预测中,仿真结果证明了本文方法的有效性和实用性.
- 回声状态网络 /
- 正则化 /
- 最小角回归 /
- 信息准则 /
- 多元时间序列
Abstract: Considering the ill-posed problem and the model scale control of echo state network, an improved echo state network based on L1-norm regularization is proposed. In order to improve the numerical stability, the proposed method adds an L1-norm penalty term in the objective function. Meanwhile, the method can also control the complexity of the network and prevent overfitting by using feature selection capability of L1-norm regularization. To solve the L1-norm regularization model, we adopt the least angle regression algorithm to calculate regularization path and select suitable model through Bayesian information criterion, which can avoid the estimations of regularization parameter. The model is applied to the time series predictions of both synthetic dataset and practical dataset. The simulation results show the effectiveness and practicality of the proposed method.
- Echo state network (ESN) /
- regularization /
- least angle regression (LARS) /
- information criterion /
- multivariate time series

HTML全文

参考文献(22)

[1]	Jaeger H, Hass H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science, 2004, 304(5667): 78-80
[2]	Qiao Jun-Fei, Bo Ying-Chun, Han Guang. Application of ESN-based multi indices dual heuristic dynamic programming on wastewater treatment process. Acta Automatica Sinica, 2013, 39(7): 1146-1151 (乔俊飞, 薄迎春, 韩广. 基于ESN的多指标DHP控制策略在污水处理过程中的应用. 自动化学报, 2013, 39(7): 1146-1151)
[3]	Ongenae F, Van Looy S, Verstraeten D, Verplancke T, Benoit D, De Turck F, Dhaene T, Schrauwen B, Decruyenaere J. Time series classification for the prediction of dialysis in critically ill patients using echo state networks. Engineering Applications of Artificial Intelligence, 2013, 26(3): 984-996
[4]	Li G Q, Niu P F, Zhang W P, Zhang Y. Control of discrete chaotic systems based on echo state network modeling with an adaptive noise canceler. Knowledge-Based Systems, 2012, 35: 35-40
[5]	Peng Yu, Wang Jian-Min, Peng Xi-Yuan. Researches on time series prediction with echo state networks. Acta Electronica Sinica, 2010, 38(2A): 148-154 (彭宇, 王建民, 彭喜元. 基于回声状态网络的时间序列预测方法研究. 电子学报, 2010, 38(2A): 148-154)
[6]	Lukosevicius M, Jaeger H. Reservoir computing approaches to recurrent neural network training. Computer Science Review, 2009, 3(3): 127-149
[7]	Rong H J, Ong Y S, Tan A H, Zhu Z. A fast pruned-extreme learning machine for classification problem. Neurocomputing, 2008, 72(1-3): 359-366
[8]	Dutoit X, Schrauwen B, Van Campenhout J, Stroobandt D, Van Brussel H, Nuttin M. Pruning and regularization in reservoir computing. Neurocomputing, 2009, 72(7-9): 1534-1546
[9]	Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A. OP-ELM: optimally pruned extreme learning machine. IEEE Transactions on Neural Networks, 2010, 21(1): 158-162
[10]	Kump P, Bai E W, Chan K S, Eichinger B, Li K. Variable selection via RIVAL (removing irrelevant variables amidst LASSO iterations) and its application to nuclear material detection. Automatica, 2012, 48(9): 2107-2115
[11]	Liu Qiao, Qin Zhi-Guang, Chen Wei, Zhang Feng-Li. Zero-norm penalized feature selection support vector machine. Acta Automatica Sinica, 2011, 37(2): 252-256 (刘峤, 秦志光, 陈伟, 张凤荔. 基于零范数特征选择的支持向量机模型. 自动化学报, 2011, 37(2): 252-256)
[12]	Han Min, Li De-Cai. An norm 1 regularization term ELM algorithm based on surrogate function and Bayesian framework. Acta Automatica Sinica, 2011, 37(11): 1344-1350 (韩敏, 李德才. 基于替代函数及贝叶斯框架的1范数ELM算法. 自动化学报, 2011, 37(11): 1344-1350)
[13]	Liu Jian-Wei, Li Shuang-Cheng, Luo Xiong-Lin. Classification algorithm of support vector machine via p-norm regularization. Acta Automatica Sinica, 2012, 38(1): 76-87 (刘建伟, 李双成, 罗雄麟. p范数正则化支持向量机分类算法. 自动化学报, 2012, 38(1): 76-87)
[14]	Miche Y, Van Heeswijk M, Bas P, Simula O, Lendasse A. TROP-ELM: a double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing, 2011, 74(16): 2413-2421
[15]	Friedman J H. Fast sparse regression and classification. International Journal of Forecasting, 2012, 28(3): 722-738
[16]	Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 1996, 58(1): 267-288
[17]	Peng Yi-Gang, Suo Jin-Li, Dai Qiong-Hai, Xu Wen-Li. From compressed sensing to low-rank matrix recovery: theory and applications. Acta Automatica Sinica, 2013, 39(7): 981-994(彭义刚, 索津莉, 戴琼海, 徐文立. 从压缩传感到低秩矩阵恢复: 理论与应用. 自动化学报, 2013, 39(7): 981-994)
[18]	Efron B, Hastie T, Johnstone I, Tibshirani R. Least angle regression. The Annals of Statistics, 2004, 32(2): 407-499
[19]	Stoica P, Selen Y. Model-order selection: a review of information criterion rules. IEEE Signal Processing Magazine, 2004, 21(4): 36-47
[20]	Watanabe S. A widely applicable Bayesian information criterion. Journal of Machine Learning Research, 2013, 14(1): 867-897
[21]	Wu C L, Chau K W. Prediction of rainfall time series using modular soft computing methods. Engineering Applications of Artificial Intelligence, 2013, 26(3): 997-1007
[22]	Box G E P, Jenkins G M, Reinsel G C. Time Series Analysis: Forecasting and Control. New Jersey, USA: John Wiley & Sons, 2008. 677-678

施引文献

资源附件(0)

访问统计

计量

文章访问数: 1935
HTML全文浏览量: 84
PDF下载量: 1028
被引次数: 0

姓名
邮箱
手机号码
标题
留言内容
验证码

留言板

一种基于L₁范数正则化的回声状态网络

doi: 10.3724/SP.J.1004.2014.02428

作者简介:
任伟杰大连理工大学电子信息与电气工程学部博士研究生. 主要研究方向为变量选择, 多元时间序列预测.E-mail: renweijie@mail.dlut.edu.cn

通讯作者:
韩敏, 大连理工大学电子信息与电气工程学部教授. 主要研究方向为神经网络, 模式识别, 混沌时间序列预测. 本文通信作者.E-mail: minhan@dlut.edu.cn

计量

An Improved Echo State Network via L₁-Norm Regularization

计量

目录

留言板

一种基于L1范数正则化的回声状态网络

doi: 10.3724/SP.J.1004.2014.02428

作者简介: 任伟杰 大连理工大学电子信息与电气工程学部博士研究生. 主要研究方向为变量选择, 多元时间序列预测.E-mail: renweijie@mail.dlut.edu.cn

通讯作者: 韩敏, 大连理工大学电子信息与电气工程学部教授. 主要研究方向为神经网络, 模式识别, 混沌时间序列预测. 本文通信作者.E-mail: minhan@dlut.edu.cn

计量

出版历程

An Improved Echo State Network via L1-Norm Regularization

计量

出版历程

目录

一种基于L₁范数正则化的回声状态网络

作者简介:
任伟杰大连理工大学电子信息与电气工程学部博士研究生. 主要研究方向为变量选择, 多元时间序列预测.E-mail: renweijie@mail.dlut.edu.cn

通讯作者:
韩敏, 大连理工大学电子信息与电气工程学部教授. 主要研究方向为神经网络, 模式识别, 混沌时间序列预测. 本文通信作者.E-mail: minhan@dlut.edu.cn

An Improved Echo State Network via L₁-Norm Regularization