大回归模型的自适应学习

戴瑞芬; 王芳; 郭雷

doi:10.16383/j.aas.c250286

大回归模型的自适应学习

doi: 10.16383/j.aas.c250286 cstr: 32138.14.j.aas.c250286

戴瑞芬^1,,
王芳^1,,
郭雷^2,

1.
山东大学数据科学研究院济南 250100
2.
中国科学院数学与系统科学研究院系统控制重点实验室北京 100190

基金项目: 国家自然科学基金(T2293773, 72371145, 12288201), 国家重点研发计划(2024YFC3307200), 山东省泰山学者专项经费(tsqn202211004) 资助

详细信息

作者简介:
戴瑞芬：山东大学数学学院博士研究生. 主要研究方向为非线性随机系统的学习与控制及其在法律人工智能中的应用. E-mail: dairuifen@mail.sdu.edu.cn

王芳：山东大学数据科学研究院教授. 主要从事计算法学、数据科学、社会复杂系统管理等领域研究. E-mail: wangfang226@sdu.edu.cn

郭雷：中国科学院院士, 中国科学院数学与系统科学研究院研究员. 主要从事系统与控制科学研究, 特别是自适应与不确定性动态系统的学习、滤波、控制与博弈等. 本文通信作者. E-mail: lguo@amss.ac.cn

计量
- 文章访问数: 11
- HTML全文浏览量: 8
- 被引次数: 0
出版历程
- 收稿日期: 2025-06-30
- 录用日期: 2025-08-28
- 网络出版日期: 2025-09-19

Adaptive Learning of Large Regression Models

DAI Rui-Fen^1
,,
WANG Fang^1
,,
GUO Lei^2
,

1.
Data Science Institute, Shandong University, Jinan 250100
2.
Key Laboratory of Systems and Control, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190

Funds: National Natural Science Foundation of China under Grant Nos. T2293773, 72371145, and 12288201, National Key Research and Development Program under Grant No. 2024YFC3307200, and Special Funds for Taishan Scholars Project of Shandong Province, China, under Grant No. tsqn202211004

More Information

Author Bio:
DAI Rui-Fen　PhD candidate at the School of Mathematics, Shandong University. Her main research focuses on the learning and control of nonlinear stochastic systems and their applications in legal artificial intelligence

WANG Fang　Professor at the Data Science Institute, Shandong University. Her primary research focuses on computational law, data science, and management of complex social systems

GUO Lei　Academician of the Chinese Academy of Sciences (CAS), professor at the Academy of Mathematics and Systems Science, CAS. His main research focuses on systems and control science, particularly the learning, filtering, control, and game theory of adaptive and uncertain dynamic systems. Corresponding author of this paper

摘要

摘要: 随着信息技术的快速发展, 特别是计算能力和数据收集能力的不断提升, 利用大参数模型对复杂场景进行建模已成为显著的发展趋势. 然而, 关于一般反馈输入下此类模型的学习问题, 在控制系统领域却鲜有研究. 基于此, 针对饱和观测下的大回归模型, 设计了一种在线扩展型自适应学习算法.该算法可随着新数据的增加自动更新算法维数和计算结果, 在无需存储历史数据的前提下, 实现学习结果的动态调整与输出结果的实时预测. 具体来讲, 在一般的非持续激励数据条件下证明了所提出算法的收敛性, 该结果可以适用于一般反馈控制系统; 还在无任何数据激励条件假设下证明了算法的预测“遗憾”具有良好的收敛性. 最后, 基于真实的故意伤害罪判决数据开展了司法量刑预测实验, 检验了所提出算法的计算效率和预测精度.
- 自适应学习 /
- 大回归模型 /
- 收敛性理论 /
- 一般数据条件 /
- 饱和观测 /
- 司法量刑
Abstract: With the rapid development of information technology and the continuous improvement of computational power and data collection capability, modeling complex scenarios using large parameter models has become a significant development trend. However, the learning problem of such models under general feedback inputs still remains lacking in the control system field. In view of this, we propose an online adaptive learning algorithm with expanding dimension for large regression models under saturated observations. This algorithm dynamically updates both algorithm dimension and computation results as new data arrive, enabling dynamic adjustment of learning outcomes and real-time prediction of outputs without storing historical data. Specifically, we prove the convergence of the proposed algorithm under general non-persistent excitation data conditions, making it applicable to general feedback control systems. Moreover, the good convergence of the prediction “regrets” of the proposed algorithm is also proved without requiring any excitation conditions. Finally, we conduct judicial sentencing prediction experiments based on real-world criminal judgment data for intentional injury cases, validating the computational efficiency and prediction accuracy of the proposed algorithm.
- Adaptive learning /
- large regression models /
- convergence theory /
- general data conditions /
- saturated observations /
- judicial sentencing
注释:

1) 1^①见https://wenshu.court.gov.cn/

2) 2^②见https://www.court.gov.cn/

HTML全文

图 1 随着参数规模递增的轻伤平均预测精度趋势变化图

Fig. 1 The trend of average prediction accuracy for minor injury cases with increasing parameter size

下载: 全尺寸图片幻灯片

图 2 随着参数规模递增的重伤平均预测精度趋势变化图

Fig. 2 The trend of average prediction accuracy for serious injury cases with increasing parameter size

下载: 全尺寸图片幻灯片

图 3 随着参数规模递增的轻伤平均预测遗憾趋势变化图

Fig. 3 The trend of average prediction regret for minor injury cases with increasing parameter size

下载: 全尺寸图片幻灯片

图 4 随着参数规模递增的重伤平均预测遗憾趋势变化图

Fig. 4 The trend of average prediction regret for serious injury cases with increasing parameter size

下载: 全尺寸图片幻灯片

图 5 算法1与算法2^[31]平均预测精度趋势对比图($ n=500 $)

Fig. 5 The trend comparison of average prediction accuracy between Algorithm 1 and Algorithm 2^[31] ($ n=500 $)

下载: 全尺寸图片幻灯片

表 1 算法复杂度分析对比表

维度算法1 算法2^[31]

时间复杂度 $ \mathrm{O}(n p_n^2) $ $ \mathrm{O}(n^2p_n^2) $

空间复杂度 $ \mathrm{O}(p_n^2) $ $ \mathrm{O}(n+p_n^2) $

下载: 导出CSV

参考文献(35)

[1]	Wei J, Tay Y, Bommasani R, Raffel C, Zoph B, Borgeaud S, et al. Emergent abilities of large language models. arXiv preprint arXiv: 2206.07682, 2022.
[2]	Zhao W X, Zhou K, Li J, Tang T, Wang X, Hou Y, et al. A survey of large language models. arXiv preprint arXiv: 2303.18223, 2023.
[3]	郭雷. 不确定性动态系统的估计, 控制与博弈. 中国科学: 信息科学, 2020, 50(9): 1327−1344 doi: 10.1360/SSI-2020-0277 Guo Lei. Estimation, control, and games of dynamical systems with uncertainty. Scientia Sinica Informationis, 2020, 50(9): 1327−1344 doi: 10.1360/SSI-2020-0277
[4]	Zhang L, Guo L. Adaptive tracking control with binary-valued output observations. arXiv preprint arXiv: 2411.05975, 2024.
[5]	Guo L. Feedback and uncertainty: Some basic problems and results. Annual Reviews in Control, 2020, 49: 27−36 doi: 10.1016/j.arcontrol.2020.04.001
[6]	Yehudai G, Ohad S. Learning a single neuron with gradient methods. In: Proceedings of the Conference on Learning Theory. Virtual, Online, Austria: PMLR, 2020. 3756-3786.
[7]	Agarap A F. Deep learning using rectified linear units (ReLU). arXiv preprint arXiv: 1803.08375, 2018.
[8]	Shamir O. The implicit bias of benign overfitting. In: Proceedings of the Conference on Learning Theory (COLT). London, United Kingdom: PMLR, 2022. 448-478.
[9]	Tobin J. Estimation of relationships for limited dependent variables. Econometrica, 1958, 26(1): 24−36 doi: 10.2307/1907382
[10]	Cook J, McDonald J. Partially adaptive estimation of interval censored regression models. Computational Economics, 2013, 42: 119−131 doi: 10.1007/s10614-012-9324-0
[11]	Heckman J J. The common structure of statistical models of truncation, sample selection and limited dependent variables and a simple estimator for such models. Annals of Economic and Social Measurement, 1976, 5(4): 475−492
[12]	Clark T G, Bradburn M J, Love S B, Altman D G. Survival analysis part I: Basic concepts and first analyses. British Journal of Cancer, 2003, 89(2): 232−238 doi: 10.1038/sj.bjc.6601118
[13]	Cleves M. An introduction to survival analysis using Stata. Stata Press, 2008.
[14]	王芳, 张蓝天, 郭雷. 非线性递推辨识理论在量刑数据分析中的应用. 中国科学: 信息科学, 2022, 52(10): 1837−1852 doi: 10.1360/SSI-2022-0325 Wang Fang, Zhang Lan-Tian, Guo Lei. Applications of nonlinear recursive identification theory in sentencing data analyses. Scientia Sinica Informationis, 2022, 52(10): 1837−1852 doi: 10.1360/SSI-2022-0325
[15]	Jin Y, Zheng X, Guo L. Adaptive sentencing prediction with guaranteed accuracy and legal interpretability. arXiv preprint arXiv: 2505.14011, 2025.
[16]	Wang L Y, Zhang J F, Yin G G. System identification using binary sensors. IEEE Transactions on Automatic Control, 2003, 48: 1892−1907 doi: 10.1109/TAC.2003.819073
[17]	Jafari K, Juillard J, Roger M. Convergence analysis of an online approach to parameter estimation problems based on binary observations. Automatica, 2012, 48: 2837−2842 doi: 10.1016/j.automatica.2012.05.050
[18]	Guo J, Zhao Y. Recursive projection algorithm on FIR system identification with binary-valued observations. Automatica, 2013, 49: 3396−3401 doi: 10.1016/j.automatica.2013.08.011
[19]	Wang Y, Zhao Y, Zhang J F, Guo J. A unified identification algorithm of FIR systems based on binary observations with time-varying thresholds. Automatica, 2022, 135: 109990 doi: 10.1016/j.automatica.2021.109990
[20]	Bercu B, Godichon A, Portier B. An efficient stochastic Newton algorithm for parameter estimation in logistic regressions. SIAM Journal on Control and Optimization, 2020, 58: 348−367 doi: 10.1137/19M1261717
[21]	Zhang L, Zhao Y, Guo L. Identification and adaptation with binary-valued observations under non-persistent excitation condition. Automatica, 2022, 138: 110158 doi: 10.1016/j.automatica.2022.110158
[22]	Zhang L, Guo L. Adaptive identification with guaranteed performance under saturated observation and nonpersistent excitation. IEEE Transactions on Automatic Control, 2023, 69(3): 1584−1599
[23]	Zhao W X, Chen H F. Markov chain approach to identifying Wiener systems. Science China Information Sciences, 2012, 55: 1201−1217 doi: 10.1007/s11432-012-4582-y
[24]	Dai R, Guo L. Estimation of IIR systems with binary-valued observations. Chinese Annals of Mathematics, Series B, 2023, 44(5): 687−702 doi: 10.1007/s11401-023-0038-5
[25]	Lai T L, Wei C Z. Least squares estimates in stochastic regression models with applications to identification and control of dynamic systems. The Annals of Statistics, 1982, 10(1): 154−166
[26]	Hannan E J, Deistler M. The Statistical Theory of Linear Systems. New York: Wiley, 1988.
[27]	Wahlberg B, Ljung L. Hard frequency-domain model error bounds from least-squares like identification techniques. IEEE Transactions on Automatic Control, 1992, 37(7): 900−912 doi: 10.1109/9.148343
[28]	Chen H F, Guo L. Identification and stochastic adaptive control. Boston, MA: Birkhäuser, 1991.
[29]	Guo L, Huang D W, Hannan E J. On ARX(∞) approximation. Journal of Multivariate Analysis, 1990, 32(1): 17−47 doi: 10.1016/0047-259X(90)90069-T
[30]	Huang D, Guo L. Estimation of nonstationary ARMAX models based on the Hannan-Rissanen method. The Annals of Statistics, 1990, 18(4): 1729−1756
[31]	Dai R, Guo L. Estimation and prediction for large models with saturated output observation and general input condition. Automatica, 2025, 177: 112321 doi: 10.1016/j.automatica.2025.112321
[32]	Guo L. Convergence and logarithm laws of self-tuning regulators. Automatica, 1995, 31(3): 435−450 doi: 10.1016/0005-1098(94)00127-5
[33]	Cheney E W. Analysis for applied mathematics. New York: Springer, 2001.
[34]	Lai T L. Asymptotically efficient adaptive control in stochastic regression models. Advances in Applied Mathematics, 1986, 7(1): 23−45 doi: 10.1016/0196-8858(86)90004-7
[35]	Kim Y. Convolutional neural networks for sentence classification. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing. Doha, Qatar: ACL, 2014. 1746-1751