2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于池的无监督线性回归主动学习

刘子昂 蒋雪 伍冬睿

刘子昂, 蒋雪, 伍冬睿. 基于池的无监督线性回归主动学习. 自动化学报, 2020, 46(x): 1−13 doi: 10.16383/j.aas.c200071
引用本文: 刘子昂, 蒋雪, 伍冬睿. 基于池的无监督线性回归主动学习. 自动化学报, 2020, 46(x): 1−13 doi: 10.16383/j.aas.c200071
Liu Zi-Ang, Jiang Xue, Wu Dong-Rui. Unsupervised pool-based active learning for linear regression. Acta Automatica Sinica, 2020, 46(x): 1−13 doi: 10.16383/j.aas.c200071
Citation: Liu Zi-Ang, Jiang Xue, Wu Dong-Rui. Unsupervised pool-based active learning for linear regression. Acta Automatica Sinica, 2020, 46(x): 1−13 doi: 10.16383/j.aas.c200071

基于池的无监督线性回归主动学习

doi: 10.16383/j.aas.c200071
基金项目: 湖北省技术创新专项资助项目(2019AEA171), 国家自然科学基金资助项目(61873321), NSFC-深圳机器人基础研究中心重点项目(U1913207), 科技部政府间国际科技创新合作重点专项(2017YFE0128300)资助
详细信息
    作者简介:

    刘子昂:2017年获得武汉理工大学自动化专业学士学位, 2020年获得华中科技大学控制科学与工程硕士学位. 主要研究方向为机器学习. E-mail: ziangliu@hust.edu.cn

    蒋雪:华中科技大学人工智能与自动化学院博士研究生. 2019年获得西南大学电子信息工程学院学士学位. 主要研究方向为机器学习, 脑机接口和情感计算. E-mail: xuejiang@hust.edu.cn

    伍冬睿:华中科技大学人工智能与自动化学院教授, 博士生导师, 图像信息处理与智能控制教育部重点实验室副主任. 主要研究方向为机器学习、脑机接口、计算智能、情感计算. 本文通信作者. E-mail: drwu@hust.edu.cn

Unsupervised Pool-Based Active Learning for Linear Regression

Funds: Supported by Technology Innovation Project of Hubei Province of China(2019AEA171), Natural Science Foundation of China(61873321), International Science and Technology Cooperation Program of China(2017YFE0128300)
  • 摘要: 在许多现实的机器学习应用场景中, 获取大量未标注的数据是很容易的, 但标注过程需要花费大量的时间和经济成本. 因此, 在这种情况下, 需要选择一些最有价值的样本进行标注, 从而只利用较少的标注数据就能训练出较好的机器学习模型. 主动学习已被广泛应用于解决这种场景下的问题. 但是, 大多数现有的主动学习方法都是基于有监督场景: 能够从少量带标签的样本中训练初始模型, 基于模型查询新的样本, 然后迭代更新模型. 无监督情况下的主动学习却很少有人考虑, 即在不知道任何标签信息的情况下最佳地选择要标注的初始训练样本. 这种场景下, 主动学习问题变得更加困难, 因为无法利用任何标签信息. 针对这一场景, 本文研究了基于池的无监督线性回归问题, 提出了一种新的主动学习方法, 该方法同时考虑了信息性、代表性和多样性这三个标准. 本文在3个不同的线性回归模型(岭回归, LASSO和线性支持向量回归)和来自不同应用领域的12个数据集上进行了广泛的实验, 验证了其有效性.
  • 图  1  基于池的ALR中样本的代表性与多样性[17]

    Fig.  1  Illustration of representativeness and diversity in pool-based ALR[17]

    图  2  $d=2$ 时IRD算法图示

    Fig.  2  Illustration of IRD when $d=2$

    图  3  12个数据集上的平均RMSE和CC(mRMSE和mCC; 重复运行100次). 回归模型为RR ( $\lambda=0.5$ ).

    Fig.  3  Mean of the RMSEs and the CCs on the 12 datasets, averaged over 100 runs. RR ( $\lambda=0.5$ ) was used as the regression model.

    图  4  12个数据集上归一化AUC-mRMSE和AUC-mCC

    Fig.  4  Normalized AUCs of the mean RMSEs and the mean CCs on the 12 datasets

    图  5  对于不同的 $M$ , 4种ALR方法的mRMSE和mCC相对于RS在12个数据集上的平均比率.

    Fig.  5  Ratios of the mean RMSEs and the mean CCs for different $M$ , averaged across 12 datasets.

    图  6  在Housing数据集上不同ALR算法所选样本(星号)的t-SNE可视化.

    Fig.  6  t-SNE visualization of the selected samples (asterisks) from different ALR approaches on the Housing dataset.

    图  7  对于不同的 $c_{\max}$ , 4种ALR算法的AUC-mRMSE和AUC-mCC相对于RS在12个数据集上的平均比率.

    Fig.  7  Ratios of AUCs of the mean RMSEs and the mean CCs for different $c_{\max}$ , averaged across 12 datasets.

    图  8  对于不同的 $\lambda$ (RR和LASSO)和 $C$ (线性SVR), 4种ALR算法的AUC-mRMSE和AUC-mCC相对于RS在12个数据集上的平均比率.

    Fig.  8  Ratios of the AUCs of the mean RMSEs and the mean CCs, averaged across 12 datasets, for different $\lambda$ (RR and LASSO) and $C$ (linear SVR).

    图  9  对于不同的 $M$ , IRD及其变体的mRMSE和mCC相对于RS在12个数据集上的平均比率.

    Fig.  9  Ratios of the mean RMSEs and the mean CCs w.r.t. different $M$ , averaged across 12 datasets.

    表  1  基于池的无监督ALR方法中考虑的标准.

    Table  1  Criteria considered in the three existing and the proposed unsupervised pool-based ALR approaches.

    方法 信息性 代表性 多样性
    现有方法 P-ALICE $\checkmark$ $-$ $-$
    GSx $-$ $-$ $\checkmark$
    RD $-$ $\checkmark$ $\checkmark$
    本文方法 IRD $\checkmark$ $\checkmark$ $\checkmark$
    下载: 导出CSV

    表  2  12个数据集的总结.

    Table  2  Summary of the 12 regression datasets.

    数据集 来源 样本个数 原始特征个数 数字型特征个数 类别型特征个数 总的特征个数
    Concrete-CS $^1$ UCI 103 7 7 0 7
    Yacht $^2$ UCI 308 6 6 0 6
    autoMPG $^3$ UCI 392 7 6 1 9
    NO2 $^4$ StatLib 500 7 7 0 7
    Housing $^5$ UCI 506 13 13 0 13
    CPS $^6$ StatLib 534 10 7 3 19
    EE-Cooling $^7$ UCI 768 7 7 0 7
    VAM-Arousal $^8$ ICME 947 46 46 0 46
    Concrete $^9$ UCI 1030 8 8 0 8
    Airfoil $^{10}$ UCI 1503 5 5 0 5
    Wine-Red $^{11}$ UCI 1599 11 11 0 11
    Wine-White $^{11}$ UCI 4898 11 11 0 11
    $^1$ https://archive.ics.uci.edu/ml/datasets/Concrete+Slump+Test
    $^2$ https://archive.ics.uci.edu/ml/datasets/Yacht+Hydrodynamics
    $^3$ https://archive.ics.uci.edu/ml/datasets/auto+mpg
    $^4$ http://lib.stat.cmu.edu/datasets/
    $^5$ https://archive.ics.uci.edu/ml/machine-learning-databases/housing/
    $^6$ http://lib.stat.cmu.edu/datasets/CPS_85_Wages
    $^7$ http://archive.ics.uci.edu/ml/datasets/energy+efficiency
    $^8$ https://dblp.uni-trier.de/db/conf/icmcs/icme2008.html
    $^9$ https://archive.ics.uci.edu/ml/datasets/Concrete+Compressive+Strength
    $^{10}$ https://archive.ics.uci.edu/ml/datasets/Airfoil+Self-Noise
    $^{11}$ https://archive.ics.uci.edu/ml/datasets/Wine+Quality
    下载: 导出CSV

    表  3  AUC-mRMSE/sRMSE和AUC-mCC/sCC的提升百分比.

    Table  3  Percentage improvements of the AUCs of the mean/std RMSEs and the mean/std CCs.

    回归模型 性能指标 相对于RS的提升百分比
    P-ALICE GSx RD IRD
    RR RMSE Mean 2.58 -2.57 4.15 8.63
    std 2.75 3.98 36.60 34.84
    CC Mean 6.54 -3.43 10.39 18.70
    std 12.74 29.47 35.03 42.97
    LASSO RMSE Mean 4.22 0.84 7.58 10.81
    std 6.77 0.85 43.45 39.84
    CC Mean 25.06 69.41 25.67 60.63
    std 6.39 31.05 22.46 29.82
    SVR RMSE Mean 4.21 0.66 5.23 12.12
    std 6.62 -0.19 33.99 38.69
    CC Mean 9.71 -1.65 12.46 28.99
    std 11.10 25.78 34.97 43.25
    下载: 导出CSV

    表  4  非参数多重检验的 $p$ 值( $\alpha=0.05$ ; 如果 $p<\alpha/2$ 拒绝 $H_0$ ).

    Table  4  $p$ -values of non-parametric multiple comparisons ( $\alpha=0.05$ ; reject $H_0$ if $p<\alpha/2$ ).

    回归模型 性能指标 IRD versus
    RS P-ALICE GSx RD
    RR RMSE 0.0000 0.0003 0.0000 0.0284
    CC 0.0000 0.0000 0.0000 0.0005
    LASSO RMSE 0.0000 0.0004 0.0000 0.0596
    CC 0.0000 0.0000 0.0000 0.0000
    SVR RMSE 0.0000 0.0000 0.0000 0.0018
    CC 0.0000 0.0000 0.0000 0.0000
    下载: 导出CSV
  • [1] Mehrabian A. Basic Dimensions for a General Psychological Theory: Implications for Personality, Social, Environmental, and Developmental Studies. Oelgeschlager, Gunn & Hain, 1980
    [2] Grimm M, Kroschel K, and Narayanan S S. The Vera Am Mittag German audio-visual emotional speech database. In: Proc. Int’l Conf. on Multimedia & Expo (ICME). Hannover, German, June 2008. 865−868
    [3] Bradley M M, Lang P J. The international affective digitized sounds (2nd edition; IADS-2): Affective ratings of sounds and instruction manual, Technical Report B-3, University of Florida, Gainesville, FL, 2007
    [4] Joo J, Wu D, Mendel J M, Bugacov A. Forecasting the post fracturing response of oil wells in a tight reservoir. In: Proc. SPE Western Regional Meeting. San Jose, CA, March 2009
    [5] Settles B. Active learning literature survey, Computer Sciences Technical Report 1648, University of Wisconsin– Madison, Madison, WI, 2009
    [6] Burbidge R, Rowland J J, King R D. Active learning for regression based on query by committee. Lecture Notes in Computer Science, 2007, 4881: 209−218
    [7] Cai W, Zhang M, Zhang Y. Batch mode active learning for regression with expected model change. IEEE Trans. on Neural Networks and Learning Systems, 2017, 28(7): 1668−1681 doi: 10.1109/TNNLS.2016.2542184
    [8] Cai W, Zhang Y, Zhou J. Maximizing expected model change for active learning in regression. In: Proc. IEEE 13th Int’l. Conf. on Data Mining. Dallas, TX, December 2013
    [9] Cohn D A, Ghahramani Z, Jordan M I. Active learning with statistical models. Journal of Artificial Intelligence Research, 1996, 4(1): 129−145
    [10] Freund Y, Seung H, Shamir E, Tishby N. Selective sampling using the query by committee algorithm. Machine Learning, 1997, 28(2-3): 133−168
    [11] MacKay D. Information-based objective functions for active data selection. Neural Computation, 1992, 4(4): 590−604 doi: 10.1162/neco.1992.4.4.590
    [12] Sugiyama M. Active learning in approximately linear regression based on conditional expectation of generalization error. Journal of Machine Learning Research, 2006, 7: 141−166
    [13] Sugiyama M, Nakajima S. Pool-based active learning in approximate linear regression. Machine Learning, 2009, 75(3): 249−274 doi: 10.1007/s10994-009-5100-3
    [14] Willett R, Nowak R, Castro R M. Faster rates in regression via active learning. In: Proc. Advances in Neural Information Processing Systems. Vancouver, Canada, December 2006. 179−186
    [15] Wu D, Lawhern V J, Gordon S, Lance B J, Lin C T. Offline EEG-based driver drowsiness estimation using enhanced batch-mode active learning (EBMAL) for regression. In: Proc. IEEE Int’l Conf. on Systems, Man and Cybernetics. Budapest, Hungary, October 2016. 730−736
    [16] Yu H, Kim S. Passive sampling for regression. In: Proc. IEEE Int’l. Conf. on Data Mining. Sydney, Australia, December 2010. 1151−1156
    [17] Wu D. Pool-based sequential active learning for regression. IEEE Trans. on Neural Networks and Learning Systems, 2019, 30(5): 1348−1359 doi: 10.1109/TNNLS.2018.2868649
    [18] Wu D, Lin C T, Huang J. Active learning for regression using greedy sampling. Information Sciences, 2019, 474: 90−105 doi: 10.1016/j.ins.2018.09.060
    [19] Liu Z, Wu D. Integrating informativeness, representativeness and diversity in pool-based sequential active learning for regression. In: Proc. Int’l Joint Conf. on Neural Networks. Glasgow, UK, July 2020
    [20] Wu D, Huang J. Affect estimation in 3D space using multitask active learning for regression. IEEE Trans. on Affective Computing, to be published
    [21] Tong S, Koller D. Support vector machine active learning with applications to text classification. Journal of Machine Learning Research, 2001, 2(Nov): 45−66
    [22] Lewis D, Catlett J. Heterogeneous uncertainty sampling for supervised learning. In: Proc. Int’l Conf. on Machine Learning (ICML). New Brunswick, NJ, July 1994. 148−156
    [23] Lewis D, Gale W. A sequential algorithm for training text classifiers. In: Proc. ACM SIGIR Conf. on Research and Development in Information Retrieval. Dublin, Ireland, July 1994. 3−12
    [24] Grimm M, Kroschel K. Emotion estimation in speech using a 3D emotion space concept. Robust Speech Recognition and Understanding, Austria: InTech, 2007. 281−300
    [25] Grimm M, Kroschel K, Mower E, Narayanan S S. Primitivesbased evaluation and estimation of emotions in speech. Speech Communication, 2007, 49: 787−800 doi: 10.1016/j.specom.2007.01.010
    [26] Wu D, Parsons T D, Mower E, Narayanan S S. Speech emotion estimation in 3D space. In: Proc. IEEE Int’l Conf. on Multimedia & Expo (ICME). Singapore, July 2010. 737−742
    [27] Wu D, Parsons T D, Narayanan S S. Acoustic feature analysis in speech emotion primitives estimation. In: Proc. InterSpeech. Makuhari, Japan, September 2010
    [28] Dunn O J. Multiple comparisons among means. Journal of the American Statistical Association, 1961, 56: 62−64
    [29] Benjamini Y, Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B (Methodological), 1995, 57: 289−300 doi: 10.1111/j.2517-6161.1995.tb02031.x
    [30] Maaten L V D, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9: 2579−2605
  • 加载中
计量
  • 文章访问数:  21
  • HTML全文浏览量:  3
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-02-17
  • 录用日期:  2020-08-14

目录

    /

    返回文章
    返回