一种面向多源领域的实例迁移学习

张倩; 李明; 王雪松; 程玉虎; 朱美强

doi:10.3724/SP.J.1004.2014.01176

一种面向多源领域的实例迁移学习

doi: 10.3724/SP.J.1004.2014.01176

1.
中国矿业大学信息与电气工程学院徐州 221116

基金项目:

国家自然科学基金（61072094，61273143），江苏省自然科学基金（BK 20130207），教育部新世纪优秀人才支持计划（NCET-10-0765），教育部高等学校博士学科点专项科研基金（20110095110016，20120095110025）资助

详细信息

作者简介:
李明中国矿业大学教授. 主要研究方向为智能控制和模式识别.E-mail：liming@cumt.edu.cn

计量
- 文章访问数: 3660
- HTML全文浏览量: 249
- PDF下载量: 1356
- 被引次数: 0
出版历程
- 收稿日期: 2013-05-02
- 修回日期: 2013-10-01
- 刊出日期: 2014-06-20

Instance-based Transfer Learning for Multi-source Domains

1.
School of Information and Electrical Engineering, China University of Mining and Technology, Xuzhou 221116

Funds:

Supported by National Natural Science Foundation of China (61072094, 61273143), Natural Science Foundation of Jiangsu Province (BK20130207), Program for New Century Excellent Talents in University (NCET-10-0765), and Specialized Research Fund for the Doctoral Program of Higher Education of China (20 110095110016, 20120095110025)

摘要

摘要: 在迁移学习最大的特点就是利用相关领域的知识来帮助完成目标领域中的学习任务，它能够有效地在相似的领域或任务之间进行信息的共享和迁移，使传统的从零开始的学习变成可积累的学习，具有成本低、效率高等优点.针对源领域数据和目标领域数据分布类似的情况，提出一种基于多源动态TrAdaBoost的实例迁移学习方法.该方法考虑多个源领域知识，使得目标任务的学习可以充分利用所有源领域信息，每次训练候选分类器时，所有源领域样本都参与学习，可以获得有利于目标任务学习的有用信息，从而避免负迁移的产生.理论分析验证了所提算法较单源迁移的优势，以及加入动态因子改善了源权重收敛导致的权重熵由源样本转移到目标样本的问题.实验结果验证了此算法在提高识别率方面的优势.
- 多源 /
- TrAdaBoost /
- 实例迁移 /
- 迁移学习
Abstract: The most remarkable characteristic of transfer learning is that it can employ the knowledge in relative domains to help perform the learning tasks in the domain of the target. With the use of different fields of knowledge for target task learning, transfer learning can transfer and share the information between similar domains or tasks, making the traditional learning from scratch an addable one, which implies that the learning efficiency is higher and the cost is lower. For the specific situation that the shared knowledge in the domains of the source and the target are sample data with similar distribution, an instance transfer learning method based on multi-sources dynamic TrAdaBoost is put forward. Integrated with the knowledge in multiple source domains, this method makes the target task learning the one that is able to make good use of the information of all source domains. Whenever candidate classifiers are trained, all the samples in all source domains are involved in learning, and the information conducive to target task learning can be obtained, so that negative transfer can be avoided. The theoretical analysis suggests that the given algorithm is better than the single source transfer. By means of adding the dynamic factor, this algorithm improves the defect that weight entropy drifts from source to target instances. The experimental results support that the given algorithm has the advantage of improving the recognition rate.
- Multi-source /
- TrAdaBoost /
- instance transfer /
- transfer learning

HTML全文

参考文献(16)

[1]	Wang Hao, Gao Yang, Chen Xing-Guo. Transfer of reinforcement learning: the state of the art. Acta Electronica Sinica, 2008, 36(12): 39-43 (王皓, 高阳, 陈兴国. 强化学习中的迁移: 方法和进展. 电子学报, 2008, 36(12): 39-43)
[2]	Yang Q. An introduction to transfer learning. In: Proceedings of the 4th International Advanced Data Mining and Applications Conference. Berlin, Heidelberg: Springer-Verlag, 2008. 1
[3]	Taylor M E, Stone P. Transfer learning for reinforcement learning domains: a survey. Journal of Machine Learning, 2009, 10: 1633-1685
[4]	Meng Jia-Na. Research on the Application of Transfer Learning on Text Classification [Ph.D. dissertation], Dalian University of Technology, China, 2011 (孟佳娜.迁移学习在文本分类中的应用 [博士学位论文]. 大连理工大学, 中国, 2011)
[5]	Gao Jun, Huang Li-Li, Sun Chang-Yin. A local weighted mean based domain adaptation learning framework. Acta Automatica Sinica, 2013, 39(7): 1037-1052 (皋军, 黄丽莉, 孙长银. 一种基于局部加权均值的领域适应学习框架. 自动化学报, 2013, 39(7): 1037-1052)
[6]	Dai W Y, Yang Q, Xue G R, Yu Y. Boosting for transfer learning. In: Proceedings of the 24th International Conference on Machine Learning. New York, USA: ACM, 2007. 193-200
[7]	Freund Y, Schapire R E. A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 1997, 55(1): 119-139
[8]	Pardoe D, Stone P. Boosting for regression transfer. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10). Haifa, Israel, 2010. 863-870
[9]	Eaton E, des Jardins M. Set-based boosting for instance-level transfer. In: Proceedings of the 2009 IEEE International Conference on Data Mining Workshops. Miami, FL: IEEE, 2009. 422-428
[10]	Eaton E. Selective Knowledge Transfer for Machine Learning [Ph.D. dissertation], University of Maryland Baltimore County, USA, 2009
[11]	Ying Wen-Hao, Wang Shi-Tong, Deng Zhao-Hong, Wang Jun. Support vector machine for domain adaptation based on class distributiond. Acta Automatica Sinica, 2012, 39(8): 1273-1288 (应文豪, 王士同, 邓赵红, 王骏. 基于类分布的领域自适应支持向量机. 自动化学报, 2012, 39(8): 1273-1288)
[12]	Al-Samir S, Reddy C K. Adaptive boosting for transfer learning using dynamic updates. In: Proceedings of the 2011 European Conference on Machine Learning and Knowledge Discovery in Databases. Berlin, Heidelberg, Springer-Verlag, 2011. 60-75
[13]	Eaton E, des Jardins M. Set-based boosting for instance-level transfer. In: Proceedings of the 2009 IEEE International Conference on Data Mining Workshops. Miami, FL: IEEE, 2009. 422-428
[14]	Zhang Qian, Li Hai-Gang, Li Ming, Cheng Yu-Hu. Instance-based transfer learning method with multi-source dynamic TrAdaBoost. Journal of China University of Mining and Technology, 2014, 43(4): 701-708(张倩, 李海港, 李明, 程玉虎. 基于多源动态TrAdaBoost的实例迁移学习方法. 中国矿业大学学报, 2014, 43(4): 701-708)
[15]	Griffin G, Holub A, Perona P. Caltech-256 Object Category Dataset, Technical Report 7694, California Institute of Technology, USA, 2007
[16]	Li Qiu-Jie, Mao Yao-Bin. AUC optimization boosting based on data rebalance. Acta Automatica Sinica, 2012, 38(9): 1467-1475 (李秋洁, 茅耀斌. 基于数据重平衡的AUC优化Boosting算法. 自动化学报, 2012, 38(9): 1467-1475)