基于极限学习机参数迁移的域适应算法

许夙晖; 慕晓冬; 柴栋; 罗畅

doi:10.16383/j.aas.2018.c160818

基于极限学习机参数迁移的域适应算法

doi: 10.16383/j.aas.2018.c160818

许夙晖^1, ,,
慕晓冬^1,,
柴栋^2,,
罗畅^3,

1.
火箭军工程大学信息工程系西安 710025
2.
北京航空工程技术研究中心北京 100076
3.
空军工程大学防空反导学院西安 710051

详细信息

作者简介:
慕晓冬  火箭军工程大学信息工程系教授.主要研究方向为智能信息处理与计算机仿真.E-mail:muxiaodong402@163.com

柴栋  北京航空工程技术研究中心工程师, 博士.主要研究方向为智能信息处理.E-mail:chaibaodong@126.com

罗畅  空军工程大学防空反导学院博士研究生.主要研究方向为智能信息处理.E-mail:luochang1988@126.com

计量
- 文章访问数: 2305
- HTML全文浏览量: 451
- PDF下载量: 929
- 被引次数: 0
出版历程
- 收稿日期: 2016-12-11
- 录用日期: 2017-03-30
- 刊出日期: 2018-02-20

Domain Adaption Algorithm with ELM Parameter Transfer

XU Su-Hui^{1
, ,},
MU Xiao-Dong^1
,,
CHAI Dong^2
,,
LUO Chang^3
,

1.
Department of Information Engineering, Rocket Force University of Engineering, Xi'an 710025
2.
Beijing Aeronautical Technology Research Institute, Beijing 100076
3.
Air and Missile Defense College, Air Force Engineering University, Xi'an 710051

More Information

Author Bio:
Professor in the Department of Information Engineering, Rocket Force University of Engineering. His research interest covers intelligent information processing and computer simulation

Ph. D, Engineer at Beijing Aeronautical Technology Research Institute. His main research interest covers intelligent information processing

Ph. D. candidate at the Air and Missile Defense College, Air Force Engineering University. His main research interest is intelligent information processing

Corresponding author: XU Su-Hui Ph. D. candidate in the Department of Information Engineering, Rocket Force University of Engineering. Her main research interest is remote sensing image processing. Corresponding author of this paper

摘要

摘要: 针对含少量标签样本的迁移学习问题，本文提出了基于极限学习机（Extreme learning machine，ELM）参数迁移的域适应算法，其核心思想是将目标域的ELM分类器参数投影到源域参数空间中，使其最大限度地与源域的分类器参数分布相同.此外，考虑到迁移中有可能带来负迁移的情况，在目标函数中引入正则项约束.本文算法与以往的域适应算法相比优势在于，其分类器参数以及转移矩阵是同时优化得到的，并且其目标函数求解过程相对简单.实验结果表明，与主流的域适应算法相比，本文算法在精度与效率上都表现出明显的优势.
- 域适应 /
- 迁移学习 /
- 极限学习机 /
- 正则化 /
- 中层语义特征 /
- 深度特征
Abstract: In allusion to transfer learning problem with a small number of labeled samples, a domain adaption method through transferring extreme learning machine (ELM) parameters is proposed in this paper. The core idea is projecting the target ELM parameters on to the source and making the parameters maximally aligned with the source. In addition, considering the transformation may cause negative transfer, a regular term is added to the objective function. Unlike the existing domain adaption method, the parameters of classifier and the transformation matrix can be calculated simultaneously, and the objective function can be easily solved. Experiments demonstrate the proposed method has potential advantages in terms of accuracy and efficiency compared to the state-of-the-art approaches.
- Domain adaption /
- transfer learning /
- extreme learning machine /
- regularization /
- middle-level feature /
- deep feature
注释:

1) 本文责任编委王占山

HTML全文

图 1 极限学习机网络结构

Fig. 1 ELM network

下载: 全尺寸图片幻灯片

图 2 bing-caltech数据集分类精度

Fig. 2 The accuracy on bing-caltech dataset

下载: 全尺寸图片幻灯片

图 3 分类精度随$L$变化曲线

Fig. 3 The accuracy curves varying with $L$

下载: 全尺寸图片幻灯片

图 4 $C_1$与$C_2$不同取值下的精度曲线

Fig. 4 The accuracy curves varying with $C_1$ and $C_2$

下载: 全尺寸图片幻灯片

表 1 本文方法与主流算法适用范围对比

Table 1 The application comparison between DAPT and previous methods

	ARC-t	GFK	HFA	MMDT	DAPT
多类分类	√	√		√	√
大规模数据集		√		√	√
解决异构特征			√	√	√

下载: 导出CSV

表 2 使用BOVW特征时算法的分类精度(%)

Table 2 Accuracy for all the methods when using BOVW feature (%)

	SVMs	SVMt	ELMs	ELMt	ARC-t	GFK	HFA	MMDT	DAPT_$n$	DAPT
a$\rightarrow$w	30.30	50.02	40.21	69.42	56.77	57.10	55.71	64.85	63.60	70.25
a$\rightarrow$d	32.20	47.28	38.66	56.42	54.37	54.14	50.18	54.37	50.75	58.19
a$\rightarrow$c	39.55	26.50	42.23	35.99	31.66	37.95	37.03	39.73	31.96	44.29
w$\rightarrow$a	28.42	39.28	38.42	52.73	43.28	41.25	43.44	50.47	48.79	54.82
w$\rightarrow$d	63.74	45.63	70.16	55.28	55.91	75.51	71.29	62.72	52.76	71.93
w$\rightarrow$c	24.55	26.15	33.31	33.74	30.50	31.59	31.92	34.81	30.45	38.66
d$\rightarrow$a	31.54	39.29	38.49	51.16	42.76	40.52	42.45	50.40	47.52	53.98
d$\rightarrow$w	66.06	47.45	80.74	67.96	59.64	80.27	78.33	74.30	62.68	81.92
d$\rightarrow$c	27.72	26.28	33.80	36.52	31.29	33.97	33.52	35.83	31.43	40.56
c$\rightarrow$a	37.18	38.57	44.76	52.06	44.88	38.38	44.11	51.00	48.83	55.98
c$\rightarrow$w	26.83	48.49	36.34	67.68	55.89	52.24	55.90	62.81	62.25	69.17
c$\rightarrow$d	34.53	45.98	40.63	57.09	54.76	57.45	50.64	52.68	49.09	59.17
acc (%)	36.88	40.08	44.81	53.00	55.77	50.03	49.54	52.83	48.34	58.24
time (s)	0.059	0.009	0.17	0.15	4.0245	0.264	3.26	0.485	0.18	0.19

下载: 导出CSV

表 3 使用深度特征时算法的分类精度(%)

Table 3 Accuracy for all the methods when using deep feature (%)

	SVMs	SVMt	ELMs	ELMt	ARC-t	GFK	HFA	MMDT	DAPT_$n$	DAPT
a$\rightarrow$w	72.81	82.00	77.53	89.92	91.04	92.41	90.23	90.92	92.47	92.83
a$\rightarrow$d	75.12	84.29	82.17	92.60	93.31	93.76	94.19	93.58	94.76	95.00
a$\rightarrow$c	76.00	67.35	80.01	76.03	76.83	81.13	80.05	81.38	78.59	82.14
w$\rightarrow$a	72.54	79.82	79.46	86.97	87.80	89.62	87.91	88.85	89.83	90.90
w$\rightarrow$d	92.24	83.74	95.43	90.94	92.68	97.96	96.25	95.24	94.45	98.46
w$\rightarrow$c	66.94	65.32	70.86	75.95	77.70	79.12	75.40	76.47	78.28	81.50
d$\rightarrow$a	79.26	78.39	83.64	86.74	86.96	89.07	87.21	87.33	90.09	91.70
d$\rightarrow$w	92.11	80.53	96.32	89.15	90.42	98.02	96.65	96.00	92.21	98.34
d$\rightarrow$c	73.41	67.08	76.51	76.03	77.96	79.77	76.67	75.54	78.83	82.41
c$\rightarrow$a	80.29	79.57	86.50	86.16	87.13	88.94	87.92	89.14	89.25	90.91
c$\rightarrow$w	70.66	82.66	74.17	89.38	90.36	90.71	89.33	89.64	91.79	92.34
c$\rightarrow$d	76.93	84.33	81.50	90.51	91.77	90.80	90.78	91.02	93.11	93.82
acc (%)	77.36	77.92	82.01	85.87	87.00	89.27	87.72	87.93	88.64	90.86
time (s)	0.204	0.039	0.61	0.55	6.67	13.55	9.26	2.84	1.66	1.74

下载: 导出CSV

参考文献(19)

[1]	庄福振, 罗平, 何清, 史忠植.迁移学习研究进展.软件学报, 2015, 26(1):26-39 http://kns.cnki.net/KCMS/detail/detail.aspx?filename=rjxb201501003&dbname=CJFD&dbcode=CJFQ Zhuang Fu-Zhen, Luo Ping, He Qing, Shi Zhong-Zhi. Survey on transfer learning research. Journal of Software, 2015, 26 (1):26-39 http://kns.cnki.net/KCMS/detail/detail.aspx?filename=rjxb201501003&dbname=CJFD&dbcode=CJFQ
[2]	Pan S J, Yang Q. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22 (10):1345-1359 doi: 10.1109/TKDE.2009.191
[3]	Weiss K, Khoshgoftaar T M, Wang D D. A survey of transfer learning. Journal of Big Data, 2016, 3 (1):Article No.9 doi: 10.1186/s40537-016-0043-6
[4]	Wang C, Mahadevan S. Heterogeneous domain adaptation using manifold alignment. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence. Barcelona, Spain: AAAI, 2011, 2: 1541-1546
[5]	Domeniconi G, Moro G, Pagliarani A, Pasolini R. Markov chain based method for in-domain and cross-domain sentiment classification. In: Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K). Lisbon, Portugal: IEEE, 2015. 127-137
[6]	Kulis B, Saenko K, Darrell T. What you saw is not what you get: domain adaptation using asymmetric kernel transforms. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence, RI, USA: IEEE, 2011. 1785-1792
[7]	Duan L X, Xu D, Tsang I W. Learning with Augmented Features for Heterogeneous Domain Adaptation. In: Proceedings of the 29th International Conference on Machine Learning. Edinburgh, UK: ARXIV, 2012. 711-718
[8]	Zhu Y, Chen Y, Lu Z, Pan S J, Xue G R, Yu Y, Yang Q. Heterogeneous Transfer Learning for Image Classification. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI, 2014. 1717-1724
[9]	Oquab M, Bottou L, Laptev I, Sivic J. Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the 2004 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, OH, USA: IEEE, 2014. 1717-1724
[10]	van Opbroek A, Ikram M A, Vernooij M W, de Bruijne M. Transfer learning improves supervised image segmentation across imaging protocols. IEEE Transactions on Medical Imaging, 2015, 34 (5):1018-1030 doi: 10.1109/TMI.2014.2366792
[11]	舒醒, 于慧敏, 郑伟伟, 谢奕, 胡浩基, 唐慧明.基于边际Fisher准则和迁移学习的小样本集分类器设计算法.自动化学报, 2016, 42 (9):1313-1321 http://www.aas.net.cn/CN/abstract/abstract18920.shtml Shu Xing, Yu Hui-Min, Zheng Wei-Wei, Xie Yi, Hu Hao-Ji, Tang Hui-Ming. Classifier-designing algorithm on a small dataset based on margin Fisher criterion and transfer learning. Acta Automatica Sinica, 2016, 42 (9):1313-1321 http://www.aas.net.cn/CN/abstract/abstract18920.shtml
[12]	Nam J, Kim S. Heterogeneous defect prediction. In: Proceedings of the 10th joint Meeting on Foundations of Software Engineering. Bergamo, Italy: ACM, 2015. 508-519
[13]	Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. In: Proceedings of the 11th European Conference on Computer Vision: Part Ⅳ. Heraklion, Greece: Springer, 2010. 213-226
[14]	Gong B Q, Shi Y, Sha F, Grauman K. Geodesic flow kernel for unsupervised domain adaptation. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence, RI, USA: IEEE, 2012. 2066-2073
[15]	Hoffman J, Rodner E, Donahue J, Darrell T, Saenko K. Efficient learning of domain-invariant image representations. In: Proceedings of the 2013 International Conference on Learning Representations. Arizona, USA, 2013.
[16]	Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of the 2004 IEEE International Joint Conference on Neural Networks. Budapest, Hungary: IEEE, 2004, 2: 985-990
[17]	Horata P, Chiewchanwattana S, Sunat K. Robust extreme learning machine. Neurocomputing, 2013, 102:31-44 doi: 10.1016/j.neucom.2011.12.045
[18]	Huang G, Song S J, Gupta J N D, Wu C. Semi-supervised and unsupervised extreme learning machines. IEEE Transactions on Cybernetics, 2014, 44 (12):2405-2417 doi: 10.1109/TCYB.2014.2307349
[19]	Bergamo A, Torresani L. Exploiting weakly-labeled Web images to improve object classification: a domain adaptation approach. In: Proceedings of the 23rd International Conference on Neural Information Processing Systems. Vancouver, British Columbia, Canada: Curran Associates Inc., 2010. 181-189