Semi-supervised Classification Model Based on Ladder Network and Improved Tri-training
-
摘要: 为了提高半监督深层生成模型的分类性能, 提出一种基于梯形网络和改进三训练法的半监督分类模型. 该模型在梯形网络框架有噪编码器的最高层添加3个分类器, 结合改进的三训练法提高图像分类性能. 首先, 用基于类别抽样的方法将有标记数据分为3份, 模型以有标记数据的标签误差和未标记数据的重构误差相结合的方式调整参数, 训练得到3个Large-margin Softmax分类器; 接着, 用改进的三训练法对未标记数据添加伪标签, 并对新的标记数据分配不同权重, 扩充训练集; 最后, 利用扩充的训练集更新模型. 训练完成后, 对分类器进行加权投票, 得到分类结果. 模型得到的梯形网络的特征有更好的低维流形表示, 可以有效地避免因为样本数据分布不均而导致的分类误差, 增强泛化能力. 模型分别在MNIST数据库, SVHN数据库和CIFAR10数据库上进行实验, 并且与其他半监督深层生成模型进行了比较, 结果表明本文所提出的模型得到了更高的分类精度.
-
关键词:
- 梯形网络 /
- 改进的三训练法 /
- 半监督学习 /
- Large-margin Softmax分类器
Abstract: In order to improve the classification performance of semi-supervised deep generation models, a new model based on ladder network and improved tri-training algorithm is proposed. This model adds three classifiers to the highest layer of the noisy coding layer of the ladder network, and improves the image classification performance by combining the improved tri-training. First, the labeled data is divided into three parts by using a class-based sampling method. The model adjusts the parameters by combining the labeled error of labeled data and the reconstruction error of unlabeled data, and is trained to get three Large-margin Softmax classifiers. Next, the improved tri-training algorithm is developed to add pseudo-labels to the unlabeled data, and different weights are assigned to the new labeled data to expand the training set. Finally, the expanded training set is applied to update the model. After the above practice, a weighted voting is performed on the classifier to obtain the classification result. The characteristics of the ladder network obtained by this model have better low-dimensional manifold representations, which can effectively avoid classification error caused by uneven distribution of sample data and enhance generalization ability. The model is tested on the MNIST, SVHN and CIFAR10 respectively. In comparison with other semi-supervised deep generation models, test results show that the model proposed in this paper has obtained state-of-the-art classification accuracy over existing semi-supervised learning methods. -
表 1 LN-TT SSC模型及混合方案在SVHN和CIFAR10数据集上的分类精度(%)
Table 1 Classification accuracy of LN-TT SSC model and hybrid scheme on SVHN and CIFAR10 (%)
模型 SVHN CIFAR10 基准实验 $(\Pi$+ softmax) 88.75 83.56 方案 A $(\Pi$+ LM-Softmax) 91.35 86.11 方案 B (VGG19 + softmax) 89.72 84.73 LN-TT SSC 93.05 88.64 表 2 MNIST数据集上的分类精度(%)
Table 2 Classification accuracy on MNIST dataset (%)
表 3 SVHN数据集上的分类精度(%)
Table 3 Classification accuracy on SVHN dataset (%)
-
[1] 刘建伟, 刘媛, 罗雄麟. 半监督学习方法. 计算机学报, 2015, 38(8): 1592-1617 doi: 10.11897/SP.J.1016.2015.01592Liu Jian-Wei, Liu Yuan, Luo Xiong-Lin. Semi-supervised learning. Journal of Computer Science, 2015, 38(8): 1592-1617 doi: 10.11897/SP.J.1016.2015.01592 [2] Chau V T N, Phung N H. Combining self-training and tri-training for course-level student classification. In: Proceedings of the 2018 International Conference on Engineering, Applied Sciences, and Technology, Phuket, Thailand: IEEE, 2018. 1−4 [3] Lu X C, Zhang J P, Li T, Zhang Y. Hyperspectral image classification based on semi-supervised rotation forest. Remote Sensing, 2017, 9(9): 924-938 doi: 10.3390/rs9090924 [4] Cheung E, Li Y. Self-training with adaptive regularization for S3VM. In: Proceedings of the 2017 International Joint Conference on Neural Networks, Anchorage, AR, USA: IEEE, 2017. 3633−3640 [5] Kvan Y, Wadysaw S. Convolutional and recurrent neural networks for face image analysis. Foundations of Computing and Decision Sciences, 2019, 44(3): 331-347 doi: 10.2478/fcds-2019-0017 [6] 罗建豪, 吴建鑫. 基于深度卷积特征的细粒度图像分类研究综述. 自动化学报, 2017, 43(8): 1306-1318Luo Jian-Hao, Wu Jian-Xin. A review of fine-grained image-classification based on depth convolution features. Acta Automatica Sinica, 2017, 43(8): 1306-1318 [7] 林懿伦, 戴星原, 李力, 王晓, 王飞跃. 人工智能研究的新前线: 生成对抗网络. 自动化学报, 2018, 44(5): 775-792Lin Yi-Lun, Dai Xing-Yuan, Li Li, Wang Xiao, Wang Fei-Yue. A new frontier in artificial intelligence research: generative adversarial networks. Acta Automatica Sinica, 2018, 44(5): 775-792 [8] Springenberg J T. Unsupervised and semi-supervised learning with categorical generative adversarial networks. In: Proceedings of the 2005 International Conference on Learning Representations, 2015. 1−20 [9] Salimans T, Goodfellow I, Zaremba W, Cheung V, Radford A, Chen X. Improved techniques for training GANs. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain: ACM, 2016. 2234−2242 [10] 付晓, 沈远彤, 李宏伟, 程晓梅. 基于半监督编码生成对抗网络的图像分类模型. 自动化学报, 2020, 46(3): 531-539Fu Xiao, Shen Yuan-Tong, Li Hong-Wei, Cheng Xiao-Mei. Image classification model based on semi-supervised coding generative adversarial networks. Acta Automatica Sinica, 2020, 46(3): 531-539 [11] Pezeshki M, Fan L X, Brakel P, Courville A, Bengio Y. Deconstructing the ladder network architecture. In: Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA: ACM, 2016. 2368−2376 [12] Rasmus A, Valpola H, Honkala M, Berglund M, Raiko T. Semi-supervised learning with ladder networks. Computer Science, 2015, 9(1): 1-9 [13] Zhao S, Song J, Ermon S. Learning hierarchical features from generative models. In Proceedings of the 34th International Conference on Machine Learning, Sydney, NSW, Australia: ACM, 2017. 4091−4099 [14] Saito K, Ushiku Y, Harada T. Asymmetric tri-training for unsupervised domain adaptation. International Conference on Machine Learning, 2017: 2988-2997 [15] Chen D D, Wang W, Gao W, Zhou Z H. Tri-net for semi-supervised deep learning. In: Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden: ACM, 2018. 2014−2020 [16] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. Computer Science and Information Systems, 2014. [17] Samuli L, Timo A. Temporal ensembling for semi-supervised learning. In: Proceedings of the 2017 International Conference on Learning Representation, 2017. 1−13 [18] Liu W Y, Wen Y D, Yu Z D, Yang M. Large-margin softmax loss for convolutional neural networks. In: Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA: ACM, 2016. 507−516 [19] Kilinc O, Uysal I. GAR: an efficient and scalable graph-based activity regularization for semi-supervised learning. Neuro-computing, 2018, 296(28): 46-54 [20] Maaten L, Hinton G. Visualizing data using t-SNE. Journal of Machine Learning Research, 2008, 9: 2579-2605 [21] Miyato T, Maeda S, Koyama M, Lshii S. Virtual adversarial training: a regularization method for supervised and semi-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 7(23): 99–112 期刊类型引用(23)
1. 徐康宇,刘元,李密青,杨圣祥,邹娟,郑金华. 进化高维多目标优化研究综述. 控制工程. 2023(08): 1436-1449 . 百度学术
2. 陈凤,张则强,刘俊琦,王沙沙. 考虑设施方向的双目标过道布置问题建模与优化. 计算机集成制造系统. 2022(06): 1717-1734 . 百度学术
3. 刘俊琦,张则强,龚举华,张裕. 受约束的过道布置问题建模及优化方法. 西南交通大学学报. 2022(06): 1376-1385 . 百度学术
4. 朱俊杰,李琳,曹力,刘晓平. 面向物流场景的关键设备与输送线混合布局方法. 厦门大学学报(自然科学版). 2021(04): 720-728 . 百度学术
5. 王闯,常丰田,高佳佳. 基于自动导引车单向导引路径网络的智能车间设备布局规划. 计算机集成制造系统. 2020(04): 939-946 . 百度学术
6. 封文清,巩敦卫. 基于在线感知Pareto前沿划分目标空间的多目标进化优化. 自动化学报. 2020(08): 1628-1643 . 本站查看
7. 李栓柱,李登攀,李灿. 基于免疫神经网络的双缸液压机同步PID控制. 机械工程师. 2019(02): 139-142 . 百度学术
8. 管超,张则强,李云鹏,贾林. 多行设备布局的一种多目标差分进化算法和线性规划混合方法. 机械工程学报. 2019(13): 160-174 . 百度学术
9. 叶坤武,魏思东,包涵. 面向驾驶舱布局优化的手部操作研究. 机械科学与技术. 2018(05): 747-752 . 百度学术
10. 周尤明,古华茂. CRQA_(OVTM)Agent支持的开放环境下协同制造装配. 自动化学报. 2018(07): 1333-1344 . 本站查看
11. 谢光,潘玉霞,李俊青. 求解混合流水车间调度的多目标优化算法. 计算机工程与设计. 2018(03): 885-889 . 百度学术
12. 王亚良,钱其晶,曹海涛,金寿松. 基于动态差分元胞多目标遗传算法的混合作业车间布局改善与优化. 中国机械工程. 2018(14): 1751-1757 . 百度学术
13. 杨挺,杨东,马光辉,吴晓锋. 多行多区域车间的设备布局优化设计方法. 工业工程与管理. 2018(05): 135-143 . 百度学术
14. 陈志旺,黄兴旺,陈志兴,赵子铮,黄丽芳. 区间多目标优化非支配排序云模型算法. 计算机工程与应用. 2017(22): 143-149 . 百度学术
15. 常春光. 基于AIA的精铜板带生产原料选购优化. 控制工程. 2017(12): 2496-2501 . 百度学术
16. 陈志旺,赵子铮,姚嘉楠,韩艳. 昂贵区间多目标优化空间数据挖掘求解策略. 控制与决策. 2017(09): 1599-1606 . 百度学术
17. 丁祥海,姚文鹏. 基于粒子群算法的多目标可重构设施布局方法. 中国机械工程. 2017(07): 852-861 . 百度学术
18. 宋中山,陈雯颖,孙翀,帖军. 一种基于数据挖掘的制造业工厂设备布局方法. 中南民族大学学报(自然科学版). 2017(04): 106-111 . 百度学术
19. 刘敏,戚佳金,刘晓胜,张树,姚友素,张芮. 一种电动汽车换电站的布局定容方法. 电气自动化. 2016(05): 26-28+67 . 百度学术
20. 刘丁,张新雨,陈亚军. 基于多目标人工鱼群算法的硅单晶直径检测图像阈值分割方法. 自动化学报. 2016(03): 431-442 . 本站查看
21. 李智. 智能优化算法研究及应用展望. 武汉轻工大学学报. 2016(04): 1-9+131 . 百度学术
22. 巩敦卫,刘益萍,孙晓燕,韩玉艳. 基于目标分解的高维多目标并行进化优化方法. 自动化学报. 2015(08): 1438-1451 . 本站查看
23. 陈志旺,白锌,杨七,黄兴旺,李国强. 区间多目标优化中决策空间约束、支配及同序解筛选策略. 自动化学报. 2015(12): 2115-2124 . 本站查看
其他类型引用(25)
-