Restricted Gaussian Classification Network
-
摘要: 针对基于一元高斯函数估计属性边缘密度的朴素贝叶斯分类器不能有效利 用属性之间的依赖信息和使用多元高斯函数估计属性联合密度的完全贝叶斯分类器 易于导致对数据的过度拟合而且高阶协方差矩阵的计算也非常困难等情况,在建立 属性联合密度分解与组合定理和属性条件密度计算定理的基础上,将朴素贝叶斯分类 器的属性选择、分类准确性标准和属性父结点的贪婪选择相结合,进行约束高斯 分类网学习与优化,并依据贝叶斯网络理论,对贝叶斯衍生分类器中属性为类提供 的信息构成进行分析.使用UCI数据库中连续属性分类数据进行实验,结果显示,经过 优化的约束高斯分类网具有良好的分类准确性.Abstract: Naive Bayes classifier using unitary Gaussian function to estimate attribute marginal density can not effectively use dependency information between attributes, and the full Bayes classifier using multivariate Gaussian function to estimate attribute joint density often leads to over fitting and the difficulty to calculate high-order covariance matrix. In this paper, based on Gaussian network theory, a decomposition and combination theorem for attribute joint density and a calculation theorem for attribute conditional density are established. A restricted Gaussian classification network is presented by combining the attribute selection of naive Bayes classifier, the evaluation criteria of classification accuracy and the greedy search of attribute parent nodes. The information composition of attributes provided for class is analyzed in Bayesian derivative classifiers. Experiment and analysis are done by using data sets with continuous attributes in UCI. The results show that restricted Gaussian classification networks have very good classification accuracy.
-
[1] Wang Shuang-Cheng, Du Rui-Jie, Liu Ying. The learning and optimization of full Bayes classifiers with continuous attributes. Chinese Journal of Computers, 2012, 35(10): 2129 -2138(王双成, 杜瑞杰, 刘颖. 连续属性完全贝叶斯分类器的学习与优化. 计算机学报, 2012, 35(10): 2129-2138) [2] Wang S C, Xu G L, Du R J. Restricted Bayesian classification networks. Science China Information Sciences, 2013, 56(7): 1-15 [3] Friedman N, Geiger D, Goldszmidt M. Bayesian network classifiers. Machine Learning, 1997, 29(2-3): 131-163 [4] Jing Y S, Pavlović V, Rehg J M. Boosted Bayesian network classifiers. Machine Learning, 2008, 73(2): 155-184 [5] Wang Zhong-Feng, Wang Zhi-Hai. An optimization algorithm of Bayesian network classifiers by derivatives of conditional log likelihood. Chinese Journal of Computers, 2012, 35(2): 364-374(王中锋, 王志海. 基于条件对数似然函数导数的贝叶斯网络分类器优化算法. 计算机学报, 2012, 35(2): 364-374) [6] Webb G I, Boughton J R, Zheng F, Ting K M, Salem H. Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification. Machine Learning, 2012, 86(2): 233-272 [7] Feng Yue-Jin, Zhang Feng-Bin. Max-relevance min-redundancy restrictive BAN classifier learning algorithm. Journal of Chongqing University, 2014, 37(6): 71-77(冯月进, 张凤斌. 最大相关最小冗余限定性贝叶斯网络分类器学习算法. 重庆大学学报, 2014, 37(6): 71-77) [8] López-Cruz P L, Larrañaga P, DeFelipe J, Bielza C. Bayesian network modeling of the consensus between experts: an application to neuron classification. International Journal of Approximate Reasoning, 2014, 55(1): 3-22 [9] John G H, Langley P. Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence (UAI-1995). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1995. 338-345 [10] Pérez A, Larrañaga P, Inza I. Supervised classification with conditional Gaussian networks: increasing the structure complexity from naive Bayes. International Journal of Approximate Reasoning, 2006, 43(1): 1-25 [11] Pérez A, Larrañaga P, Inza I. Bayesian classifiers based on kernel density estimation: flexible classifiers. International Journal of Approximate Reasoning, 2009, 50(2): 341-362 [12] Bounhas M, Mellouli K, Prade H, Serrurier M. Possibilistic classifiers for numerical data. Soft Computing, 2013, 17(5): 733-751 [13] He Y L, Wang R, Kwong S, Wang X Z. Bayesian classifiers based on probability density estimation and their applications to simultaneous fault diagnosis. Information Sciences, 2014, 259(2): 252-268 [14] Xia Zhan-Guo, Xia Shi-Xiong, Cai Shi-Yu, Wan Ling. Semi-supervised Gaussian process classification algorithm addressing the class imbalance. Journal on Communications, 2013, 34(5): 42-51(夏战国, 夏士雄, 蔡世玉, 万玲. 类不均衡的半监督高斯过程分类算法. 通信学报, 2013, 34(5): 42-51) [15] Liu G Q, Wu J X, Zhou S P. Probabilistic classifiers with a generalized Gaussian scale mixture prior. Pattern Recognition, 2013, 46(1): 332-345 [16] Dong W Y, Zhou M C. Gaussian classifier-based evolutionary strategy for multimodal optimization. IEEE Transactions on Neural Networks and Learning Systems, 2014, 25(6): 1200-1216 [17] Pavani S K, Gomez D D, Frangi A F. Gaussian weak classifiers based on co-occurring Haar-like features for face detection. Pattern Analysis and Applications, 2014, 17(2): 431439 [18] Geiger D, Heckerman D. Learning Gaussian Networks. Technical Report MSR-TR-94-10, Microsoft Research, Advanced Technology Division, 1994. [19] Pearl J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, California, USA: Morgan Kaufmann Publishers Inc., 1988. 383-408 [20] Shachter R D, Kenley C R. Gaussian influence diagrams. Management Science, 1989, 35(5): 527-550 [21] Kohavi R. A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI). San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1995. 1137-1143 [22] Quinlan J R. Induction of decision trees. Machine Learning, 1986, 1(1): 81-106 [23] Wang Shuang-Cheng, Yuan Sen-Miao. Research on learning Bayesian networks structure with missing data. Journal of Software, 2004, 15(7): 1042-1048(王双成, 苑森淼. 具有丢失数据的贝叶斯网络结构学习研究. 软件学报, 2004, 15(7): 1042-1048) [24] Fayyad U M, Irani K B. Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the 13th International Joint Conference on Artificial Intelligence. Chambery, France, 1993. 1022-1027 [25] Demşar J. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 2006, 7(1): 1-30
点击查看大图
计量
- 文章访问数: 1541
- HTML全文浏览量: 43
- PDF下载量: 1130
- 被引次数: 0