-
摘要: 主成分分析（Principle component analysis，PCA）是一种被广泛应用的降维方法.然而经典PCA的构造基于L2-模导致了其对离群点和噪声点敏感，同时经典PCA也不具备稀疏性的特点.针对此问题，本文提出基于Lp-模的稀疏主成分分析降维方法（LpSPCA）.LpSPCA通过极大化带有稀疏正则项的Lp-模样本方差，使得其在降维的同时保证了稀疏性和鲁棒性.LpSPCA可用简单的迭代算法求解，并且当p≥1时该算法的收敛性可在理论上保证.此外通过选择不同的p值，LpSPCA可应用于更广泛的数据类型.人工数据及人脸数据上的实验结果表明，本文所提出的LpSPCA不仅具有较好的降维效果，并且具有较强的抗噪能力.Abstract: Principle component analysis (PCA) is a widely applied dimensionality reduction method. However, the construction of classical PCA is based on L2-norm, which leads to its sensitivity to outliers and noises, as well as sparsity. To solve this problem, the paper proposes a sparse principal component analysis method based on Lp-norm for dimensionality reduction (LpSPCA). In particular, LpSPCA maximizes the Lp-norm variance with sparse regularization term, which ensures the sparseness and robustness while reducing dimensions. LpSPCA can be solved by a simple iterative algorithm, and its convergence is theoretically guaranteed when p≥1. Besides, by choosing a different p, LpSPCA can be used for more types of data sets. Experimental results on both synthetic and human face data sets demonstrate that the proposed LpSPCA not only has better dimensionality reduction ability but also has strong anti-noise property.
-
Key words:
- Principal component analysis (PCA) /
- sparseness /
- robustness /
- dimensionality reduction /
- Lp-norm
-
图 1 PCA和LpSPCA在人工数据集上所得到的第1个主成分及 将数据用LpSPCA、PCA、RSPCA、LpPCA投影到该主成分所张成空间上的重构误差
Fig. 1 The -rst principal components of the arti-cial data set obtained by classical PCA and LpSPCA, and reconstruction errors in the spaces spanned by the -rst principal components obtained by LpSPCA, PCA, RSPCA and LpPCA, respectively
图 2 稀疏度k对LpSPCA平均重构误差在带遮盖噪声和哑噪声的Yale 人脸数据上的影响
Fig. 2 The influence of parsity k to the ARCE of LpSPCA on the occluded or dummy Yale face database
图 3 带遮盖噪声的Yale人脸数据及LpSPCA、PCA、RSPCA和LpPCA在该数据上用前30个相应主成分的重构效果图及 将数据用各方法投影到该1 ~ 70个主成分上的重构误差
Fig. 3 Occluded Yale face database and face reconstruction pictures of the data set constructed by the first thirty principal components obtained by LpSPCA,PCA,RSPCA and LpPCA and reconstruction errors in the spaces spanned by 1 ~ 70 principal components obtained by PCA,RSPCA,LpPCA and LpSPCA,respectively
图 4 原始Yale人脸数据及LpSPCA、PCA、RSPCA和LpPCA 在带哑噪声的Yale数据上用前30个相应主成分的\\重构效果图及将数据用 各方法投影到该1 ~ 70个主成分上的重构误差
Fig. 4 Original Yale face database and face reconstruction pictures of its dummy data set constructed by the first thirty principal components obtained by LpSPCA,PCA,RSPCA and LpPCA and reconstruction errors in the spaces spanned by 1 ~ 70 principal components obtained by PCA,RSPCA,LpPCA and LpSPCA,respectively
表 1 PCA和RSPCA在人工数据集2上提取的前两个主成分
Table 1 The first two PCs extracted by PCA and RSPCA on artificial data set 2
PCA RSPCA PC1 PC2 PC1 PC2 0.0608 -0.1238 0 0.0998 -0.0527 0.0422 0 0.4954 0.0389 0.1514 0 0.4048 -0.1343 -0.1754 -0.3608 0 0.167 -0.1062 0.3679 0 0.2074 0.1841 0 0 0.2135 0.0237 0 0 -0.1254 0.1933 0.2713 0 
下载: 导出CSV
表 2 LpPCA 在人工数据集2 上所提取的前两个主成分
Table 2 The first two PCs extracted by LpPCA on artificial data set 2
LpPCA (L0.5) LpPCA (L1) LpPCA (L1.5) LpPCA (L2) PC1 PC2 PC1 PC2 PC1 PC2 PC1 PC2 0.161 0.1896 -0.1924 -0.1657 0.0588 -0.1483 -0.1807 -0.1945 -0.1433 0.0855 0.1783 0.0865 -0.1833 -0.036 0.0399 0.1858 -0.1472 -0.1807 0.1511 -0.0542 -0.1771 0.0428 0.1957 -0.1054 0.0773 0.0125 -0.1056 0.131 -0.035 0.1067 0.2457 -0.0457 -0.0942 0.0566 0.1713 0.1395 -0.2173 -0.1576 -0.0184 0.0696 0.1948 -0.1646 0.05 -0.1428 -0.238 0.2375 0.1488 -0.1129 0.0831 0.1925 -0.1428 0.1499 0.2135 0.0899 0.0831 0.1925 -0.0991 0.1179 0.0085 -0.1306 -0.1254 0.1812 -0.0991 0.1179
下载: 导出CSV
表 3 LpSPCA 在人工数据集2 上所提取的前两个主成分
Table 3 The first two PCs extracted by LpSPCA on artificial data set 2
LpPCA (L0.5) LpPCA (L1) LpPCA (L1.5) LpPCA (L2) PC1 PC2 PC1 PC2 PC1 PC2 PC1 PC2 0 0.2978 0 -0.4474 0 0.1285 0 0.1126 0 -0.674 0 -0.1696 0 0.4496 -0.0625 0 0 0.0282 0 -0.383 0 0.4220 0 0.4801 0.2726 0 0.2567 0 0.5954 0 0 0.4073 -0.4027 0 0.3741 0 -0.2469 0 -0.5845 0 0.3247 0 0.3693 0 0 0 -0.353 0 0 0 0 0 0 0 0 0 0 0 0 0 0.2713 0 0 0
下载: 导出CSV
-
[1] Jolliffe I T. Principal Component Analysis. New York:Springer Verlag, 1986. 1-2 http://www.scirp.org/reference/ReferencesPapers.aspx?ReferenceID=1356395 [2] Tao D P, Lin X, Jin L W, Li X L. Principal component 2-D long short-term memory for font recognition on single Chinese characters. IEEE Transactions on Cybernetics, 2015, 46(3):756-765 https://www.researchgate.net/publication/274394889_Principal_Component_2-D_Long_Short-Term_Memory_for_Font_Recognition_on_Single_Chinese_Characters [3] Jolliffe I T, Cadima J. Principal component analysis:a review and recent developments. Philosophical Transactions of the Royal Society A:Mathematical, Physical and Engineering Sciences, 2016, 374(2065):20150202 doi: 10.1098/rsta.2015.0202 [4] 李伟, 焦松, 陆凌云, 杨明. 基于特征差异的仿真模型验证及选择方法. 自动化学报, 2014, 40(10):2134-2144 http://en.cnki.com.cn/Article_en/CJFDTOTAL-MOTO201410007.htmLi Wei, Jiao Song, Lu Ling-Yun, Yang Ming. Validation and selection of simulation model based on the feature differences. Acta Automatica Sinica, 2014, 40(10):2134-2144 http://en.cnki.com.cn/Article_en/CJFDTOTAL-MOTO201410007.htm [5] 韩敏, 许美玲, 任伟杰. 多元混沌时间序列的相关状态机预测模型研究. 自动化学报, 2014, 40(5):822-829 https://www.researchgate.net/publication/281891771_Research_on_multivariate_chaotic_time_series_prediction_using_mRSM_modelHan Min, Xu Mei-Ling, Ren Wei-Jie. Research on multivariate chaotic time series prediction using mRSM model. Acta Automatica Sinica, 2014, 40(5):822-829 https://www.researchgate.net/publication/281891771_Research_on_multivariate_chaotic_time_series_prediction_using_mRSM_model [6] 樊继聪, 王友清, 秦泗钊. 联合指标独立成分分析在多变量过程故障诊断中的应用. 自动化学报, 2013, 39(5):494-501 https://www.researchgate.net/publication/271231082_Combined_Indices_for_ICA_and_Their_Applications_to_Multivariate_Process_Fault_DiagnosisFan Ji-Cong, Wang You-Qing, Qin S Joe. Combined indices for ICA and their applications to multivariate process fault diagnosis. Acta Automatica Sinica, 2013, 39(5):494-501 https://www.researchgate.net/publication/271231082_Combined_Indices_for_ICA_and_Their_Applications_to_Multivariate_Process_Fault_Diagnosis [7] Zou H, Hastie T, Tibshirani R. Sparse principal component analysis. Journal of Computational and Graphical Statistics, 2006, 15(2):265-286 doi: 10.1198/106186006X113430 [8] Shen H P, Huang J Z. Sparse principal component analysis via regularized low rank matrix approximation. Journal of Multivariate Analysis, 2008, 99(6):1015-1034 doi: 10.1016/j.jmva.2007.06.007 [9] Johnstone I M, Lu A Y. On consistency and sparsity for principal components analysis in high dimensions. Journal of the American Statistical Association, 2009, 104(486):682-693 doi: 10.1198/jasa.2009.0121 [10] Journée M, Nesterov Y, Richtárik P, Sepulchre R. Generalized power method for sparse principal component analysis. The Journal of Machine Learning Research, 2010, 11:517-553 http://www.docin.com/p-721019144.html [11] Croux C, Filzmoser P. Robust factorization of a data matrix. In:Proceedings of the 13th Computational Statistics Symposium. Bristol, Great Britain:Physica-Verlag HD, 1998. 245-250 http://cn.bing.com/academic/profile?id=aa85ede68a85c9d204516038d4913d33&encoded=0&v=paper_preview&mkt=zh-cn [12] Brooks J P, Dulá J H, Boone E L. A pure L1-norm principal component analysis. Computational Statistics and Data Analysis, 2013, 61:83-98 doi: 10.1016/j.csda.2012.11.007 [13] Ke Q F, Kanade T. Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming. In:Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, CA, USA:IEEE, 2005, 1:739-746 [14] Ding C, Zhou D, He X F, Zha H Y. R1-PCA:rotational invariant L1-norm principal component analysis for robust subspace factorization. In:Proceedings of the 23rd International Conference on Machine Learning. Pittsburgh, USA:ACM, 2006. 281-288 http://cn.bing.com/academic/profile?id=aa5ba5364c85c7d92f7a58a86f9607bd&encoded=0&v=paper_preview&mkt=zh-cn [15] Kwak N. Principal component analysis based on L1-norm maximization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(9):1672-1680 doi: 10.1109/TPAMI.2008.114 [16] Liang Z Z, Xia S X, Zhou Y, Zhang L, Li Y F. Feature extraction based on L_p-norm generalized principal component analysis. Pattern Recognition Letters, 2013, 34(9):1037-1045 doi: 10.1016/j.patrec.2013.01.030 [17] Kwak N. Principal component analysis by L_p-norm maximization. IEEE Transactions on Cybernetics, 2014, 44(5):594-609 doi: 10.1109/TCYB.2013.2262936 [18] Meng D Y, Zhao Q, Xu Z B. Improve robustness of sparse PCA by L1-norm maximization. Pattern Recognition, 2012, 45(1):487-497 doi: 10.1016/j.patcog.2011.07.009 [19] Wang R, Nie F P, Yang X J, Gao F F, Yao M L. Robust 2DPCA with non-greedy ι1-norm maximization for image analysis. IEEE Transactions on Cybernetics, 2015, 45(5):1108-1112 [20] Lu G F, Zou J, Wang Y. L1-norm and maximum margin criterion based discriminant locality preserving projections via trace Lasso. Pattern Recognition, 2016, 55:207-214 doi: 10.1016/j.patcog.2016.01.029 [21] Li C N, Shao Y H, Deng N Y. Robust L1-norm two-dimensional linear discriminant analysis. Neural Networks, 2015, 65:92-104 doi: 10.1016/j.neunet.2015.01.003 [22] Jolliffe I T, Trendafilov N T, Uddin M. A modified principal component technique based on the LASSO. Journal of Computational and Graphical Statistics, 2003, 12(3):531-547 doi: 10.1198/1061860032148 [23] Yu H, Yang J. A direct LDA algorithm for high-dimensional data——with application to face recognition. Pattern Recognition, 2001, 34(10):2067-2070 doi: 10.1016/S0031-3203(00)00162-X [24] Yang J, Zhang D, Frangi A F, Yang J Y. Two-dimensional PCA:a new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(1):131-137 doi: 10.1109/TPAMI.2004.1261097 [25] He X F, Yan S C, Hu Y X, Niyogi P, Zhang H J. Face recognition using Laplacianfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(3):328-340 doi: 10.1109/TPAMI.2005.55 [26] Cai D, He X, Han J, Zhang H J. Orthogonal Laplacianfaces for face recognition. IEEE Transactions on Image Processing, 2006, 15(11):3608-3614 doi: 10.1109/TIP.2006.881945 -
计量
- 文章访问数: 2373
- HTML全文浏览量: 200
- PDF下载量: 1318
- 被引次数: 0
