基于相关性的Swarm联邦降维方法

李文平; 杜选

doi:10.16383/j.aas.c220690

基于相关性的Swarm联邦降维方法

doi: 10.16383/j.aas.c220690

李文平^1,,
杜选^1,

1.
嘉兴学院信息科学与工程学院嘉兴 314001

基金项目: 嘉兴市科技特派员专项项目(K2022A015)资助

详细信息

作者简介:
李文平：嘉兴学院信息科学与工程学院副教授. 主要研究方向为隐私保护技术. 本文通讯作者. E-mail: liwenping@hrbeu.edu.cn

杜选：嘉兴学院信息科学与工程学院副教授. 主要研究方向为隐私保护技术. E-mail: duxuan@zjxu.edu.cn

计量
- 文章访问数: 111
- HTML全文浏览量: 84
- 被引次数: 0
出版历程
- 收稿日期: 2022-09-01
- 录用日期: 2023-04-12
- 网络出版日期: 2023-10-18

Swarm Federated Dimensionality Reduction Method Based on Correlation

LI Wen-Ping^1
,,
DU Xuan^1
,

1.
College of Information Science and Engineering, Jiaxing University, Jiaxing 314001

Funds: Supported by JiaXing Science and Technology Commissioner Special Project (K2022A015)

More Information

Author Bio:
LI Wen-Ping　Associate professor at the College of Information Science and Engineering, Jiaxing University. His main research interest is privacy protection. Corresponding author of this paper

DU Xuan　Associate professor at the College of Information Science and Engineering, Jiaxing University. His main research interest is privacy protection

摘要

摘要: 联邦学习(Federated learning, FL)在解决人工智能(Artificial intelligence, AI)面临的隐私泄露和数据孤岛问题方面具有显著优势. 针对联邦学习的已有研究未考虑联邦数据之间的关联性和高维性问题, 提出一种基于联邦数据相关性的去中心化联邦降维方法. 该方法基于Swarm学习(Swarm learning, SL)思想, 通过分离耦合特征, 构建典型相关分析(Canonical correlation analysis, CCA)的Swarm联邦框架, 以提取Swarm节点的低维关联特征. 为保护协作参数的隐私安全, 还构建了一种随机扰乱策略来隐藏Swarm特征隐私. 在真实数据集上的实验验证了所提方法的有效性.
- 隐私保护 /
- Swarm学习 /
- 联邦学习 /
- 典型相关分析
Abstract: Federated learning (FL) has significant advantages in solving the problems of privacy disclosure and data islands faced by artificial intelligence (AI). Previous studies on federated learning do not consider the problems of relevance and high dimensionality of data distributed among different federations. Based on the relevance of federated data, a decentralized federated dimensionality reduction method is proposed. This method draws on the idea of Swarm learning (SL). Based on the separation of coupling features, a Swarm framework for canonical correlation analysis (CCA) is constructed to extract the low dimensional correlation features of Swarm nodes. In order to protect the privacy of collaboration parameters, a random disturbance strategy is also constructed to hide the privacy of Swarm features. Experiments on real data sets verify the effectiveness of the proposed method.
- Privacy protection /
- Swarm learning (SL) /
- federated learning (FL) /
- canonical correlation analysis (CCA)

HTML全文

图 1 SCCA协作序列

Fig. 1 Collaboration sequences of the SCCA

下载: 全尺寸图片幻灯片

图 2 来自IMDB-WIKI的图像示例

Fig. 2 The sample images selected from IMDB-WIKI

下载: 全尺寸图片幻灯片

图 3 主向量对分类精度的影响

Fig. 3 Influence of the principal vectors on classification accuracy

下载: 全尺寸图片幻灯片

图 4 主向量对数据量的影响

Fig. 4 Influence of the principal vectors on data size

下载: 全尺寸图片幻灯片

图 5 性别识别精度

Fig. 5 Recognition accuracy for gender

下载: 全尺寸图片幻灯片

图 6 训练时间比较

Fig. 6 Comparison of training time

下载: 全尺寸图片幻灯片

图 7 降维方法比较

Fig. 7 Comparison of dimension reduction methods

下载: 全尺寸图片幻灯片

图 8 分类实例

Fig. 8 An instance of classification

下载: 全尺寸图片幻灯片

参考文献(29)

[1]	Kels C G. HIPAA in the era of data sharing. Journal of the American Medical Association, 2020, 323(5): 476-477 doi: 10.1001/jama.2019.19645
[2]	Raymond N. Reboot ethical review for the age of big data. Nature, 2019, 568(7752): 277-277 doi: 10.1038/d41586-019-01164-z
[3]	Taddeo M, Floridi L. How AI can be a force for good. Science, 2018, 361(6404): 751-752 doi: 10.1126/science.aat5991
[4]	Jobin A, Ienca M, Vayena E. The global landscape of AI ethics guidelines. Nature Machine Intelligence, 2019, 1(9): 389-399 doi: 10.1038/s42256-019-0088-2
[5]	Stahl B C, Wright D. Ethics and privacy in AI and big data: Implementing responsible research and innovation. IEEE Security & Privacy, 2018, 16(3): 26-33
[6]	Stadler T, Troncoso C. Why the search for a privacy-preserving data sharing mechanism is failing. Nature Computational Science, 2022, 2(4): 208-210 doi: 10.1038/s43588-022-00236-x
[7]	Wu C H,Wu F Z, Lyu L J, Huang Y F, Xie X. Communication-efficient federated learning via knowledge distillation. Nature Communications, 2022, 13(1): Article No. 2032(2022)
[8]	Nguyen D C, Ding M, Pathirana P N, Seneviratne A, Li J, Poor H V. Federated learning for internet of things: A comprehensive survey. IEEE Communications Surveys & Tutorials, 2021, 23(3): 1622-1658
[9]	梁锋, 羊恩跃, 潘微科, 杨强, 明仲. 基于联邦学习的推荐系统综述. 中国科学: 信息科学, 2022, 52(5): 713-741 doi: 10.1360/SSI-2021-0329 Liang Feng, Yang En-Yue, Pan Wei-Ke, Yang Qiang, Ming Zhong. Survey of recommender systems based on federated learning. Scientia Sinica Informationis, 2022, 52(5): 713-741 doi: 10.1360/SSI-2021-0329
[10]	郭艳卿, 王鑫磊, 付海燕, 刘航, 姚明. 面向隐私安全的联邦决策树算法. 计算机学报, 2021, 44(10): 2090-2103 doi: 10.11897/SP.J.1016.2021.02090 Guo Yan-Qing, Wang Xin-Lei, Fu Hai-Yan, Liu Hang, Yao Ming. Federated decision tree algorithm for privacy security. Chinese Journal of Computers, 2021, 44(10): 2090-2103 doi: 10.11897/SP.J.1016.2021.02090
[11]	张泽辉, 富瑶, 高铁杠. 支持数据隐私保护的联邦深度神经网络模型研究. 自动化学报, 2022, 48(5): 1273-1284 Zhang Ze-Hui, Fu Yao, Gao Tie-Gang. Research on federated deep neural network model for data privacy preserving. Acta Automatica Sinica, 2022, 48(5): 1273-1284
[12]	高胜, 袁丽萍, 朱建明, 马鑫迪, 章睿, 马建峰. 一种基于区块链的隐私保护异步联邦学习. 中国科学: 信息科学, 2021, 51(10): 1755-1774 doi: 10.1360/SSI-2021-0087 Gao Sheng, Yuan Li-Ping, Zhu Jian-Ming, Ma Xin-Di, Zhang Rui, Ma Jian-Feng. A blockchain-based privacy-preserving asynchronous federated learning. Scientia Sinica Informationis, 2021, 51(10): 1755-1774 doi: 10.1360/SSI-2021-0087
[13]	朱建明, 张沁楠, 高胜, 丁庆洋, 袁丽萍. 基于区块链的隐私保护可信联邦学习模型. 计算机学报, 2021, 44(12): 2464-2484 doi: 10.11897/SP.J.1016.2021.02464 Zhu Jian-Ming, Zhang Qin-Nan, Gao Sheng, Ding Qing-Yang, Yuan Li-Ping. Privacy preserving and trustworthy federated learning model based on blokchain. Chinese Journal of Computers, 2021, 44(12): 2464-2484 doi: 10.11897/SP.J.1016.2021.02464
[14]	冯霁, 蔡其志, 姜远. 联邦学习下对抗训练样本表示的研究. 中国科学: 信息科学, 2021, 51(6): 900-911 doi: 10.1360/SSI-2019-0145 Feng Ji, Cai Qi-Zhi, Jiang Yuan. Towards training time attacks for federated machine learning systems. Scientia Sinica Informationis, 2021, 51(6): 900-911 doi: 10.1360/SSI-2019-0145
[15]	张泽辉, 李庆丹, 富瑶, 何宁昕, 高铁杠. 面向非独立同分布数据的自适应联邦深度学习算法. 自动化学报, doi: 10.16383/j.aas.c201018 Zhang Ze-Hui, Li Qing-Dan, Fu Yao, He Ning-Xin, Gao Tie-Gang. Adaptive federated deep learning with Non-IID data. Acta Automatica Sinica, doi: 10.16383/j.aas.c201018
[16]	朱静, 王飞跃, 王戈, 田永林, 袁勇, 王晓, 等. 联邦控制: 面向信息安全和权益保护的分布式控制方法. 自动化学报, 2021, 47(8): 1912-1920 Zhu Jing, Wang Fei-Yue, Wang Ge, Tian Yong-Lin, Yuan Yong, Wang Xiao, et al. Federated control: A distributed control approach towards information security and rights protection. Acta Automatica Sinica, 2021, 47(8): 1912-1920
[17]	方晨, 郭渊博, 王一丰, 胡永进, 马佳利, 张晗, 等. 基于区块链和联邦学习的边缘计算隐私保护方法. 通信学报, 2021, 42(11): 28-40 Fang Chen, Guo Yuan-Bo, Wang Yi-Feng, Hu Yong-Jin, Ma Jia-Li, Zhang Han, et al. Edge computing privacy protection method based on blockchain and federated learning. Journal on Communications, 2021, 42(11): 28-40
[18]	张沁楠, 朱建明, 高胜, 熊泽辉, 丁庆洋, 朴桂荣. 基于区块链和贝叶斯博弈的联邦学习激励机制. 中国科学: 信息科学, 2022, 52(6): 971-991 doi: 10.1360/SSI-2022-0020 Zhang Qin-Nan, Zhu Jian-Ming, Gao Sheng, Xiong Ze-Hui, Ding Qing-Yang, Piao Gui-Rong. Incentive mechanism for federated learning based on block-chain and Bayesian game. Scientia Sinica Informationis, 2022, 52(6): 971-991 doi: 10.1360/SSI-2022-0020
[19]	Warnat-Herresthal S, Schultze H, Shastry K L, Manamohan S, Mukherjee S, Garg V, et al. Swarm learning for decentralized and confidential clinical machine learning. Nature, 2021, 594(7862): 265-270 doi: 10.1038/s41586-021-03583-3
[20]	Sun C, Wu S T, Cui T. User selection for federated learning in a wireless environment: A process to minimize the negative effect of training data correlation and improve performance. IEEE Vehicular Technology Magazine, 2022, 17(3): 26-33 doi: 10.1109/MVT.2022.3153274
[21]	Uddin M P, Xiang Y, LU X Q, Yearwood J, Gao L X. Mutual information driven federated learning. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(7): 1526-1538
[22]	Gao Y, Zhang G M, Zhang C C, Wang J K, Yang L T, Zhao Y L. Federated tensor decomposition-based feature extraction approach for industrial IoT. IEEE Transactions on Industrial Informatics, 2021, 17(12): 8541-8549 doi: 10.1109/TII.2021.3074152
[23]	Hotelling H. The most predictable criterion. Journal of Educational Psychology, 1935, 26(2): 139-142 doi: 10.1037/h0058165
[24]	Yang X H, Liu W F, Liu W, Tao D C. A survey on canonical correlation analysis. IEEE Transactions on Knowledge and Data Engineering, 2021, 33(6): 2349-2368 doi: 10.1109/TKDE.2019.2958342
[25]	Ewerbring L M, Luk F T. Canonical correlations and generalized SVD: Applications and new algorithms. Journal of Computational and Applied Mathematics, 1989, 27(1-2): 37-52 doi: 10.1016/0377-0427(89)90360-9
[26]	Uurtio V, Monteiro J M, Kandola J, Shawe-Taylor J, Fernandez-Reyes D, Rousu J. A tutorial on canonical correlation methods. ACM Computing Surveys, 2018, 50(6): Article No. 95(2018)
[27]	Drmač Z. Accurate computation of the product-induced singular value decomposition with application. Siam Journal on Numerical Analysis, 1998, 35(5): 1969-1994 doi: 10.1137/S0036142995292633
[28]	Reyes-Ortiz J L, Oneto L, Sama A, Parra X, Anguita D. Transition-aware human activity recognition using smartphones. Neurocomputing, 2016, 171: 754-767 doi: 10.1016/j.neucom.2015.07.085
[29]	Rothe R, Timofte R, Gool L V. Deep expectation of real and apparent age from a single image without facial landmarks. International Journal of Computer Vision, 2018, 126(2): 144-157