基于隐私保护的联邦推荐算法综述

张洪磊; 李浥东; 邬俊; 陈乃月; 董海荣

doi:10.16383/j.aas.c211189

基于隐私保护的联邦推荐算法综述

doi: 10.16383/j.aas.c211189

1.
北京交通大学计算机与信息技术学院北京 100044
2.
北京交通大学轨道交通控制与安全国家重点实验室北京 100044

基金项目: 国家自然科学基金(U1934220)资助

详细信息

作者简介:
张洪磊：北京交通大学计算机与信息技术学院博士研究生. 主要研究方向为推荐系统与隐私保护. E-mail: honglei.zhang@bjtu.edu.cn

李浥东：北京交通大学计算机与信息技术学院教授. 主要研究方向为大数据分析与安全, 数据隐私保护与先进计算. E-mail: ydli@bjtu.edu.cn

邬俊：北京交通大学计算机与信息技术学院副教授. 主要研究方向为信息检索与推荐系统. E-mail: wuj@bjtu.edu.cn

陈乃月：北京交通大学计算机与信息技术学院讲师. 主要研究方向为社交网络, 数据挖掘与联邦学习. 本文通信作者. E-mail: nychen@bjtu.edu.cn

董海荣：北京交通大学轨道交通控制与安全国家重点实验室教授. 主要研究方向为列车运行智能控制与优化和调度控制一体化. E-mail: hrdong@bjtu.edu.cn

计量
- 文章访问数: 3117
- HTML全文浏览量: 1045
- PDF下载量: 927
- 被引次数: 0
出版历程
- 收稿日期: 2021-12-13
- 录用日期: 2022-06-23
- 网络出版日期: 2022-07-21
- 刊出日期: 2022-09-16

A Survey on Privacy-preserving Federated Recommender Systems

1.
School of Computer and Information Technology, Beijing Jiaotong University, Beijing 100044
2.
State Key Laboratory of Rail Traffic Control and Safety, Beijing Jiaotong University, Beijing 100044

Funds: Supported by National Natural Science Foundation of China (U1934220)

More Information

Author Bio:
ZHANG Hong-Lei　Ph.D. candidate at the School of Computer and Information Technology, Beijing Jiaotong University. His research interest covers recommender systems and privacy protection

LI Yi-Dong　Professor at the School of Computer and Information Technology, Beijing Jiaotong University. His research interest covers big data analysis and security, data privacy protection, and advanced computing

WU Jun　Associate professor at the School of Computer and Information Technology, Beijing Jiaotong University. His research interest covers information retrieval and recommender systems

CHEN Nai-Yue　Lecturer at the School of Computer and Information Technology, Beijing Jiaotong University. Her research interest covers social networks, data mining, and federated learning. Corresponding author of this paper

DONG Hai-Rong　Professor at the State Key Laboratory of Rail Tra-ffic Control and Safety, Beijing Jiaotong University. Her research interest covers intelligent control and optimization of train operation, and integration of scheduling and control

摘要

摘要: 推荐系统通过集中式的存储与训练用户对物品的海量行为信息以及内容特征, 旨在为用户提供个性化的信息服务与决策支持. 然而, 海量数据背后存在大量的用户个人信息以及敏感数据, 因此如何在保证用户隐私与数据安全的前提下分析用户行为模式成为了近年来研究的热点. 联邦学习作为新兴的隐私保护范式, 能够协调多个参与方通过模型参数或者梯度等信息共同学习无损的全局共享模型, 同时保证所有的原始数据保存在用户的终端设备, 较之于传统的集中式存储与训练模式, 实现了从根源上保护用户隐私的目的, 因此得到了众多推荐系统领域研究学者们的广泛关注. 基于此, 对近年来基于联邦学习范式的隐私保护推荐算法进行全面综述、系统分类与深度分析. 具体的, 首先综述经典的推荐算法以及所面临的问题, 然后介绍基于隐私保护的推荐系统与目前存在的挑战, 随后从多个维度综述结合联邦学习技术的推荐算法, 最后对该方向做出系统性的总结并对未来研究方向与发展趋势进行展望.
- 推荐系统 /
- 联邦学习 /
- 隐私保护 /
- 协同过滤
Abstract: Recommender systems aim to provide users with personalized information services and decision support, by mining the centrally stored historical behaviors of users on items and their inherent attributes. However, there is numerous users＇sensitive information behind the massive data, therefore, how to mine users＇behavior patterns on the premise of ensuring users＇privacy and data security has become a hotspot. As an emerging privacy-preserving paradigm, federated learning can coordinate multiple participants to jointly train a lossless global shared model by transmitting model parameters or gradients, and ensure that all original data are saved in the local terminal devices, compared with the traditional centralized storage and training mode. Therefore, it has been widely concerned by many researchers in the field of recommender systems. Based on this, the paper will provide a comprehensive survey, systematic taxonomy, and in-depth analysis of federated recommender systems. Specifically, we first summarize the classical recommendation algorithms and potential problems; then introduce the taxonomy and current challenges on privacy-preserving recommendation methods, and then summarize the privacy-preserving federated recommendation models from multiple dimensions. Finally, we conclude this paper and discuss possible research directions in this area.
- Recommender systems /
- federated learning /
- privacy protection /
- collaborative filtering

HTML全文

图 1 主流推荐模型发展历程

Fig. 1 Timeline of mainstream recommendation models

下载: 全尺寸图片幻灯片

图 2 联邦推荐系统训练流程图

Fig. 2 The procedure of federated recommender systems

下载: 全尺寸图片幻灯片

图 3 联邦推荐系统研究方向总结

Fig. 3 Summary of research directions for federated recommender systems

下载: 全尺寸图片幻灯片

图 4 联邦推荐算法基本框架

Fig. 4 The general framework for federated recommender systems

下载: 全尺寸图片幻灯片

图 5 FedFast 算法示意图

Fig. 5 The diagram of FedFast model

下载: 全尺寸图片幻灯片

图 6 HPFL算法示意图

Fig. 6 The diagram of HPFL model

下载: 全尺寸图片幻灯片

图 7 FedRec++算法示意图

Fig. 7 The diagram of FedRec++ model

下载: 全尺寸图片幻灯片

图 8 FedCT 算法示意图

Fig. 8 The diagram of FedCT model

下载: 全尺寸图片幻灯片

图 9 LDP-FedRec 算法示意图

Fig. 9 The diagram of LDP-FedRec model

下载: 全尺寸图片幻灯片

表 1 基于隐私保护的推荐算法总结

Table 1 Summary of privacy-preserving recommendation algorithms

隐私保护方法	保护阶段	具体实现机制	优点	缺点	代表文献
匿名化方法	数据收集	泛化、抑制、聚类等	实现简单, 可快速获取发布数据	容易受到去匿名化与差分攻击等威胁	[14, 78–80]
数据扰动方法	模型训练模型测试	随机扰动、差分隐私	可以从理论角度保证数据隐私与安全	假设太强, 往往导致数据的可用性降低	[81–87]
密码学方法	模型训练模型测试	多方安全计算、同态加密等	可提供安全可靠的加密数据	计算复杂度高, 需要加密与解密过程	[20–22, 88–90]
对抗学习方法	模型训练模型测试	常规对抗训练、虚拟对抗训练等	可根据攻击目标灵活设计损失函数进行端到端训练	模型训练过程难以收敛, 容易出现模式崩塌等问题	[15, 37, 91–94]
联邦学习方法	数据收集模型训练模型测试	横向、纵向、迁移联邦学习等	保护隐私的同时实现分布式训练以解决数据孤岛问题	需要克服数据异质性以及通信效率等问题	[23–25, 95–97]

下载: 导出CSV

表 2 联邦推荐算法主要研究方向以及实现机制总结

Table 2 Summary of main research directions and realization mechanisms of federated recommender systems

研究方向	潜在挑战	研究目的	适用场景	具体实现机制	代表文献
基础联邦推荐算法　框架	如何针对基础推荐模型以及推荐场景设计合理的联邦学习框架	根据具体场景设计合理的联邦推荐算法	数据来源单一且数据噪声小	基于显式/隐式数据的联邦推荐框架、基于图/序列数据的联邦推荐框架	[5, 25–32]
基于效率增强的联　邦推荐算法	如何保证联邦推荐算法模型的快速收敛	通过压缩与聚类等技术实现较低通信成本	大规模推荐系统	利用强化学习减少参数通信成本、利用聚类实现模型的快速收敛等	[107–110]
缓解异质性的个性　联邦推荐算法	如何有效建模多种复杂异质性的关系	利用个性化联邦缓解客户端分布偏斜问题	数据来源多样且复杂	层次化建模、元学习方法以及迁移学习等个性化联邦技术	[111–114]
基于性能增强的联　邦推荐算法	如何有效防止分布式训练过程中的信息丢失	利用去噪等机制弥补与集中式模型的差距	数据噪声多且对推荐精度要求高	负样本修正、模型正则化、梯度去噪以及迁移学习等技术	[31, 115–118]
基于隐私增强的联　邦推荐算法	如何在保证有效性同时提高隐私保护能力	利用辅助技术实现隐私保护的有效提升	对隐私要求严格的场景, 比如金融、医疗等行业	差分隐私、本地差分隐私、同态加密以及秘钥共享等技术	[119–124]
防御攻击的鲁棒联　邦推荐算法	如何实现有效攻击并提出对应的防御机制	通过分析攻击的可行性来提高其鲁棒性	用户设备不稳定且容易被攻击	中毒攻击、托攻击以及拜占庭攻击等	[125–128]

下载: 导出CSV

表 3 联邦推荐算法常用工具库总结

Table 3 Summary of commonly used repositories in federated recommender systems

工具库名称	受众定位	适用场景	隐私保护机制	代码库链接	发布单位
FATE	工业产品学术研究	横向联邦学习纵向联邦学习联邦迁移学习	多方安全计算同态加密差分隐私	https://github.com/FederatedAI/FATE	微众银行
TFF	学术研究	横向联邦学习	差分隐私	https://github.com/tensorflow/federated	谷歌
PaddleFL	学术研究	横向联邦学习纵向联邦学习	多方安全计算差分隐私	https://github.com/PaddlePaddle/PaddleFL	百度
PySyft	学术研究	横向联邦学习	多方安全计算同态加密差分隐私	https://github.com/OpenMined/PySyft	OpenMind
OpenFL	学术研究	横向联邦学习	可信执行环境	https://github.com/intel/openfl	英特尔
FedML	学术研究	横向联邦学习纵向联邦学习	差分隐私密码学算法	https://github.com/FedML-AI/FedML	南加州大学
FederatedRec	学术研究	横向联邦学习纵向联邦学习	同态加密差分隐私	https://fate.fedai.org/federatedml/	微众银行
EFLS	工业产品学术研究	纵向联邦学习	差分隐私前向加密	https://github.com/alibaba/Elastic-Federated-Learning-Solution	阿里巴巴
FedBPR	学术研究	横向联邦学习	单一梯度上传	https://github.com/sisinflab/FedBPR	SisinfLab
FedGNN	学术研究	横向联邦学习	差分隐私伪标签生成	https://github.com/wuch15/FedGNN	微软亚洲研究院
FedAttack	学术研究	横向联邦学习	标签翻转	https://github.com/rdz98/FedRecAttack	清华大学

下载: 导出CSV

表 4 联邦推荐算法常用数据集总结

Table 4 Summary of commonly used datasets in federated recommender systems

数据集名称	场景	主要描述信息	敏感隐私信息	评分范围	引用次数
MovieLens-1M	电影	该数据集包含 6040 个用户对 3952 部电影共 1000209 个评分记录	用户行为记录以及用户属性信息	1 ~ 5	15
MovieLens-100k	电影	该数据集包括 943 个用户对 1682 部电影共 100000 个评分记录	用户行为记录以及用户属性信息	1 ~ 5	6
Amazon	综合	该数据集包含多种领域, 如音乐、电影、书籍、体育等多个子数据集	用户行为记录	0 ~ 5	3
Last.FM	音乐	该数据集包括 1892个用户, 17632 首歌曲, 以及 92834 个评分记录	用户收听记录	0 ~ 1	2
MIND-small	新闻	该数据集包含 50001 个用户对 25659 条新闻共 8584442 个评分记录	用户阅读记录	0 ~ 1	2
Douban	电影	该数据集包括 129490 个用户对 58541 部电影共 16830839 个评分记录	用户行为记录以及社交信息	0 ~ 5	2
Ciao	电影	该数据集包括 7375 个用户对 105114 部电影共 284086 个评分记录	用户行为记录以及社交信息	1 ~ 5	2
Filmtrust	电影	数据集包括 1508 个用户对 2071 部电影共 35497 个评分记录	用户行为记录以及社交信息	0.5 ~ 4	1
Book-Crossings	书籍	该数据集包括 105284 个用户, 340557 本书, 以及 1149780 个评分记录	用户行为记录与用户属性信息	1 ~ 10	1
Epinions	购物	该数据集包括 116260 个用户对 41269 个物品共 188478 个评分记录	用户行为记录以及社交信息	1 ~ 5	1
Yelp	购物	该数据集包括 7975 个用户, 9323 个物品以及 79087 个交互记录	用户购买记录与标签信息	1 ~ 5	1
MovieLens-10M	电影	该数据集包括 138493 个用户对 27278 部电影共 20000263 个评分记录	用户行为记录与用户属性信息	0.5 ~ 5	1

下载: 导出CSV

参考文献(132)

[1]	Jacoby J. Perspectives on information overload. Journal of Consumer Research, 1984, 10(4): 432-435 doi: 10.1086/208981
[2]	Goldberg D, Nichols D, Oki B M, Terry D. Using collaborative filtering to weave an information tapestry. Communications of the ACM, 1992, 35(12): 61-70 doi: 10.1145/138859.138867
[3]	饶子昀, 张毅, 刘俊涛, 曹万华. 应用知识图谱的推荐方法与系统. 自动化学报, 2021, 46(9): 2061-2077 doi: 10.16383/j.aas.c200128 Rao Zi-Yun, Zhang Yi, Liu Jun-Tao, Cao Wan-Hua. Recommendation methods and systems using knowledge graph. Acta Automatica Sinica, 2021, 47(9): 2061-2077) doi: 10.16383/j.aas.c200128
[4]	Tang J, Hu X, Liu H. Social recommendation: A review. Social Network Analysis and Mining, 2013, 3(4): 1113-1133 doi: 10.1007/s13278-013-0141-9
[5]	Qi T, Wu F, Wu C, Huang Y, Xie X. Privacy-preserving news recommendation model learning. In: Proceedings of the Findings of the Conference on Empirical Methods in Natural Language Processing. Virtual Event: 2020. 1423−1432
[6]	Cheng H T, Koc L, Harmsen J, Shaked T, Chandra T, Aradhye H, et al. Wide & deep learning for recommender systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. Boston, MA, USA: 2016. 7−10
[7]	Pazzani M J, Billsus D. Content-based recommendation systems. In: Proceedings of the Adaptive Web. Berlin, Heidelberg: 2007. 325−341
[8]	Shi Y, Larson M, Hanjalic A. Collaborative filtering beyond the user-item matrix: A survey of the state of the art and future challenges. ACM Computing Surveys, 2014, 47(1): 1-45
[9]	Zhang S, Yao L, Sun A, Tay Y. Deep learning based recommender system: A survey and new perspectives. ACM Computing Surveys, 2019, 52(1): 1-38
[10]	Zhang H, Liu G, Wu J. Social collaborative filtering ensemble. In: Proceedings of the Pacific Rim International Conference on Artificial Intelligence. Nanjing, China: 2018. 1005−1017
[11]	Kim D, Park C, Oh J, Lee S, Yu H. Convolutional matrix factorization for document context-aware recommendation. In: Proceedings of the 10th ACM Conference on Recommender Systems. Boston, MA, USA: 2016. 233−240
[12]	He R, McAuley J. VBPR: Visual Bayesian personalized ranking from implicit feedback. In: Proceedings of the AAAI Conference on Artificial Intelligence. Phoenix, Arizona, USA: 2016. 144−150
[13]	Han D, Li J, Yang L, Zeng Z. A recommender system to address the cold start problem for app usage prediction. International Journal of Machine Learning and Cybernetics, 2019, 10(9): 2257-2268 doi: 10.1007/s13042-018-0864-z
[14]	Narayanan A, Shmatikov V. Robust deanonymization of large sparse datasets. In: Proceedings of the IEEE Symposium on Security and Privacy. Oakland, California, USA: 2008. 111−125
[15]	Beigi G, Mosallanezhad A, Guo R, Alvari H, Nou A, Liu H, et al. Privacy-aware recommendation with private-attribute protection using adversarial learning. In: Proceedings of the 13th International Conference on Web Search and Data Mining. Houston, TX, USA, 2020. 34−42
[16]	Meng X, Wang S, Shu K, Li J, Chen B, Liu H, et al. Personalized privacy-preserving social recommendation. In: Proceedings of the AAAI Conference on Artificial Intelligence. New Orleans, Louisiana, USA: 2018. 3796−3803
[17]	Zhan J, Hsieh C L, Wang I C, Hsu T S, Liau C J, Wang D W. Privacy-preserving collaborative recommender systems. IEEE Transactions on Systems, Man, and Cybernetics. 2010, 40(4): 472-476 doi: 10.1109/TSMCC.2010.2040275
[18]	张锋, 常会友. 基于分布式数据的隐私保持协同过滤推荐研究. 计算机学报, 2006, 8: 1487-1495 doi: 10.3321/j.issn:0254-4164.2006.08.028 Zhang Feng, Chang Hui-You. Research on privacy-preserving collaborative filtering recommendation based on distributed data. Chinese Journal of Computers, 2006, 8: 1487-1495) doi: 10.3321/j.issn:0254-4164.2006.08.028
[19]	Polat H, Du W. Privacy-preserving collaborative filtering using randomized perturbation techniques. In: Proceedings of the Third IEEE International Conference on Data Mining. Melbourne, Florida, USA: IEEE, 2003. 625−628
[20]	Berlioz A, Friedman A, Kaafar M A, Boreli R, Berkovsky S. Applying differential privacy to matrix factorization. In: Proceedings of the 9th ACM Conference on Recommender Syste-ms. Vienna, Austria: 2015. 107−114
[21]	Kim S, Kim J, Koo D, Kim Y, Yoon H, Shin J. Efficient privacy-preserving matrix factorization via fully homomorphic encryption. In: Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security. Xi＇an, China: 2016. 617−628
[22]	Shmueli E, Tassa T. Secure multi-party protocols for item-based collaborative filtering. In: Proceedings of the Eleventh ACM Conference on Recommender Systems. Como, Italy: 2017. 89−97
[23]	Konecny J, McMahan B, Ramage D. Federated optimization: Distributed optimization beyond the datacenter[Online], available: https://arxiv.org/abs/1511.03575, November 11, 2015
[24]	McMahan B, Moore E, Ramage D, Hampson S, Arcas B A. Communication-efficient learning of deep networks from decen-tralized data. In: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. Fort Lauderdale, USA: 2017. 1273−1282
[25]	Ammad M, Ivannikova E, Khan S A, Oyomno W, Fu Q, Tan K E, et al. Federated collaborative filtering for privacy-prese-rving personalized recommendation system[Online], available: https://arxiv.org/abs/1901.09888, January 29, 2019
[26]	Lin G, Liang F, Pan W, Ming Z. Fedrec: federated recommendation with explicit feedback. IEEE Intelligent Systems, 2020, 36(5): 21-30
[27]	Anelli V W, Deldjoo Y, Di Noia T, Ferrara A, Narducci F. User-controlled federated matrix factorization for recommender systems. Journal of Intelligent Information Systems. 2022, 58(2): 287-309 doi: 10.1007/s10844-021-00688-z
[28]	Perifanis V, Efraimidis P S. Federated neural collaborative filtering. Knowledge-based Systems. 2022, 242(2022): 108441
[29]	Wu C, Wu F, Cao Y, Huang Y, Xie X. Fedgnn: Federated gra-ph neural network for privacy-preserving recommendation [Online], available: https://arxiv.org/abs/2102.04925, February 9, 2021
[30]	Han J, Ma Y, Mei Q, Liu X. DeepRec: On-device deep learning for privacy-preserving sequential recommendation in mobile commerce. In: Proceedings of the Web Conference. Virtual Event: 2021. 900−911
[31]	Liu S, Xu S, Yu W, Fu Z, Zhang Y, Marian A. FedCT: Federated collaborative transfer for recommendation. In: Proceeding-s of the 44th International ACM SIGIR Conference on Resear-ch and Development in Information Retrieval. Virtual Event: 2021. 716−725
[32]	Liu Z, Yang L, Fan Z, Peng H, Yu P S. Federated social recommendation with graph neural network. ACM Transaction on Intelligent Systems and Technology 2022,13(4): 1-24
[33]	Smith B, Linden G. Two decades of recommender systems at amazon. com. IEEE internet computing, 2017, 21(3): 12-18 doi: 10.1109/MIC.2017.72
[34]	Gomez-Uribe C A, Hunt N. The netflix recommender system: Algorithms, business value, and innovation. ACM Transactions on Management Information Systems, 2015, 6(4): 1-19
[35]	Resnick P, Iacovou N, Suchak M, Bergstrom P, Riedl J. Grouplens: An open architecture for collaborative filtering of netnews. In: Proceedings of the Conference on Computer Supported Cooperative Work. Chapel Hill, NC, USA: 1994. 175−186
[36]	Sarwar B, Karypis G, Konstan J, Riedl J. Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web. Hong Kong, China: 2001. 285−295
[37]	Deldjoo Y, Di Noia T, Merra F A. Adversarial machine learning in recommender systems: State of the art and challenges. In: Proceedings of the 13th International Conference on Web Search and Data Mining. New York, NY, USA: 2020. 869−872
[38]	Billsus D, Pazzani M J. Learning collaborative information filt-ers. In: Proceedings of the Fifteenth International Conference on Machine Learning. Madison, Wisconsin, USA: 1998. 46−54
[39]	He X, Pan J, Jin O, Xu T, Liu B, Xu T, et al. Practical lessons from predicting clicks on ads at facebook. In: Proceedings of the Eighth International Workshop on Data Mining for Online Advertising. New York, NY, USA: 2014. 1−9
[40]	Koren Y, Bell R, Volinsky C. Matrix factorization techniques for recommender systems. Computer, 2009 42(8): 30-37 doi: 10.1109/MC.2009.263
[41]	Mnih A, Salakhutdinov R R. Probabilistic matrix factorization. In: Proceedings of the Twenty-first Annual Conference on Neur-al Information Processing Systems. Vancouver, British Colum-bia, Canada: 2008. 1257−1264
[42]	Johnson C C. Logistic matrix factorization for implicit feedback data. In: Proceedings of the Annual Conference on Neural Information Processing Systems. Vancouver, British Colum-bia, Canada: 2014. 27(78): 1−9
[43]	Rendle S. Factorization machines. In: Proceedings of the IEEE International Conference on Data Mining. Sydney, Australia: IEEE, 2010. 995−1000
[44]	Koren Y. Factorization meets the neighborhood: A multi-faceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge discovery and data mining. Las Vegas, Nevada, USA: 2008. 426−434
[45]	Hu L, Sun A, Liu Y. Your neighbors affect your ratings: On geographical neighborhood influence to rating prediction. In: Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval. Gold Coast, QLD, Australia: 2014. 345−354
[46]	李慧, 马小平, 施珺, 李存华, 仲兆满, 蔡虹. 复杂网络环境下基于信任传递的推荐模型研究. 自动化学报, 2018, 44(2): 363-376 doi: 10.16383/j.aas.2018.c160395 Li Hui, Ma Xiao-Ping, Shi Jun, Li Cun-Hua, Zhong Zhao-Man, Cai Hong. A recommendation model by means of trust transition in complex network environment. Acta Automatica Sinica, 2018, 44(2): 363-376) doi: 10.16383/j.aas.2018.c160395
[47]	Rendle S, Freudenthaler C, Gantner Z. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the Twenty-fifth Conference on Uncertainty in Artificial Intelligen-ce. Montreal, Canada: 2009. 452−461
[48]	Mao K, Zhu J, Wang J, Dai Q, Dong Z, Xiao X, et al. SimpleX: A simple and strong baseline for collaborative filtering. In: Proceedings of the 30th International Conference on Information and Knowledge Management. Virtual Event: 2021. 1243− 1252
[49]	Hsieh C K, Yang L, Cui Y. Collaborative metric learning. In: Proceedings of the 26th International Conference on World Wide Web. Perth, Australia: 2017. 193−201
[50]	Weimer M, Karatzoglou A, Le Q. Cofirank: Maximum margin matrix factorization for collaborative ranking. In: Proceedings of the 21st Annual Conference on Neural Information Process-ing Systems. Vancouver, British Columbia, Canada: 2007. 222− 230
[51]	Shi Y, Karatzoglou A, Baltrunas L, Larson M, Oliver N, Hanjalic. Climf: Learning to maximize reciprocal rank with collaborative less-is-more filtering. In: Proceedings of the sixth ACM Conference on Recommender Systems. Dublin, Ireland: 2012. 139−146
[52]	Liu W, Xi Y, Qin J, Sun F, Chen B, Zhang W, et al. Neural re-ranking in multi-stage recommender systems: A review[Online], available: https://arxiv.org/abs/2202.06602, February 14, 2022
[53]	Xia F, Liu T Y, Wang J. Listwise approach to learning to rank: Theory and algorithm. In: Proceedings of the 25th International Conference on Machine learning. Helsinki, Finland: 2008. 1192−1199
[54]	李金忠, 刘关俊, 闫春钢, 蒋昌俊. 排序学习研究进展与展望. 自动化学报, 2018, 44(8): 1345-1369 doi: 10.16383/j.aas.2018.c170246 Li Jin-Zhong, Liu Guan-Jun, Yan Chun-Gang, Jiang Chang-Jun. Research advances and prospects of learning to rank. Acta Automatica Sinica, 2018, 44(8): 1345-1369) doi: 10.16383/j.aas.2018.c170246
[55]	Sedhain S, Menon A K, Sanner S. Autorec: Autoencoders meet collaborative filtering. In: Proceedings of the 24th International Conference on World Wide Web. Florence, Italy: 2015. 111− 112
[56]	Wu Y, DuBois C, Zheng A X, Ester M. Collaborative denoising auto-encoders for top-n recommender systems. In: Proceedings of the ninth ACM International Conference on Web Sear-ch and Data Mining. San Francisco, USA: 2016. 153−162
[57]	Liang D, Krishnan R G, Hoffman M D. Variational autoencoders for collaborative filtering. In: Proceedings of the 2018 World Wide Web Conference. Lyon, France: 2018. 689−698
[58]	Hornik K, Stinchcombe M, White H. Multilayer feedforward networks are universal approximators. Neural networks, 1989, 2(5): 359-366 doi: 10.1016/0893-6080(89)90020-8
[59]	He X, Liao L, Zhang H, Nie L, Hu X, Chua T S. Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web. Paris, France: 2017. 173−182
[60]	Guo H, Tang R, Ye Y. DeepFM: A factorization-machine based neural network for CTR prediction. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Melbourne, Australia: 2017. 1725−1731
[61]	Barkan O, Koenigstein N. Item2vec: Neural item embedding for collaborative filtering. In: Proceedings of the 26th International Workshop on Machine Learning for Signal Processing. Vietri sul Mare, Salerno, Italy: 2016. 1−6
[62]	Deng Z H, Huang L, Wang C D. Deepcf: A unified framework of representation learning and matching function learning in recommender system. In: Proceedings of the AAAI Conference on Artificial Intelligence. Honolulu, Hawaii, USA: 2019. 61−68
[63]	Zheng L, Noroozi V, Yu P S. Joint deep modeling of users and items using reviews for recommendation. In: Proceedings of the tenth ACM International Conference on Web Search and Data Mining. Cambridge, UK: 2017. 425−434
[64]	Tang J, Wang K. Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining. Marina Del Rey, CA, USA: 2018. 565−573
[65]	冯永, 陈以刚, 强保华. 融合社交因素和评论文本卷积网络模型的汽车推荐研究. 自动化学报, 2019, 45(3): 518-529 Feng Yong, Chen Yi-Gang, Qiang Bao-Hua. Social and comment text CNN model based automobile recommendation. Acta Automatica Sinica, 2019, 45(3): 518-529)
[66]	Huang J, Zhao W X, Dou H. Improving sequential recommendation with knowledge-enhanced memory networks. In: Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Ann Arbor, USA: 2018. 505−514
[67]	de Souza Pereira Moreira G, Rabhi S, Lee J M. Trans-formers4Rec: Bridging the gap between NLP and sequential /session-based recommendation. In: Proceedings of the Fifteen-th ACM Conference on Recommender Systems. Amsterdam, Netherlands: 2021. 143−153
[68]	Tim Donkers, Benedikt Loepp, Jurgen Ziegler. Sequential user-based recurrent neural network recommendations. In: Proceedings of the Eleventh ACM Conference on Recommender Syste-ms. Como, Italy: 2017. 152−160
[69]	Fan W, Ma Y, Li Q, He Y, Zhao E, Tang J, et al. Graph neural networks for social recommendation. In: Proceedings of the World Wide Web Conference. San Francisco, CA, USA: 2019. 417−426
[70]	Hu J, Qian S, Fang Q. Efficient graph deep learning in tensorflow with TF_geometric. In: Proceedings of the ACM International Conference on Multimedia. Virtual Event: 2021. 3775− 3778
[71]	Wang X, He X, Wang M. Neural graph collaborative filtering. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and development in Information Retrieval. Paris, France: 2019. 165−174
[72]	Wu L, Sun P, Hong R. Socialgcn: An efficient graph convolutional network based model for social recommendation [Online], available: https://arxiv.org/abs/1811.02815, November 7, 2018
[73]	Zhang Q, Wang J, Huang H, Huang X, Gong Y. Hashtag recommendation for multimodal microblog using co-attention network. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence. Melbourne, Austra-lia: 2017. 3420−3426
[74]	Canny J. Collaborative filtering with privacy via factor analysis. In: Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. Tampere, Finland: 2002. 238−245
[75]	Bilge A, Kaleli C, Yakut I, Gunes I, Polat H. A survey of privacy-preserving collaborative filtering schemes. International Journal of Software Engineering and Knowledge Engineering, 2013, 23(08): 1085-1108. doi: 10.1142/S0218194013500320
[76]	Hu G, Yang Q. PrivNet: Safeguarding private attributes in transfer learning for recommendation. In: Proceedings of the Findings of the Conference on Empirical Methods in Natural Language Processing. Virtual Event: 2020. 4506−4516
[77]	周俊, 董晓蕾, 曹珍富. 推荐系统的隐私保护研究进展. 计算机研究与发展, 2019, 56: 2033-2048 doi: 10.7544/issn1000-1239.2019.20190541 Zhou Jun, Dong Xiao-Lei, Cao Zhen-Fu. Research advances on privacy preserving in recommender systems. Journal of Computer Research and Development, 2019, 56: 2033-2048) doi: 10.7544/issn1000-1239.2019.20190541
[78]	Chang C C, Thompson B, Wang H. Towards publishing recommendation data with predictive anonymization. In: Proceedings of the 5th ACM Symposium on Information, Computer and Communications Security. Beijing, China: 2010. 24−35
[79]	Sakuma J, Osame T. Recommendation with k-anonymized ratings. Transactions on Data Privacy, 2018, 11(1): 47-60
[80]	Ruoxuan Wei, Hui Tian, Hong Shen. Improving k-anonymity based privacy preservation for collaborative filtering. Computers & Electrical Engineering, 2018, 67: 509-519
[81]	Weinsberg U, Bhagat S, Ioannidis S. BlurMe: Inferring and obfuscating user gender based on ratings. In: Proceedings of the sixth ACM Conference on Recommender Systems. Dublin, Ireland: 2012. 195−202
[82]	Parra-Arnau J, Rebollo-Monedero D, Forné J. Optimal forgery and suppression of ratings for privacy enhancement in recommendation systems. Entropy, 2014, 16(3): 1586-1631 doi: 10.3390/e16031586
[83]	McSherry F, Mironov I. Differentially private recommender systems: Building privacy into the netflix prize contenders. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Paris, France: 2009. 627−636
[84]	Jorgensen Z, Yu T. A privacy-preserving framework for personalized, social recommendations. In: Proceedings of the 17th International Conference on Extending Database Technology. Athens, Greece: 2014. 582
[85]	Jingyu H, Chang X, Sheng Z. Differentially private matrix factorization. In: Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. Buenos Aires, Argen-tina: 2015. 1763−1770
[86]	Zhu X, Sun Y. Differential privacy for collaborative filtering recommender algorithm. In: Proceedings of the 2016 ACM on International Workshop on Security And Privacy Analytics. New Orleans, LA, USA: 2016. 9−16
[87]	Shin H, Kim S, Shin J. Privacy enhanced matrix factorization for recommendation with local differential privacy. IEEE Transactions on Knowledge and Data Engineering, 2018, 30(9): 1770-1782 doi: 10.1109/TKDE.2018.2805356
[88]	Chaochao C, Liang L, Bingzhe W. Secure social recommendation based on secret sharing. In: Proceedings of the 24th European Conference on Artificial Intelligence. Santiago de Compostela, Spain: 2020. 506−512
[89]	Shmueli E, Tassa T. Mediated secure multi-party protocols for collaborative filtering. ACM Transactions on Intelligent Systems and Technology, 2020, 11(2): 1-25
[90]	Jinsu Kim, Dongyoung Koo, Yuna Kim, Hyunsoo Yoon, Junbum Shin, Sungwook Kim. Efficient Privacy-Preserving Matrix Factorization for Recommendation via Fully Homomorphic Encryption. ACM Transactions on Privacy and Security, 2018, 21(4): 1-17
[91]	Goodfellow I J, Shlens J, Szegedy C. Explaining and harnessing adversarial examples. In: Proceedings of the 3rd International Conference on Learning Representations. San Diego, CA, USA, 2015.
[92]	He X, He Z, Du X, Chua T S. Adversarial personalized ranking for recommendation. In: Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Ann Arbor, MI, USA: 2018. 355−364
[93]	Li K, Luo G, Ye Y. Adversarial privacy-preserving graph embedding against inference attack. IEEE Internet of Things Journal, 2020, 8(8): 6904-6915
[94]	Xiao T, Tsai Y H, Sohn K. Adversarial learning of privacy-preserving and task-oriented representations. In: Proceedings of the AAAI Conference on Artificial Intelligence. New York, USA: 2020. 12434−12441
[95]	Kairouz P, McMahan H B, Avent B. Advances and open problems in federated learning. Foundations and Trends in Mach-ine Learning. 14(1−2): 1−210
[96]	Li Q, Wen Z, Wu Z, Hu S, Wang N, Li Y, et al. A survey on federated learning systems: Vision, hype and reality for data privacy and protection[Online], available: https://arxiv.org/abs/1907.09693, July 23, 2019
[97]	Yang Q, Liu Y, Chen T. Federated machine learning: Concept and applications. ACM Transactions on Intelligent Systems and Technology, 2019, 10(2): 1-19
[98]	Mireshghallah F, Taram M, Vepakomma P, Singh A, Raskar R, Esmaeilzadeh H. Privacy in deep learning: A survey [Online], available: https://arxiv.org/abs/2004.12254, April 25, 2020
[99]	Dwork C, Roth A, Others. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 2014, 9(3-4): 211-407
[100]	Rebollo-Monedero D, Parra-Arnau J, Forné J. An information-theoretic privacy criterion for query forgery in information retrieval. In: Proceedings of the International Conference on Security Technology. Jeju Island, Korea: 2011. 146−154
[101]	Mo F, Shamsabadi A S, Katevas K. Darknetz: Towards model privacy at the edge using trusted execution environments. In: Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. Toronto, Canada: 2020. 161−174
[102]	Gupta O, Raskar R. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications, 2018, 116: 1-8 doi: 10.1016/j.jnca.2018.05.003
[103]	Chong C, Fei S, Min Z, Bolin D. Recommendation unlearning. In: Proceedings of the ACM Web Conference. Virtual Event: 2022. 2768−2777
[104]	Zhang M, Sapra K, Fidler S, Yeung S, Alvarez J M. Personalized federated learning with first order model optimization. In: Proceedings of the 9th International Conference on Learning Representations. Virtual Event. 2021.
[105]	Liu B, Guo Y, Chen X. PFA: Privacy-preserving federated adaptation for effective model personalization. In: Proceedings of the Web Conference. Ljubljana, Slovenia: 2021. 923−934
[106]	Collins L, Hassani H, Mokhtari A, Shakkottai S. Exploiting shared representations for personalized federated learning. In: Proceedings of the 38th International Conference on Machine Learning. Virtual Event: 2021. 2089−2099
[107]	Muhammad K, Wang Q, Reilly-Morgan D, Tragos E, Smyth B, Hurley N, et al. Fedfast: Going beyond average for faster training of federated recommender systems. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Virtual Event: 2020. 1234−1242
[108]	Khan F K, Flanagan A, Tan K E. A payload optimization method for federated recommender systems. In: Proceedings of the Fifteenth ACM Conference on Recommender Systems. Ams-terdam, Netherlands: 2021. 432−442
[109]	Yi J, Wu F, Wu C, Liu R, Sun G, Xie X. Efficient-FedRec: Efficient federated learning framework for privacy-preserving news recommendation. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Virtual Event: 2021. 2814−2824
[110]	Zhang H, Luo F, Wu J, He X, Li Y. LightFR: Lightweight federated recommendation with privacy-preserving matrix factorization [Online], available: https://arxiv.org/abs/2206.11743, June 23, 2022
[111]	Lin Y, Ren P, Chen Z. Meta matrix factorization for federated rating predictions. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Virtual Event: 2020. 981−990
[112]	Wang Q, Yin H, Chen T, Yu J, Zhou A, Zhang X. Fast-adapting and privacy-preserving federated recommender system. The VLDB Journal. 2021: 1-20
[113]	Wu J, Liu Q, Huang Z, Ning Y, Wang H, Chen E, et al. Hierarchical personalized federated learning for user modeling. In: Proceedings of the Web Conference. Virtual Event: 2021. 957− 968
[114]	谌明, 张蕾, 马天翼. 一种基于注意力联邦蒸馏的推荐方法. 软件学报, 2021, 32(12): 3852-3868 doi: 10.13328/j.cnki.jos.006128 Chen Ming, Zhang Lei, Ma Tian-Yi. Recommendation approach based on attentive federated distillation. Journal of Software, 2021, 32(12): 3852-3868) doi: 10.13328/j.cnki.jos.006128
[115]	Liang F, Pan W, Ming Z. FedRec++: Lossless federated recommendation with explicit feedback. In: Proceedings of the AAAI Conference on Artificial Intelligence. Virtual Event: 2021. 4224−4231
[116]	Ning L, Singhal K, Zhou E X. Learning federated representations and recommendations with limited negatives[Online], available: https://arxiv.org/abs/2108.07931, August 18, 2021
[117]	Flanagan A, Oyomno W, Grigorievskiy A, Tan K E, Khan S A, Ammad-Ud-Din M. Federated multi-view matrix factorization for personalized recommendations. In: Proceedings of European Conference on Machine Learning and Knowledge Discovery in Databases. Ghent, Belgium: 2020. 324−347
[118]	Saikishore Kalloori, Severin Klingler. Horizontal cross-silo federated recommender systems. In: Proceedings of the Fifteenth ACM Conference on Recommender Systems. Amsterdam, Neth-erlands: 2021. 680−684
[119]	Minto L, Haller M, Livshits B. Stronger privacy for federated collaborative filtering with implicit feedback. In: Proceedings of the Fifteenth ACM Conference on Recommender Systems. Amsterdam, Netherlands: 2021. 342−350
[120]	Chai D, Wang L, Chen K, Yang Q. Secure federated matrix factorization. IEEE Intelligent Systems, 2020, 36(5): 11-20
[121]	Yang L, Tan B, Liu B, Zheng V W, Chen K, Yang Q. Practical and secure federated recommendation with personalized masks [Online], available: https://arxiv.org/abs/2109.02464, Au-gust 18, 2021
[122]	Lin Z, Pan W, Ming Z. FR-FMSS: Federated recommendation via fake marks and secret sharing. In: Proceedings of the Fifteenth ACM Conference on Recommender Systems. Amsterd-am, Netherland: 2021. 668−673
[123]	Li T, Song L, Fragouli C. Federated recommendation system via differential privacy. In: Proceedings of the IEEE International Symposium on Information Theory. Los Angeles, USA: IEEE, 2020. 2592−2597
[124]	Müllner P, Kowald D, Lex E. Robustness of meta matrix factorization against strict privacy constraints. In: Proceedings of the 43rd European Conference on Information Retrieval Research. Virtual Event: 2021. 107−119
[125]	Zhang S, Yin H, Chen T. PipAttack: Poisoning federated recommender systems for manipulating item promotion. In: Proc-eedings of the Fifteenth ACM International Conference on Web Search and Data Mining. New York, USA: 2022. 1415−1423
[126]	Jiang Y, Zhou Y, Wu D, Li C, Wang Y. On the detection of shilling attacks in federated collaborative filtering. In: Proceedings of the 2020 International Symposium on Reliable Distributed Systems. Shanghai, China: IEEE, 2020. 185−194
[127]	Chen C, Zhang J, Tung A K, Kankanhalli M, Chen G. Robust federated recommendation system [Online], available: https://arxiv.org/abs/2006.08259, June 15, 2020
[128]	Wu C, Wu F, Qi T, Huang Y, Xie X. FedAttack: Effective and covert poisoning attack on federated recommendation via hard sampling [Online], available: https://arxiv.org/abs/2202.04975, February 10, 2022
[129]	Orekondy T, Oh S J, Zhang Y, Schiele B, Fritz M. Gradient-leaks: Understanding and controlling deanonymization in federated learning [Online], available: https://arxiv.org/abs/1805.05838, May 15, 2018
[130]	Hospedales T, Antoniou A, Micaelli P, Storkey A. Meta-learning in neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2021: 1-1 doi: 10.1109/TPAMI.2021.3079209
[131]	Wang Z, Song M, Zhang Z, Song Y, Wang Q, Qi H. Beyond inferring class representatives: User-level privacy leakage from federated learning. In: Proceedings of the IEEE Conference on Computer Communications. Paris, France: IEEE, 2019. 2512− 2520
[132]	He X, Jia J, Backes M, Gong N Z, Zhang Y. Stealing links from graph neural networks. In: Proceedings of the 30th USEN-IX Security Symposium. Virtual Event: 2021. 2669−2686