2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

融合注意力机制的增强受限玻尔兹曼机驱动的交互式分布估计算法

暴琳 孙晓燕 巩敦卫 张勇

暴琳, 孙晓燕, 巩敦卫, 张勇. 融合注意力机制的增强受限玻尔兹曼机驱动的交互式分布估计算法. 自动化学报, 2021, x(x): 1−13 doi: 10.16383/j.aas.c200604
引用本文: 暴琳, 孙晓燕, 巩敦卫, 张勇. 融合注意力机制的增强受限玻尔兹曼机驱动的交互式分布估计算法. 自动化学报, 2021, x(x): 1−13 doi: 10.16383/j.aas.c200604
Bao Lin, Sun Xiao-Yan, Gong Dun-Wei, Zhang Yong. Enhanced restricted boltzmann machine-driven interactive estimation of distribution algorithms with attention mechanism. Acta Automatica Sinica, 2021, x(x): 1−13 doi: 10.16383/j.aas.c200604
Citation: Bao Lin, Sun Xiao-Yan, Gong Dun-Wei, Zhang Yong. Enhanced restricted boltzmann machine-driven interactive estimation of distribution algorithms with attention mechanism. Acta Automatica Sinica, 2021, x(x): 1−13 doi: 10.16383/j.aas.c200604

融合注意力机制的增强受限玻尔兹曼机驱动的交互式分布估计算法

doi: 10.16383/j.aas.c200604
基金项目: 国家自然科学基金(61876184)资助
详细信息
    作者简介:

    暴琳:江苏科技大学电子信息学院讲师. 2020年获中国矿业大学控制理论与控制工程专业博士学位. 主要研究方向为进化计算与机器学习. E-mail: baolin_zj@163.com

    孙晓燕:中国矿业大学信息与控制工程学院教授. 2009年获控制理论与控制工程专业博士学位. 主要研究方向为进化计算和机器学习. 本文通信作者. E-mail: xysun78@126.com

    巩敦卫:中国矿业大学信息与控制工程学院教授. 1999年获控制理论与控制工程专业博士学位. 主要研究方向为超目标优化计算智能、动态不确定性优化、软件测试、调度、路径规划、大数据处理及分析. E-mail: dwgong@vip.163.com

    张勇:中国矿业大学信息与控制工程学院教授. 2009年获控制理论与控制工程专业博士学位. 主要研究方向为群体智能和机器学习. E-mail: yongzh401@126.com

Enhanced Restricted Boltzmann Machine-driven Interactive Estimation of Distribution Algorithms with Attention Mechanism

Funds: Supported by National Natural Science Foundation of P. R. China (61876184)
More Information
    Author Bio:

    BAO Lin Lecturer at the School of Electronics and Information, Jiangsu University of Science and Technology. She received her Ph. D. degree of control theory and control engineering from China University of Mining and Technology in 2020. Her research interest covers evolutionary computation and machine learning

    SUN Xiao-Yan Professor at the School of Information and Control Engineering, China University of Mining and Technology. She received his Ph. D. degree of control theory and control engineering from China University of Mining and Technology in 2009. Her research interest covers evolutionary computation and machine learning. Corresponding author of this paper

    GONG Dun-Wei Professor at the School of Information and Control Engineering, China University of Mining and Technology. He received his Ph. D. degree in control theory and control engineering from China University of Mining and Technology, Xuzhou, China, in 1999. His current research interests include computation intelligence in many-objective optimization, dynamic and uncertain optimization, as well as applications in software engineering, scheduling, path planning, big data processing and analysis

    ZHANG Yong Professor at the School of Information and Control Engineering, China University of Mining and Technology. He received his Ph. D. degree of control theory and control engineering from China University of Mining and Technology in 2009. His research interest covers swarm intelligence and machine learning

  • 摘要: 面向用户生成内容的进化搜索在大数据及个性化服务领域已引起广泛关注, 其关键在于基于多源异构用户生成内容构建用户认知偏好模型, 进而设计高效的进化搜索机制. 针对此, 本文提出了融合注意力机制的受 限玻尔兹曼机偏好认知代理模型构建机制, 并应用于交互式分布估计算法, 设计含用户生成内容的个性化进化搜索策略. 基于用户群体提供的文本评论, 以及搜索物品的类别文本, 构建无监督受限玻尔兹曼机模型提取广义特征; 设计注意力机制, 融合广义特征, 获取对用户认知偏好高度相关特征的集成; 利用该特征再次训练受限玻尔兹曼机, 实现对用户偏好认知代理模型的构建; 根据用户偏好认知代理模型, 给出交互式分布估计算法概率更新模型以及物品适应度评价函数, 实现物品个性化进化搜索. 算法在亚马逊个性化搜索实例的应用验证了用户认知偏好模型的可靠性, 以及个性化进化搜索的有效性.
  • 图  1  AM-ERBM-IEDA算法框架

    Fig.  1  The Framework of AM-ERBM-IEDA algorithm

    图  2  基于注意力机制和RBM的用户认知偏好模型

    Fig.  2  AM-based RBM user preference model with multi-source heterogeneous data

    图  3  测试用户个性化搜索实验. (a)RMSE (b)HR (c)AP

    Fig.  3  Experimental results of a test user. (a)RMSE (b)HR (c)AP

    图  4  CDs_and_Vinyl数据集测试用户个性化搜索实验. (a)RMSE (b)HR (c)AP

    Fig.  4  Experimental results of a test user on CDs_and_Vinyl. (a)RMSE (b)HR (c)AP

    图  5  Music数据集某用户个性化搜索实验. (a) RMSE (b) HR (c) AP

    Fig.  5  Experimental results of a test user on Music. (a) RMSE (b) HR (c)AP

    图  6  Games数据集某用户个性化搜索实验. (a) RMSE (b) HR (c) AP

    Fig.  6  Experimental results of a test user on Games. (a) RMSE (b) HR (c) AP

    表  1  数据集统计信息

    Table  1  Statistical information of datasets

    数据集# 用户# 项目# 评分
    Digital_Music (Music)478235266414836006
    Video_Games (Games)826767502101324753
    Apps_for_Android (Apps)1323884612752638173
    Kindle_Store (Kindle)14068904305303205467
    CDs_and_Vinyl (CDs)15785974863603749004
    Movies_and_TV (Movies)20886202009414607047
    Yelp19124941803477778794
    下载: 导出CSV

    表  2  算法的实验参数

    Table  2  Experimental parameters of our algorithm

    参数数值
    ${n_1}$类别标签数量
    ${n_2}$200
    $m$(0.8−1.2)倍类别标签数量
    学习率0.1
    动量0.5−0.9
    训练次数20
    $Pop$0.3倍测试数据集规模
    $k$10
    $\alpha $0.3
    $\beta $0.2
    $N$10
    下载: 导出CSV

    表  3  对比实验结果

    Table  3  Experiments compared with popular recommendation algorithms

    算法RandomPopularityBPRMFCnnMFATRankRBMEDADRBMRBM-MsHAtRBM-MsH
    MusicRMSE3.1441.8983.1302.1981.2981.2641.2971.221*
    HR0.07650.07930.07640.07420.07780.07840.09240.09060.0951*
    MAP0.7610.7230.8110.7280.7780.8110.8870.8800.879*
    Time(s)0.0200.1820.494276.7162.9000.2211.5990.6721.766*
    GamesRMSE3.5161.9733.4972.4821.2851.3321.2711.242*
    HR0.08100.09300.07530.09450.08690.08040.08150.08090.0985*
    MAP0.7470.8730.7070.9150.7850.7360.7600.7770.827*
    Time(s)0.0140.1960.402131.7163.0950.1512.3460.7192.785*
    AppsRMSE3.1642.1463.1192.6991.5231.5431.5071.486*
    HR0.07990.07950.08520.07010.08870.07590.07460.07600.0818*
    MAP0.7360.7140.7360.6880.7590.7180.7120.7480.771*
    Time(s)0.0140.1700.34490.4892.5730.1030.6460.3881.476*
    KindleRMSE4.3192.2844.3172.2131.4371.5491.4451.168*
    HR0.02980.02220.02780.02210.03010.02860.02950.02970.0308*
    MAP0.9140.9200.8570.8330.9000.8940.8670.8750.926*
    Time(s)0.0140.7611.205416.5328.74510.06026.2237.22423.478*
    CDsRMSE4.2182.1824.2172.6941.4821.5341.4321.241*
    HR0.01190.01360.01010.01070.01080.01100.01100.01050.0147*
    MAP0.8470.8250.8260.8170.8440.8450.8520.8380.921*
    Time(s)0.0163.8335.406884.51932.3075.34531.38228.11135.836*
    MoviesRMSE3.0681.9603.0292.2711.1911.1851.1671.176*
    HR0.01340.01530.01440.01830.01660.01380.01540.01710.0173*
    MAP0.6680.7690.7020.8380.6820.6720.7660.8000.770*
    Time(s)0.0142.1863.261506.12518.3410.4659.4681.81510.978*
    YelpRMSE3.1321.7093.1942.1950.9981.0250.9890.967*
    HR0.01850.02280.01960.02590.02100.02080.02320.02330.0268*
    MAP0.6710.7750.7350.8710.7830.7350.8120.8860.912*
    Time(s)0.01765.50427.7294824.915159.44623.32526.66911.32620.551
    下载: 导出CSV

    表  4  测试用户个性化搜索实验结果

    Table  4  Experimental results of a test user

    百分比 (%)测试用户
    RMSEHRAP
    100.8740.004550.876
    200.7660.004590.947
    300.7250.007000.977
    400.6920.009441
    500.6800.011681
    600.6790.010200.895
    700.6780.014260.924
    800.6440.019610.721
    900.6200.040680.812
    下载: 导出CSV

    表  5  对比实验结果

    Table  5  Comparative experimental results among compared IECs

    算法IEDARBMIGARBMEDADRBMIEDARIEDA-MsHAtRIEDA-MsH
    MusicRMSE1.1601.2041.4800.9550.955
    HR0.01840.02220.02360.02300.02860.0305*
    AP0.6010.8150.8970.9140.9310.956*
    GamesRMSE1.3311.3511.5601.1871.176*
    HR0.02310.02050.02010.02380.02450.0246
    AP0.7100.7640.7870.8700.8790.928*
    AppsRMSE1.5371.5341.6301.5741.572*
    HR0.03300.03250.03240.03500.03510.0354*
    AP0.6390.6570.6380.7510.7360.779*
    KindleRMSE0.9080.9001.0640.7000.711*
    HR0.007560.007580.007700.007600.008740.00888*
    AP0.7520.7830.7400.7330.8340.853*
    CDsRMSE1.4061.4051.5891.3881.386
    HR0.003960.004260.004520.004800.004800.00486*
    AP0.8180.8490.8900.9310.9290.923*
    MoviesRMSE1.2751.2761.2101.1881.132*
    HR0.006900.007380.006960.007420.008400.00851*
    AP0.4850.5390.4990.5260.6300.642*
    YelpRMSE0.7520.7490.8960.7230.746*
    HR0.004690.004690.005900.006460.006980.00970*
    AP0.5160.5820.6370.7020.7540.924*
    下载: 导出CSV
  • [1] 李金忠, 刘关俊, 闫春钢, 蒋昌俊. 排序学习研究进展与展. 自动化学报, 2018, 44(08): 1345−1369

    Li J Z, Liu G J, Yan C G, J C J. Research advances and prospects of learning to rank. Acta Automatica Sinica, 2018, 44(08): 1345−1369
    [2] Batmaz Z, Yurekli A, Bilge A, Kaleli C. A review on deep learning for recommender systems: challenges and remedies. Artificial Intelligence Review, 2019, 52(1): 1−37 doi: 10.1007/s10462-018-9654-y
    [3] Wang Z K, Liu H Z, Du Y P, Wu Z H, Zhang X. Unified embedding model over heterogeneous information network for personalized recommendation. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. AAAI Press, 2019.3813−3819.
    [4] 饶子昀, 张毅, 刘俊涛, 曹万华. 应用知识图谱的推荐方法与系统[J/OL]. 自动化学报: 1−17[2021-04-14]. (in Chinese).

    Rao Z Y, Zhang Y, Liu J T, Cao W H. Recommendation methods and systems using knowledge graph. Acta Automatica Sinica, to be published.
    [5] Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conf. on Uncertainty in Artificial Intelligence. AUAI Press, 2009.452−461.
    [6] He X N, Liao L Z, Zhang H W, Nie L Q, Hu X, Chua T. Neural collaborative filtering. In: Proceedings of the 26th international conference on world wide web. International World Wide Web Conferences Steering Committee, 2017.173−182.
    [7] Truong Q T, Salah A, Lauw H W. Bilateral variational autoencoder for collaborative filtering. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining, 2021.292−300.
    [8] Bao L, Sun X Y, Chen Y, Gong D W, Zhang Y W. Restricted Boltzmann machine-driven interactive estimation of distribution algorithm for personalized search. Knowledge-Based Systems, 2020, to be published.
    [9] Kim D, Park C, Oh J, Lee S, Yu H. Convolutional matrix factorization for document context-aware recommendation. In: Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 2016.233−240.
    [10] Jin J, Qin J, Fang Y C, Du K, Yu Y, Zhang Z, et al. An Efficient neighborhood-based interaction model for recommendation on heterogeneous graph. KDD '20: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 2020.
    [11] Mnih V, Heess N, Graves A, Kavukcuoglu K. Recurrent models of visual attention. Advances in neural information processing systems, 2014.
    [12] Zhou C, Bai J Z, Song J S, Liu X F, Zhao Z C, Chen X S, et al. ATRank: An attention-based user behavior modeling framework for recommendation. In: Proceedings of Thirty-Second AAAI Conference on Artificial Intelligence. 2018.
    [13] 汤文兵, 任正云, 韩芳. 基于注意力机制的协同卷积动态推荐网络. 自动化学报: 1-11. (in Chinese).

    Tang W B, Ren Z Y, Han F. Attention-based collaborative convolutional dynamic network for recommendation. Acta Automatica Sinica, to be published.
    [14] Li J C, Wang Y J, Mcauley J. Time interval aware self-attention for sequential recommendation. In: Proceedings of the Thirteenth ACM International Conference on Web Search and Data Mining (WSDM20), February 3-7, 2020, Houston, TX, USA. ACM, New York, NY, USA, 322−330.
    [15] Sun X Y, Gong D W, Jin Y C, Chen S S. A new surrogate-assisted interactive genetic algorithm with weighted semi-supervised learning. IEEE Transactions on Cybernetics, 2013, 42(2): 685−697
    [16] Zhou G R, Mou N, Fan Y, Pi Q, Bian W J, Zhou C, et al. Deep interest evolution network for click-through rate prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence. 2019, 33: 5941−5948.
    [17] Chen Y, Jin Y C, Sun X Y. Language model based interactive estimation of distribution algorithm. Knowledge-Based Systems, 2020, to be published.
    [18] Salameh M, Singh S, Li S W, Krishnamurthy M. Surrogate vibration modeling approach for design optimization of electric machines. IEEE Transactions on Transportation Electrification, 2020, 6(3): 1126−1133 doi: 10.1109/TTE.2020.3017232
    [19] Gao Y, Yang T, Bozhko S, Wheeler P, Dragičević T. Filter design and optimization of electromechanical actuation systems using search and surrogate algorithms for more-electric aircraft applications. IEEE Transactions on Transportation Electrification, 2020, 6(4): 1434−1447 doi: 10.1109/TTE.2020.3019729
    [20] Bao L, Sun X Y, Chen Y, Man G Y, Shao H. Restricted Boltzmann machine-assisted estimation of distribution algorithm for complex problems. Complexity, 2018.
    [21] Zhao F, Zeng Z, Liu H, Lan R, Fan J. Semisupervised approach to surrogate-assisted multiobjective kernel intuitionistic fuzzy clustering algorithm for color image segmentation. IEEE Transactions on Fuzzy Systems, 2020, 28(6): 1023−1034 doi: 10.1109/TFUZZ.2020.2973121
    [22] Min A, Ong Y S, Gupta A, Goh C. Multiproblem surrogates: transfer evolutionary multiobjective optimization of computationally expensive problems. IEEE transactions on evolutionary computation, 2019, 23(1): 15−28 doi: 10.1109/TEVC.2017.2783441
    [23] Wang S, Liu J, Jin Y. Surrogate-assisted robust optimization of large-scale networks based on graph embedding. IEEE Transactions on Evolutionary Computation, 2020, 24(4): 735−749 doi: 10.1109/TEVC.2019.2950935
    [24] Cai X W, Gao L, Li X Y. Efficient generalized surrogate-assisted evolutionary algorithm for high-dimensional expensive problems. IEEE Transactions on Evolutionary Computation, 2020, 24(2): 365−379 doi: 10.1109/TEVC.2019.2919762
    [25] Le Q, Mikolov T. Distributed representations of sentences and documents. In: Proceedings of International conference on machine learning. 2014.1188−1196.
    [26] Hinton G E. Training products of experts by minimizing contrastive divergence. Neural Computation, 2002, 14(8): 1771−1800 doi: 10.1162/089976602760128018
    [27] Feng S, Chen C L P. A fuzzy restricted Boltzmann machine: novel learning algorithms based on the crisp possibilistic mean value of fuzzy numbers. IEEE Transactions on Fuzzy Systems, 2018, 26(1): 117−130 doi: 10.1109/TFUZZ.2016.2639064
  • 加载中
计量
  • 文章访问数:  246
  • HTML全文浏览量:  38
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-07-29
  • 修回日期:  2021-04-02
  • 网络出版日期:  2021-06-01

目录

    /

    返回文章
    返回