2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于文本引导的注意力图像转发预测排序网络

潘文雯 赵洲 俞俊 吴飞

潘文雯, 赵洲, 俞俊, 吴飞. 基于文本引导的注意力图像转发预测排序网络. 自动化学报, 2021, 47(x): 1−10 doi: 10.16383/j.aas.c200629
引用本文: 潘文雯, 赵洲, 俞俊, 吴飞. 基于文本引导的注意力图像转发预测排序网络. 自动化学报, 2021, 47(x): 1−10 doi: 10.16383/j.aas.c200629
Pan Wen-Wen, Zhao Zhou, Yu Jun, Wu Fei. Textually guided ranking network for attentional image retweet modeling. Acta Automatica Sinica, 2021, 47(x): 1−10 doi: 10.16383/j.aas.c200629
Citation: Pan Wen-Wen, Zhao Zhou, Yu Jun, Wu Fei. Textually guided ranking network for attentional image retweet modeling. Acta Automatica Sinica, 2021, 47(x): 1−10 doi: 10.16383/j.aas.c200629

基于文本引导的注意力图像转发预测排序网络

doi: 10.16383/j.aas.c200629
基金项目: 重点研发计划(2018AAA0100603); 浙江省自然基金浙江省杰青(LR19F020006); 自然基金重点项目(61836002); 广东省联合基金(U1611461); 应急重点基金(61751209); China Knowledge Centre for Engineering Sciences and Technology
详细信息
    作者简介:

    潘文雯:2018年获得西安交通大学计算机科学与技术学士学位, 目前正在攻读浙江大学计算机科学博士学位. 研究领域包括机器学习和自然语言处理. Email: wenwenpan@zju.edu.cn

    赵洲:2010年获中国香港科技大学计算机科学学士学位, 2015年获中国香港科技大学计算机科学博士学位. 他目前是浙江大学计算机科学学院副教授. 他的研究兴趣包括机器学习和数据挖掘. 本文通讯作者. Email: zhaozhou@zju.edu.cn

    俞俊:2009年获浙江大学计算机应用专业博士学, 教授, 杭州电子科技博士生导师. 2009-2011任新加坡南洋理工大学计算机系博士后, 2011-2014年任厦门大学信息学院副教授. 研究领域包括计算机动画图像处理, 机器学习等. Email: yujun@hdu.edu.cn

    吴飞:1996年获兰州大学计算机科学学士学位, 1999年获中国澳门大学计算机科学硕士学位, 2002年获浙江大学计算机科学博士学位. 浙江大学求是特聘教授, 博士生导师. 研究领域为人工智能、多媒体分析与检索和统计学习理论. Email: wufei@cs.zju.edu.cn

Textually Guided Ranking Network for Attentional Image Retweet Modeling

Funds: Support by National Key R&D Program of China (2018AAA0100603); Zhejiang Natural Science Foundation LR19F02000; National Natural Science Foundation of China under Grant(61836002, U1611461, 61751209); China Knowledge Centre for Engineering Sciences and Technology
More Information
    Author Bio:

    PAN Wen-Wen received the B.E. degree in computer science and technology from Xi'an Jiaotong University, China, in 2018, where she is currently pursuing the Ph.D. degrees in computer science in Zhejiang University. Her research interests include machine learning and natural language processing

    ZHAO Zhou received the B.S. and Ph.D. degrees in computer science from The Hong Kong University of Science and Technology, China, in 2010 and 2015, respectively. He is currently an Associate Professor with the College of Computer Science, Zhejiang University. His research interests include machine learning and data mining. Corresponding author of this paper

    YU Jun received the B.Eng. and Ph.D. degrees from Zhejiang University, Zhejiang, China, in 2009. He is currently a Professor with the School of Computer Science and Technology, Hangzhou Dianzi University. His research interests include computer animation image processing, machine learning, etc

    WU Fei received the B.Sc. degree in computer science from Lanzhou University in 1996, the M.Sc. degree in computer science from the University of Macau, China, in 1999, and the Ph.D. degree in computer science from Zhejiang University in 2002. He is Qiushi Distinguished Professor and Doctoral Supervisor of Zhejiang University. His research interests include artificial intelligence, multimedia analysis and retrieval, and statistical learning theory

  • 摘要: 转发预测在社交媒体网站(Social media sites, SMS)中是一个很有挑战性的问题. 本文研究了SMS中的图像转发预测问题, 预测用户再次转发图像推特的图像共享行为. 与现有的研究不同, 本文首先提出异构图像转发建模网络(IRM), 所利用的是用户之前转发图像推特中的相关内容、之后在SMS中的联系和被转发者的偏好三方面的内容. 在此基础上, 提出文本引导的多模态神经网络, 构建新型多方面注意力排序网络学习框架, 从而学习预测任务中的联合图像推特表征和用户偏好表征. 在Twitter的大规模数据集上进行的大量实验表明, 我们的方法较之现有的解决方案而言取得了更好的效果.
  • 图  1  图像推特行为示例

    Fig.  1  An example of image retweet behavior

    图  2  用于图像转发预测的注意多方面排序网络学习纵览

    Fig.  2  The overview of textually guided ranking network for attentional image retweet modeling

    图  3  文本引导的多模融合网络

    Fig.  3  Textually guided multi-modal fusion network

    图  4  AMNL+在图像转发预测任务中的实验结果

    Fig.  4  Experimental result of AMNL+ on the image retweet prediction task

    图  5  随着epoch客观价值和运行时间的变化

    Fig.  5  Objective value and running time versus the number of epochs

    表  1  不同方法的Precision@1结果

    Table  1  Experimental results on precision@1 of different approaches

    方法Precision@1
    60%70%80%
    RRFM0.62530.64740.6583
    VBPR0.63990.65250.6793
    D-RNN0.70010.71910.7385
    IRBLRUS0.71930.72950.7516
    ADABPR0.63940.64880.6692
    CITING0.74630.76080.7773
    AMNL0.86910.89750.9008
    AMNL+0.93410.94440.9585
    下载: 导出CSV

    表  2  不同方法的Precision@3结果

    Table  2  Experimental results on precision@3 of different approaches

    方法Precision@3
    60%70%80%
    RRFM0.59730.62840.6400
    VBPR0.60820.63040.6432
    D-RNN0.64680.67020.6879
    IRBLRUS0.65930.66840.6813
    ADABPR0.59800.61980.6301
    CITING0.73040.74670.7677
    AMNL0.75190.77910.7959
    AMNL+0.86800.87960.8823
    下载: 导出CSV

    表  3  不同方法的AUC结果

    Table  3  Experimental results on AUC of different approaches

    方法AUC
    60%70%80%
    RRFM0.50320.51950.5282
    VBPR0.54910.57990.5814
    D-RNN0.68340.69730.6999
    IRBLRUS0.71450.73420.7440
    ADABPR0.53930.56010.5782
    CITING0.58020.59820.6425
    AMNL0.77030.79980.8486
    AMNL+0.87920.89860.9126
    下载: 导出CSV

    表  4  用80%的数据进行训练, 消融实验的实验结果

    Table  4  Experimental results with different modalities and components using 80% of the data for training

    方法Precision@1Precision@3AUC
    AMNL+i0.84270.76730.8204
    AMNLd0.78920.77190.7962
    AMNLhfunc0.85980.79000.8095
    AMNL0.90080.79590.8486
    AMNL+i0.92270.82760.8724
    AMNL+hfunc0.91990.81950.8689
    AMNL+0.95850.88230.9126
    下载: 导出CSV
  • [1] Zhang Q, Gong Y, Guo Y, Huang X. Retweet behavior prediction using hierarchical dirichlet process. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI'15), Texas, USA: AAAI Press, 2015. 29(1). 403–409.
    [2] Chen T, He X, Kan M Y. Context-aware image tweet modelling and recommendation. In: Proceedings of the 24th ACM international conference on Multimedia (MM '16), New York, USA: ACM Press, 2016. 1018–1027.
    [3] Zhang J, Tang J, Li J, Xing C. Who influenced you? predicting retweet via social influence locality. In: ACM Transactions on Knowledge Discovery from Data (TKDD), 2015. 9(3): 1–26.
    [4] Firdaus S N, Ding C, Sadeghian A. Topic specific emotion detection for retweet prediction. In: International Journal of Machine Learning and Cybernetics, 2019. 10(8): 2071–2083.
    [5] Szegedy C, Toshev A, Erhan D. Deep neural networks for object detection. In: Proceedings of the 26th International Conference on Neural Information Processing Systems (NIPS'13), Lake Tahoe, Nevada, United States: 2013. 26: 2553–2561.
    [6] Zhao Z, Yang Q, Lu H, et al. Social-aware movie recommendation via multimodal network learning. In: IEEE Transactions on Multimedia, 2017. 20(2): 430–440.
    [7] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations (ICLR’15), San Diego, CA, USA: 2015.
    [8] Hochreiter S, Schmidhuber J. Long short-term memory. In: Neural computation, 1997. 9(8): 1735–1780.
    [9] Hoang T A, Lim E P. Retweeting: An act of viral users, susceptible users, or viral topics? In: Proceedings of the 2013 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, Austin, Texas, USA: SIAM Press, 2013. 569–577.
    [10] Jiang B, Lu Z, Li N, Wu J, Jiang Z. Retweet prediction using social-aware probabilistic matrix factorization. In: International Conference on Computational Science, Wuxi, China: Springer Press, 2018. 316–327.
    [11] Atrey P K, Hossain M A, El Saddik A, Kankanhalli MS. Multimodal fusion for multimedia analysis: a survey. In: Multimedia systems, 2010. 16(6): 345–379.
    [12] Yuan Z, Sang J, Xu C, Liu Y. A unified framework of latent feature learning in social media. In: IEEE Transactions on Multimedia, China, 2014. 16(6): 1624–1635.
    [13] Luong M T, Pham H, Manning C D. Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP’15), Lisbon, Portugal: The Association for Computational Linguistics Press, 2015. 1412–1421.
    [14] Nie L, Yan S, Wang M, Hong R, Chua TS. Harvesting visual concepts for image search with complex queries. In: Proceedings of the 20th ACM international conference on Multimedia (MM '12), New York, USA: ACM Press, 2012. 59–68.
    [15] Yang Y, Tang J, Leung C, et al. Rain: Social role-aware information diffusion. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI'15), Texas, USA: AAAI Press, 2015. 29(1).
    [16] Chen J, Li H, Wu Z, Hossain MS. Sentiment analysis of the correlation between regular tweets and retweets. In: 2017 IEEE 16th international symposium on network computing and applications, Hawaii USA: IEEE Press, 2017. 1–5.
    [17] Macskassy S, Michelson M. Why do people retweet? anti-homophily wins the day! In: Proceedings of the International AAAI Conference on Web and Social Media, Barcelona, Catalonia, Spain: AAAI Press, 2011. 5(1).
    [18] Xu Z, Zhang Y, Wu Y, Yang Q. Modeling user posting behavior on social media. In: Proceedings of the 35th international ACM SIGIR conference on research and development in information retrieval (SIGIR'12), Portland, USA: ACM Press, 2012. 545–554.
    [19] Luo Z, Osborne M, Tang J, Wang T. Who will retweet me? Finding retweeters in Twitter. In: Proceedings of the 36th international ACM SIGIR conference on Research and development in information retrieval (SIGIR '13), Dublin, Ireland: ACM Press, 2013. 869–872.
    [20] Bourigault S, Lagnier C, Lamprier S, Denoyer L, Gallinari P. Learning social network embeddings for predicting information diffusion. In: Proceedings of the 7th ACM international conference on Web search and data mining, New York, USA: ACM Press, 2014. 393–402.
    [21] Bi B, Cho J. Modeling a retweet network via an adaptive bayesian approach. In: Proceedings of the 25th international conference on world wide web, Montreal, Canada: ACM Press, 2016. 459–469.
    [22] Jiang B, Liang J, Sha Y, Wang L. Message clustering based matrix factorization model for retweeting behavior prediction. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management (CIKM’15), Melbourne, Australia: ACM Press, 2015. 1843–1846.
    [23] Wang B, Wang C, Bu J, et al. Whom to mention: expand the diffusion of tweets by@ recommendation on micro-blogging systems. In: Proceedings of the 22nd international conference on World Wide Web (WWW'13), Rio de Janeiro, Brazil: ACM Press, 2013. 1331–1340.
    [24] Liu Y, Zhao J, Xiao Y. C-RBFNN: A user retweet behavior prediction method for hotspot topics based on improved RBF neural network. In: Neurocomputing, 2018. 275: 733–746.
    [25] Firdaus S N, Ding C, Sadeghian A. Retweet prediction considering user's difference as an author and retweeter. In: 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM’16), 2016, San Francisco, USA: IEEE Press, 2016. 852–859.
    [26] Feng W, Wang J. Retweet or not? Personalized tweet re-ranking. In: Proceedings of the sixth ACM international conference on Web search and data mining, Rome, Italy: ACM Press, 2013. 577–586.
    [27] Peng H K, Zhu J, Piao D, Yan R, Zhang Y. Retweet modeling using conditional random fields. In: 2011 IEEE 11th International Conference on Data Mining Workshops, Vancouver, BC, Canada: IEEE Press, 2011. 336–343.
    [28] Chen K, Chen T, Zheng G, et al. Collaborative personalized tweet recommendation. In: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval (SIGIR'12), Portland, USA: ACM Press, 2012. 661–670.
    [29] Nie L, Song X, Chua T S. Learning from multiple social networks. In: Synthesis Lectures on Information Concepts, Anchorage, AK, USA: IEEE Press, 2016. 8(2): 1–118.
    [30] Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Boston, MA, USA: IEEE Press, 2015. 1–9.
    [31] Zhang H, Kyaw Z, Chang S F, Chua T S. Visual translation embedding network for visual relation detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA: IEEE Press, 2017. 5532–5540.
    [32] Zhao W, Guan Z, Luo H, Peng J, Fan J. Deep Multiple Instance Hashing for Object-based Image Retrieval. In: International Joint Conference on Artificial Intelligence, Melbourne, Australia: ijcai. org, 2017. 3504–3510.
    [33] Zhao Z, Lin J, Jiang X, et al. Video question answering via hierarchical dual-level attention network learning. In: Proceedings of the 25th ACM international conference on Multimedia, Mountain View, CA, USA: ACM Press, 2017. 1050–1058.
    [34] Nair V, Hinton G E. Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, ser. ICML’10. Haifa, Israel: Omni Press, 2010. 807–814.
    [35] Zhao Z, Yang Q, Cai D, et al. Video Question Answering via Hierarchical Spatio-Temporal Attention Networks. In: International Joint Conference on Artificial Intelligence (IJCAI), Melbourne, Australia: ijcai. org, 2017. 2, 19–25.
    [36] Java A, Song X, Finin T, Tseng B. Why we twitter: understanding microblogging usage and communities. In: Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 workshop on Web mining and social network analysis, CA, USA: Springer Press, 2007. 56–65.
    [37] Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing, Doha, Qatar: ACL Press, 2014. 1532–1543.
    [38] He R, McAuley J. VBPR: visual bayesian personalized ranking from implicit feedback. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI'16), Phoenix, Arizona, USA: AAAI Press, 2016. 30(1).
    [39] Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, Montreal, QC, Canada: AUAI Press, 2009. 452–461.
    [40] Li H, Hong R, Lian D, et al. A Relaxed Ranking-Based Factor Model for Recommender System from Implicit Feedback. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, New York, USA: IJCAI Press, 2016. 1683–1689.
  • 加载中
计量
  • 文章访问数:  21
  • HTML全文浏览量:  9
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-06-29
  • 修回日期:  2020-10-15
  • 网络出版日期:  2021-03-23

目录

    /

    返回文章
    返回