-
摘要: 近年来,细粒度情感分析因其在商业决策、舆情分析等领域的重要作用而受到学术界和工业界的广泛关注.评价对象抽取作为情感分析的基本任务之一,是进行细粒度情感分析的关键问题.本文针对评价对象抽取问题的起源、当前主流研究方法和趋势进行了梳理,首先详细阐述评价对象抽取问题的基本概念并对其进行形式化表示,然后结合近年来的研究对评价对象抽取方法进行归纳和总结,并重点分析基于频率、基于模板规则、基于图论、基于条件随机场和基于深度学习的评价对象抽取方法,随后回顾评价对象抽取的评测情况和可用的语料资源,最后分析评价对象抽取的若干难点问题,同时对评价对象抽取研究进展和发展趋势进行总结和展望.Abstract: In recent years, fine-grained sentiment analysis has received extensive attention from academia and industry for its important role in business decision-making and public opinion analysis. Opinion target extraction, as one of the basic tasks of sentiment analysis, has been the crux of fine-grained sentiment analysis for years. In this paper, the origins, state of the art, and research directions of opinion target extraction are discussed. We first elaborate the basic concepts of opinion target extraction and formalize the problem, and then based on recent literature, we conclude and summarize the approaches and techniques, which we divide into five categories:frequency-based method, pattern-based method, graph-based method, CRF-based method and deep learning based method. We also review several evaluation contests and collect the available corpus resources on opinion target extraction. At the end, the challenges arisen in opinion target extraction are analyzed as well as the probable future is given.1) 本文责任编委 赵铁军
-
表 1 基于频率的抽取方法比较
Table 1 Comparison of frequency-based extraction methods
文献 方法 级别 数据集 语言 实验结果1 文献[3]Hu等2004 关联规则挖掘频繁项集+剪枝+根据评价词获取非频繁评价对象 语料 Digital Camera 1 English 0.747 0.822 0.783 Digital Camera 2 English 0.710 0.792 0.749 Cellular Phone English 0.718 0.761 0.739 MP3 Player English 0.692 0.818 0.750 DVD Player English 0.743 0.797 0.770 文献[4]Popescu等2005 PMI Assessment 语料 Digital Camera 1 English 0.89 0.80 0.84 Digital Camera 2 English 0.87 0.74 0.80 Cellular Phone English 0.89 0.74 0.81 MP3 Player English 0.86 0.80 0.83 DVD Player English 0.90 0.78 0.84 文献[8]Xu等2012 Skip-Bigram +关联规则+剪枝 语料 Mobile Phone Chinese 0.4081 0.9529 0.5715 Digital Camera Chinese 0.3828 0.7153 0.4987 文献[9]Li等2015 文献[3]的基础上+基于词序的过滤+基于PMI-IR的过滤 语料 Mobile Phone 1 Chinese 0.732 0.667 0.698 Mobile Phone 2 Chinese 0.791 0.850 0.819 Digital Camera 1 Chinese 0.721 0.756 0.738 Digital Camera 2 Chinese 0.719 0.639 0.676 1实验结果分别表示精确率(Precision)、召回率(Recall)和F1值. 表 2 基于模板规则的抽取方法比较
Table 2 Comparison of pattern-based extraction methods
文献 方法 级别 数据集 语言 实验结果1 文献[10]Zhuang等2006 Dependency template, Supervised 语料 IMDb Movie Reviews English 0.483 0.585 0.5292 文献[19]刘鸿宇等2010 句法路径+词频过滤+ PMI过滤+名词剪枝 句子 Car Chinese 0.3126 0.4127 0.3557 (S) 0.4513 0.5958 0.5136 (L) Camera Chinese 0.3219 0.4006 0.3569 (S) 0.4556 0.567 0.5052 (L) Phone Chinese 0.3472 0.381 0.3633 (S) 0.4976 0.546 0.5206 (L) NoteBook Chinese 0.3379 0.4566 0.3883 (S) 0.4693 0.6342 0.5394 (L) 文献[13]Qiu等2011 Double propagation 语料 D1 English 0.87 0.81 0.84 D2 English 0.90 0.81 0.85 D3 English 0.90 0.86 0.88 D4 English 0.81 0.84 0.82 D5 English 0.92 0.86 0.89 1实验结果中S表示Strict评测方式, L表示Lenient评测方式.
2Based on feature-opinion pairs.表 3 基于图论的抽取方法比较
Table 3 Comparison of graph-based extraction methods
文献 方法 级别 数据集 语言 实验结果 文献[18]Liu等2012 词对齐+随机游走 语料 Camera Chinese 0.75 0.81 0.78 Car Chinese 0.71 0.71 0.71 Laptop Chinese 0.61 0.85 0.71 Phone Chinese 0.83 0.74 0.78 Hotel English 0.71 0.80 0.75 MP3 English 0.70 0.82 0.76 Restaurant Chinese 0.80 0.84 0.82 D1 English 0.84 0.85 0.84 D2 English 0.87 0.85 0.86 D3 English 0.88 0.89 0.88 D4 English 0.81 0.85 0.83 D5 English 0.89 0.87 0.88 文献[20]Xu等2013 依存句法分析+随机游走+直推式支持向量机 语料 D1 English 0.86 0.82 0.84 D2 English 0.88 0.83 0.85 D3 English 0.89 0.86 0.87 D4 English 0.83 0.86 0.84 D5 English 0.89 0.85 0.87 文献[21]Zhou等2013 标签传播 句子 Tencent Weibo Chinese 0.43 0.39 0.41 (Strict)
0.61 0.55 0.58 (Soft)表 4 IOB和IOBES标注例子
Table 4 Example of IOB and IOBES annotation
A few tips : skip the turnip cake and roast pork buns . IOB O O O O O O B I O B I I O IOBES O O O O O O B E O B I E O 表 5 基于CRF模型抽取方法比较
Table 5 Comparison of CRF-based extraction methods
文献 方法 级别 数据集 语言 实验结果 文献[25]Jakob等20101 CRF (Lexicon-related features) 句子 Movies English 0.749 0.661 0.702 Web Services English 0.722 0.526 0.609 Cars English 0.622 0.414 0.497 Cameras English 0.614 0.423 0.500 文献[27]徐冰等2011 CRF (浅层句法特征+启发式位置特征) 句子 数码相机 Chinese 0.5097 0.3579 0.4206 (S) 0.7295 0.5122 0.6019 (L) 手机 Chinese 0.5679 0.3631 0.4430 (S) 0.7761 0.4962 0.6054 (L) 笔记本 Chinese 0.5843 0.4200 0.4887 (S) 0.7692 0.5529 0.6433 (L) 汽车 Chinese 0.4302 0.2060 0.2786 (S) 0.6265 0.3001 0.4058 (L) 文献[31]Liao等2016 CRF (Lexicon-related features + Syntactic and semantic information) 句子 COAE 2014微博数据 Chinese 0.6901 0.4605 0.5523 (S) 0.7432 0.4855 0.5873 (L) 1作者还进行了跨领域的实验, 这里只给出单领域的结果. 表 6 基于深度学习的抽取方法比较
Table 6 Comparison of deep learning based extraction methods
文献 方法 级别 数据集 语言 实验结果 文献[30]Liu等2015 Recurrent Neural Networks + Word Embedding + Linguistic Features 句子 Laptop1 English 0.7457 (F1)3 0.7500 (F1)4 Restaurant1 English 0.8206 (F1)3 0.8082 (F1)4 文献[28]Wang等2016 Dependency-Tree Recursive Neural Networks + CRF + Linguistic/Lexicon Features (Name List and POS Tag) 句子 Laptop1 English 0.7809 (F1) Restaurant1 English 0.8473 (F1) 文献[29]Yin等2016 Word Embedding + Dependency Path Embedding + CRF 句子 Laptop1 English 0.7516 (F1) Restaurant1 English 0.8497 (F1) Restaurant2 English 0.6973 (F1) 文献[32]Poria等2016 Convolutional Neural Network + Linguistic Patterns 句子 Laptop1 English 0.8672 0.7835 0.8232 Restaurant1 English 0.8827 0.8610 0.8717 1SemEval-2014 Task 4数据集
2SemEval-2015 Task 12数据集
3双向Elman Type RNN + Amazon词向量+语言学特征
4LSTM-RNN + Amazon词向量+语言学特征表 7 在Customer Review Datasets上的实验结果
Table 7 Experimental results on Customer Review Datasets
文献 方法 实验结果 准确率(%) 召回率(%) F1值 文献[3], 2004 Frequency-based 72.20 79.80 75.81 文献[4], 2005 PMI 88.20 77.20 82.33 文献[13], 2011 Double Propagation 88.00 83.60 85.74 文献[18], 2012 Word Alignment + Random Walk 85.80 86.20 86.00 文献[35], 2014 Rule-based 89.41 91.42 90.40 文献[32], 2016 CNN-based 90.19 86.18 88.14 表 8 在SemEval-2014 Task 4 ABSA Datasets上的实验结果
Table 8 Experimental results on SemEval-2014 Task 4 ABSA Datasets
文献 方法 实验结果1 准确率(%) 召回率(%) F1值 文献[36], 2014 Frequency-based 23.00 25.00 24.00 37.00 40.00 38.00 文献[37], 2014 Pattern-based 32.10 42.50 36.60 57.50 64.50 60.80 文献[38], 2014 CRF-based N/A N/A 74.55 N/A N/A 79.62 文献[32], 2016 Convolutional Neural Network + Linguistic Patterns 86.72 78.35 82.32 88.27 86.10 87.17 1上行表示Laptop领域数据集实验结果, 下行表示Restaurant领域数据集实验结果. 表 9 各种方法优缺点比较
Table 9 Comparison of advantages and disadvantages of various methods
方法 优点 缺点 基于频率的方法 方法简单直观, 在产品评论领域可以取得较好效果 抽取非频繁评价对象需要额外手段 相似领域间迁移容易 难以捕捉句子含义 不相似领域间迁移效果差, 如新闻评论领域 基于模板规则的方法 时间复杂度低 依赖依存句法分析器的质量 不需要大量标注语料 编制模板规则需要专家知识 基于图论的方法 结合了评价对象和情感词的共现频率关系 没考虑无形容词性评价词的句子 领域可移植性强 频繁噪音词和非频繁长尾词需要进一步筛选 基于CRF的方法 识别准确率高 需要大量训练语料 可以任意添加特征 模型训练时间长 领域迁移则需要重新训练 基于深度学习的方法 避免大量特征工程工作 领域相关词向量训练耗时 识别效果好 需要大量标注训练语料 从一定程度上能从语义角度分析评价对象 模型训练时间复杂度高 领域迁移困难 -
[1] Liu B. Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies. San Rafael, France:Morgan and Claypool Publishers, 2012. [2] Pang B, Lee L. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval. Delft, Netherlands:Now Publishers Inc, 2008. [3] Hu M Q, Liu B. Mining and summarizing customer reviews. In:Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Seattle, USA:ACM, 2004. 168-177 [4] Popescu A M, Etzioni O. Extracting product features and opinions from reviews. In:Proceedings of the 2005 Conference on Human Language Technology and Empirical Methods in Natural Language Processing. Vancouver, Canada:Association for Computational Linguistics, 2005. 339-346 [5] Blair-Goldensohn S, Hannan K, McDonald R, Neylon T, Reis G A, Reynar J. Building a sentiment summarizer for local service reviews. In:Proceedings of the 2008 WWW Workshop on NLP in the Information Explosion Era (NLPIX 2008). Beijing, China:ACM, 2008. 339-348 [6] Scaffidi C, Bierhoff K, Chang E, Felker M, Ng H, Jin C. Red opal:product-feature scoring from reviews. In:Proceedings of the 8th ACM Conference on Electronic Commerce (ACMEC 2007). San Diego, USA:ACM, 2007. 182-191 [7] Yi J, Nasukawa T, Bunescu R, Niblack W. Sentiment analyzer:extracting sentiments about a given topic using natural language processing techniques. In:Proceedings of the 3rd IEEE International Conference on Data Mining. Melbourne, USA:IEEE, 2003. 427-434 [8] Xu G, Huang C R, Wang H F. Extracting Chinese product features:representing a sequence by a set of skip-bigrams. In:Proceedings of the 2012 Chinese Lexical Semantics. Wuhan, China:Springer-Verlag, 2012. 72-83 [9] Li S, Zhou L N, Li Y J. Improving aspect extraction by augmenting a frequency-based method with web-based similarity measures. Information Processing and Management, 2015, 51(1):58-67 doi: 10.1016/j.ipm.2014.08.005 [10] Zhuang L, Jing F, Zhu X Y. Movie review mining and summarization. In:Proceedings of the 15th ACM international Conference on Information and Knowledge Management. Arlington, Virginia, USA:ACM, 2006. 43-50 [11] Jakob N, Gurevych I. Using anaphora resolution to improve opinion target identification in movie reviews. In:Proceedings of the 2010 ACL Conference Short Papers. Uppsala, Sweden:Association for Computational Linguistics, 2010. 263-268 [12] 宋晓雷, 王素格, 李红霞.面向特定领域的产品评价对象自动识别研究.中文信息学报, 2010, 24(1):89-93 http://cdmd.cnki.com.cn/Article/CDMD-10464-1015902382.htmSong Xiao-Lei, Wang Su-Ge, Li Hong-Xia. Research on comment target recognition for specific domain products. Journal of Chinese Information Processing, 2010, 24(1):89-93 http://cdmd.cnki.com.cn/Article/CDMD-10464-1015902382.htm [13] Qiu G, Liu B, Bu J, Chen C. Opinion word expansion and target extraction through double propagation. Computational Linguistics, 2011, 37(1):9-27 doi: 10.1162/coli_a_00034 [14] Zhang L, Liu B, Lim S H, O'Brien-Strain E. Extracting and ranking product features in opinion documents. In:Proceedings of the 23rd International Conference on Computational Linguistics:Posters. Beijing, China:Association for Computational Linguistics, 2010. 1462-1470 [15] 赵妍妍, 秦兵, 车万翔, 刘挺.基于句法路径的情感评价单元识别.软件学报, 2011, 22(5):887-898 http://mall.cnki.net/magazine/Article/RJXB201105005.htmZhao Yan-Yan, Qin Bing, Che Wan-Xiang, Liu Ting. Appraisal expression recognition based on syntactic path. Journal of Software, 2011, 22(5):887-898 http://mall.cnki.net/magazine/Article/RJXB201105005.htm [16] Liu Q, Gao Z Q, Liu B, Zhang Y L. Automated rule selection for aspect extraction in opinion mining. In:Proceedings of the 24th International Conference on Artificial Intelligence. Buenos Aires, Argentina:AAAI Press, 2015. 1291-1297 [17] 江腾蛟, 万常选, 刘德喜, 刘喜平, 廖国琼.基于语义分析的评价对象——情感词对抽取.计算机学报, 2017, 40(3):617-633 doi: 10.11897/SP.J.1016.2017.00617Jiang Teng-Jiao, Wan Chang-Xuan, Liu De-Xi, Liu Xi-Ping, Liao Guo-Qiong. Extracting target-opinion pairs based on semantic analysis. Chinese Journal of Computers, 2017, 40(3):617-633 doi: 10.11897/SP.J.1016.2017.00617 [18] Liu K, Xu L H, Zhao J. Opinion target extraction using word-based translation model. In:Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. Jeju, South Korea:Association for Computational Linguistics, 2012. 1346-1356 [19] 刘鸿宇, 赵妍妍, 秦兵, 刘挺.评价对象抽取及其倾向性分析.中文信息学报, 2010, 24(1):84-89, 122 http://cdmd.cnki.com.cn/Article/CDMD-11232-1014031421.htmLiu Hong-Yu, Zhao Yan-Yan, Qin Bing, Liu Ting. Comment target extraction and sentiment classification. Journal of Chinese Information Processing, 2010, 24(1):84-89, 122 http://cdmd.cnki.com.cn/Article/CDMD-11232-1014031421.htm [20] Xu L H, Liu K, Lai S W, Chen Y B, Zhao J. Mining opinion words and opinion targets in a two-stage framework. In:Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics. Sofia, Bulgaria:Association for Computational Linguistics, 2013. 1764-1773 [21] Zhou X J, Wan X J, Xiao J G. Collective opinion target extraction in Chinese microblogs. In:Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, Washington, USA:Association for Computational Linguistics, 2013. 1840-1850 [22] Jin W, Ho H H, Srihari R K. A novel lexicalized HMM-based learning framework for web opinion mining. In:Proceedings of the 26th Annual International Conference on Machine Learning. Montreal, Canada:ACM, 2009. 465-472 [23] McCallum A, Freitag D, Pereira F C N. Maximum entropy Markov models for information extraction and segmentation. In:Proceedings of the 17th International Conference on Machine Learning. Burlington, Massachusetts, USA:Morgan Kaufmann Publishers, 2000. 591-598 [24] Lafferty J D, McCallum A, Pereira F C N. Conditional random fields:probabilistic models for segmenting and labeling sequence data. In:Proceedings of the 18th International Conference on Machine Learning. Burlington, Massachusetts, USA:Morgan Kaufmann Publishers, 2001. 282-289 [25] Jakob N, Gurevych I. Extracting opinion targets in a single-and cross-domain setting with conditional random fields. In:Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Cambridge, Massachusetts, USA:Association for Computational Linguistics, 2010. 1035-1045 [26] 张莉, 钱玲飞, 许鑫.基于核心句及句法关系的评价对象抽取.中文信息学报, 2011, 25(3):23-29 https://www.wenkuxiazai.com/doc/874acfbbf121dd36a32d8277.htmlZhang Li, Qian Ling-Fei, Xu Xin. Comment target extraction based on nuclear sentences and syntactic relations. Journal of Chinese Information Processing, 2011, 25(3):23-29 https://www.wenkuxiazai.com/doc/874acfbbf121dd36a32d8277.html [27] 徐冰, 赵铁军, 王山雨, 郑德权.基于浅层句法特征的评价对象抽取研究.自动化学报, 2011, 37(10):1241-1247 http://www.aas.net.cn/CN/abstract/abstract17613.shtmlXu Bing, Zhao Tie-Jun, Wang Shan-Yu, Zheng De-Quan. Extraction of opinion targets based on shallow parsing features. Acta Automatica Sinica, 2011, 37(10):1241-1247 http://www.aas.net.cn/CN/abstract/abstract17613.shtml [28] Wang W Y, Pan S J, Dahlmeier D, Xiao X K. Recursive neural conditional random fields for aspect-based sentiment analysis. In:Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin, Texas, USA:Association for Computational Linguistics, 2016. 616-626 [29] Yin Y C, Wei F R, Dong L, Xu K M, Zhang M, Zhou M. Unsupervised word and dependency path embeddings for aspect term extraction. In:Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI-16). New York, USA:AAAI Press, 2016. 2979-2985 [30] Liu P F, Joty S R, Meng H M. Fine-grained opinion mining with recurrent neural networks and word embeddings. In:Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal:Association for Computational Linguistics, 2015. 1433-1443 [31] Liao C, Feng C, Yang S, Huang H Y. A hybrid method of domain lexicon construction for opinion targets extraction using syntax and semantics. Journal of Computer Science and Technology, 2016, 31(3):595-603 doi: 10.1007/s11390-016-1649-z [32] Poria S, Cambria E, Gelbukh A. Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems, 2016, 108:42-49 doi: 10.1016/j.knosys.2016.06.009 [33] Manning C D. Computational linguistics and deep learning. Computational Linguistics, 2015, 41(4):701-707 doi: 10.1162/COLI_a_00239 [34] 许延祥, 罗铁坚, 周佳, 王竹.评价文本中意见分布规律研究.中文信息学报, 2014, 28(3):150-158 http://zhifu.cnki.net/Alipay.aspxXu Yan-Xiang, Luo Tie-Jian, Zhou Jia, Wang Zhu. Research on opinion distribution in reviews. Journal of Chinese Information Processing, 2014, 28(3):150-158 http://zhifu.cnki.net/Alipay.aspx [35] Poria S, Cambria E, Ku L W, Gui C, Gelbukh A. A rule-based approach to aspect extraction from product reviews. In:Proceedings of the 2nd Workshop on Natural Language Processing for Social Media (SocialNLP). Dublin, Ireland:Association for Computational Linguistics, 2014. 28-37 [36] Bornebusch F, Cancino G, Diepenbeck M, Drechsler R, Djomkam S, Fanseu A N, Jalali M, Michael M, Mohsen J, Nitze M, Plump C, Soeken M, Tchambo F, Toni, Ziegler H. iTac:aspect based sentiment analysis using sentiment trees and dictionaries. In:Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin, Ireland:Association for Computational Linguistics, 2014. 351-355 [37] García-Pablos A, Rigau G, Cuadros M C S. V3:unsupervised generation of domain aspect terms for aspect based sentiment analysis. In:Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin, Ireland:Association for Computational Linguistics, 2014. 833-837 [38] Chernyshevich M. IHS R & D Belarus:cross-domain extraction of product features using conditional random fields. In:Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin, Ireland:Association for Computational Linguistics, 2014. 309-313 [39] Fang L, Liu B, Huang M L. Leveraging large data with weak supervision for joint feature and opinion word extraction. Journal of Computer Science and Technology, 2015, 30(4):903-916 doi: 10.1007/s11390-015-1569-3 [40] Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S. Semeval-2014 task 4:aspect based sentiment analysis. In:Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin, Ireland:Association for Computational Linguistics, 2014. 27-35 [41] Toh Z, Wang W T. Dlirec:aspect term extraction and term polarity classification system. In:Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin, Ireland:Association for Computational Linguistics, 2014. 235-240 [42] Brun C, Popa D N, Roux C. Xrce:hybrid classification for aspect-based sentiment analysis. In:Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Dublin, Ireland:Association for Computational Linguistics, 2014. 838-842 [43] Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I. Semeval-2015 task 12:aspect based sentiment analysis. In:Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). Denver, Colorado, USA:Association for Computational Linguistics, 2015. 486-495 [44] Saralegi I S V X, Agerri R. EliXa:a modular and flexible ABSA platform. In:Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). Denver, Colorado, USA:Association for Computational Linguistics, 2015. 748-752 [45] Toh Z, Su J. NLANGP:supervised machine learning system for aspect category classification and opinion target extraction. In:Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). Denver, Colorado, USA:Association for Computational Linguistics, 2015. 496-501 [46] Zhang S, Jia W J, Xia Y J, Meng Y, Yu H. Extracting product features and sentiments from Chinese customer reviews. In:Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC'10). Valletta, Malta:European Language Resources Association (ELRA), 2010. 1142-1145 [47] 侯敏, 滕永林, 李雪燕, 陈毓麒, 郑双美, 侯明午, 周红照.话题型微博语言特点及其情感分析策略研究.语言文字应用, 2013, (2):135-143 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=yywzyy201302017Hou Min, Teng Yong-Lin, Li Xue-Yan, Chen Yu-Qi, Zheng Shuang-Mei, Hou Ming-Wu, Zhou Hong-Zhao. Study on the linguistic features of the topic-oriented microblog and the strategies for its sentiment analysis. Applied Linguistics, 2013, (2):135-143 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=yywzyy201302017 [48] Wiebe J, Wilson T, Cardie C. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation, 2005, 39(2-3):165-210 doi: 10.1007/s10579-005-7880-9 [49] Toprak C, Jakob N, Gurevych I. Sentence and expression level annotation of opinions in user-generated discourse. In:Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics. Uppsala, Sweden:Association for Computational Linguistics, 2010. 575-584 [50] Farra N, Mckeown K, Habash N. Annotating targets of opinions in Arabic using crowdsourcing. In:Proceedings of the 2nd Workshop on Arabic Natural Language Processing. Beijing, China:Association for Computational Linguistics, 2015. 89-98 [51] Cruz I, Gelbukh A, Sidorov G. Implicit aspect indicator extraction for aspect-based opinion mining. International Journal of Computational Linguistics and Applications, 2014, 5(2):135-152 [52] Xu R, Xia Y, Wong K F, Li W J. Opinion annotation in on-line Chinese product reviews. In:Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC'08). Marrakech, Morocco:European Language Resources Association (ELRA), 2008. 1625-1632 [53] Wang W, Xu H, Huang X Q. Implicit feature detection via a constrained topic model and SVM. In:Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, Washington, USA:Association for Computational Linguistics, 2013. 903-907 [54] Zeng L W, Li F. A classification-based approach for implicit feature identification. In:Proceedings of the 12th China National Conference on Computational Linguistics and the First International Symposium on Natural Language Processing Based on Naturally Annotated Big Data. Suzhou, China:Springer, 2013. 190-202 [55] Su Q, Xiang K, Wang H F, Sun B, Yu S W. Using pointwise mutual information to identify implicit features in customer reviews. In:Proceedings of the 21st International Conference on Computer Processing of Oriental Languages:Beyond the Orient:the Research Challenges Ahead. Singapore, Singapore:Springer, 2006. 22-30 [56] Zhou X J, Wan X J, Xiao J G. Cross-language opinion target extraction in review texts. In:Proceedings of the 12th International Conference on Data Mining. Brussels, Belgium:IEEE, 2012. 1200-1205 [57] Zhai Z W, Liu B, Xu H, Jia P F. Grouping product features using semi-supervised learning with soft-constraints. In:Proceedings of the 23rd International Conference on Computational Linguistics. Beijing, China:Association for Computational Linguistics, 2010. 1272-1280 [58] Zhai Z W, Liu B, Xu H, Jia P F. Clustering product features for opinion mining. In:Proceedings of the 4th ACM International Conference on Web Search and Data Mining. Hong Kong, China:ACM, 2011. 347-354 [59] 杨源, 马云龙, 林鸿飞.评论挖掘中产品属性归类问题研究.中文信息学报, 2012, 26(3):104-109 http://cdmd.cnki.com.cn/Article/CDMD-10055-1013129443.htmYang Yuan, Ma Yun-Long, Lin Hong-Fei. Clustering product features in opinion mining. Journal of Chinese Information Processing, 2012, 26(3):104-109 http://cdmd.cnki.com.cn/Article/CDMD-10055-1013129443.htm [60] Zhang Y, Liu M, Xia H X. Clustering context-dependent opinion target words in Chinese product reviews. Journal of Computer Science and Technology, 2015, 30(5):1109-1119 doi: 10.1007/s11390-015-1586-2