深度学习认知计算综述

陈伟宏; 安吉尧; 李仁发; 李万里

doi:10.16383/j.aas.2017.c160690

深度学习认知计算综述

doi: 10.16383/j.aas.2017.c160690

陈伟宏^1,2,,
安吉尧^1,2, ,,
李仁发^1,2,,
李万里^1,2,

1.
湖南大学信息科学与工程学院长沙 410082
2.
嵌入式与网络计算湖南省重点实验室长沙 410082

基金项目:

国家自然科学基金 61672217

国家自然科学基金 61370097

详细信息

作者简介:
陈伟宏湖南大学信息科学与工程学院博士研究生, 湖南城市学院教授.2006年获得湖南大学硕士学位.主要研究方向为信息物理系统, 分布式计算, 机器学习.E-mail:whchen@hnu.edu.cn

李仁发湖南大学教授, 华中科技大学博士.主要研究方向为嵌入式系统, 信息物理系统, 人工智能与机器视觉.E-mail:lirenfa@hnu.edu.cn

李万里湖南大学信息科学与工程学院博士研究生.2014年获得湖南大学学士学位.主要研究方向为机器学习, 计算机视觉, 智能交通系统和驾驶员行为分析.E-mail:liwanli@hnu.edu.cn

通讯作者:
安吉尧湖南大学教授.2012年获得湖南大学博士学位.主要研究方向为信息物理系统, 并行与分布式计算, 计算智能.本文通信作者.E-mail:anbobcn@aliyun.com

计量
- 文章访问数: 4603
- HTML全文浏览量: 973
- PDF下载量: 3030
- 被引次数: 0
出版历程
- 收稿日期: 2016-10-10
- 录用日期: 2017-06-22
- 刊出日期: 2017-11-20

Review on Deep-learning-based Cognitive Computing

CHEN Wei-Hong^{1,2
,},
AN Ji-Yao^{1,2
, ,},
LI Ren-Fa^{1,2
,},
LI Wan-Li^{1,2
,}

1.
College of Computer Science and Electronic Engineering, Hunan University, Changsha 410082
2.
Key Laboratory for Embedded and Network Computing of Hunan Province, Changsha 410082

Funds:

National Natural Science Foundation of China 61672217

National Natural Science Foundation of China 61370097

More Information

Author Bio:
Ph. D. candidate at the College of Computer Science and Electronic Engineering, Hunan University, and professor at Hunan City University. She received her master degree from Hunan University in 2006. Her research interest covers cyber physical systems, distributed computing, and machine learning

Professor at Hunan University. He received his Ph. D. degree from Huazhong University of Science and Technology. His research interest covers embedded system, cyber-physical systems, artificial intelligence, and machine vision

Ph. D. candidate at the College of Computer Science and Electronic Engineering, Hunan University. He received his bachelor degree from Hunan University in 2014. His research interest covers machine learning, computer vision, intelligent transportation systems, and driver behavior analysis

Corresponding author: AN Ji-Yao Professor at Hunan University. He received his Ph. D. degree from Hunan University in 2012. His research interest covers cyber-physical systems, parallel and distributed computing, and computing intelligence. Corresponding author of this paper

摘要

摘要: 随着大数据和智能时代的到来，机器学习的研究重心已开始从感知领域转移到认知计算（Cognitive computing，CC）领域，如何提升对大规模数据的认知能力已成为智能科学与技术的一大研究热点，最近的深度学习有望开启大数据认知计算领域的研究新热潮.本文总结了近年来大数据环境下基于深度学习的认知计算研究进展，分别从深度学习数据表示、认知模型、深度学习并行计算及其应用等方面进行了前沿概况、比较和分析，对面向大数据的深度学习认知计算的挑战和发展趋势进行了总结、思考与展望.
- 深度学习 /
- 认知计算 /
- 张量数据表示 /
- 并行计算 /
- 大数据
Abstract: With the advent of the era of big data and artificial intelligence, the research focus of machine learning has shifted from perception domain to cognitive computing (CC) domain. How to improve the cognitive ability through big data is becoming a research hotspot of intelligence science and technology, in which recent deep learning has been expected to spark a new wave of research on cognitive computing. This paper summarizes the research progress of cognitive computing based on deep learning in recent years. And, comparison and analysis of recent progress in deep learning and cognitive computing are presented from three aspects, that is, deep learning data representation, cognitive models, parallel computing and its applications in the big data environment. Finally, some challenges and development trends of cognitive computing based on deep learning for big data are investigated to for cast the future research.
- Deep learning (DL) /
- cognitive computing (CC) /
- tensor data representation /
- parallel computing /
- big data
注释:

1) 本文责任编委张化光

HTML全文

图 1 深度学习认知计算示意图

Fig. 1 Diagram of deep learning cognitive computing

下载: 全尺寸图片幻灯片

图 2 基于张量的数据表示和处理框架

Fig. 2 Tensor-based data representation and processing framework

下载: 全尺寸图片幻灯片

图 3 深度学习一般模型

Fig. 3 The deep learning cognitive model

下载: 全尺寸图片幻灯片

图 4 深度卷积网模型LeNet

Fig. 4 Deep convolution network model of LeNet

下载: 全尺寸图片幻灯片

图 5 自编码器结构

Fig. 5 The structure of the auto-encoder

下载: 全尺寸图片幻灯片

图 6 RNN结构

Fig. 6 The structure of RNN

下载: 全尺寸图片幻灯片

图 7 基于GPU的数据并行和模型并行混合架构

Fig. 7 The hybrid architecture based on data parallel and model parallel

下载: 全尺寸图片幻灯片

参考文献(85)

[1]	Zorzi M, Zanella A, Testolin A, De Filippo, De Grazia M, Zorzi M. Cognition-based networks:a new perspective on network optimization using learning and distributed intelligence. IEEE Access, 2015, 3:1512-1530 doi: 10.1109/ACCESS.2015.2471178
[2]	Chen Y, Argentinis J, Weber G. IBM Watson:How cognitive computing can be applied to big data challenges in life sciences research. Clinical Therapeutics, 2016, 38(4):688-701 doi: 10.1016/j.clinthera.2015.12.001
[3]	Kelly J. Computing, cognition and the future of knowing[Online], available:https://www.research.ibm.com/soft-ware/IBMResearch/multimedia/Computing_Cognition_Wh-itePaper.pdf, June 2, 2017
[4]	余凯, 贾磊, 陈雨强, 徐伟.深度学习的昨天, 今天和明天.计算机研究与发展, 2013, 50(9):1799-1804 doi: 10.7544/issn1000-1239.2013.20131180 Yu Kai, Jia Lei, Chen Yu-Qiang, Xu Wei. Deep learning:yesterday, today, tomorrow. Journal of Computer Research and Development, 2013, 50(9):1799-1804 doi: 10.7544/issn1000-1239.2013.20131180
[5]	LeCun Y, Bengio Y, Hinton G. Deep learning. Nature, 2015, 521(7553):436-444 doi: 10.1038/nature14539
[6]	Schmidhuber J. Deep learning in neural networks:an overview. Neural Networks, 2015, 61:85-117 doi: 10.1016/j.neunet.2014.09.003
[7]	Wang F Y, Zhang J J, Zheng X H, Wang X, Yuan Y, Dai X X, Zhang J, Yang L Q. Where does AlphaGo go:from Church-Turing thesis to AlphaGo thesis and beyond. IEEE/CAA Journal of Automatica Sinica, 2016, 3(2):113-120 doi: 10.1109/JAS.2016.7471613
[8]	Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks. Science, 2006, 313(5786):504-507 doi: 10.1126/science.1127647
[9]	Ludwig L. Toward an integral cognitive theory of memory and technology. Extended Artificial Memory. Kaiserslautern:Technical University of Kaiserslautern, 2013. 16-32
[10]	Silver D, Huang A, Maddison C J, Guez A, Sifre L, van den Driessche G, Schrittwieser J, Antonoglou I, Panneershelvam V, Lanctot M, Dieleman S, Grewe D, Nham J, Kalchbrenner N, Sutskever I, Lillicrap T, Leach M, Kavukcuoglu K, Graepel T, Hassabis D. Mastering the game of Go with deep neural networks and tree search. Nature, 2016, 529(7587):484-489 doi: 10.1038/nature16961
[11]	Ji S W, Xu W, Yang M, Yu K. 3D convolutional neural networks for human action recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1):221-231 doi: 10.1109/TPAMI.2012.59
[12]	Lake B M, Salakhutdinov R, Tenenbaum J B. Human-level concept learning through probabilistic program induction. Science, 2015, 350(6266):1332-1338 doi: 10.1126/science.aab3050
[13]	孙旭, 李晓光, 李嘉锋, 卓力.基于深度学习的图像超分辨率复原研究进展.自动化学报, 2017, 43(5):697-709 http://www.aas.net.cn/CN/abstract/abstract19048.shtml Sun Xu, Li Xiao-Guang, Li Jia-Feng, Zhuo Li. Review on deep learning based image super-resolution restoration algorithms. Acta Automatica Sinica, 2017, 43(5):697-709 http://www.aas.net.cn/CN/abstract/abstract19048.shtml
[14]	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In:Proceedings of the 2012 Advances in Neural Information Processing Systems. Lake Tahoe, Nevada, US:NIPS, 2012. 1097-1105 http://dl.acm.org/citation.cfm?id=3065386
[15]	Tompson J, Jain A, LeCun Y, Bregler C. Joint training of a convolutional network and a graphical model for human pose estimation. In:Proceedings of the 2014 Advances in Neural Information Processing Systems. Montreal, Quebec, Canada:NYU, 2014. 1799-1807 http://www.oalib.com/paper/4082161
[16]	Yu D, Deng L. Automatic Speech Recognition:a Deep Learning Approach. New York:Springer, 2015. 58-80
[17]	Hinton G E, Deng L, Yu D, Dahl G, Mohamed A, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Kingsbury B, Sainath T. Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Processing Magazine, 2012, 29(6):82-97 doi: 10.1109/MSP.2012.2205597
[18]	Bordes A, Chopra S, Weston J. Question answering with subgraph embeddings[Online], available:http://arxiv.org/abs/1406.3676v3, September 4, 2014
[19]	Ma J S, Sheridan R P, Liaw A, Dahl G E, Svetnik V. Deep neural nets as a method for quantitative structure-activity relationships. Journal of Chemical Information and Modeling, 2015, 55(2):263-274 doi: 10.1021/ci500747n
[20]	He Y H, Xiang S M, Kang C C, Wang J, Pan C H. Cross-modal retrieval via deep and bidirectional representation learning. IEEE Transactions on Multimedia, 2016, 18(7):1363-1377 doi: 10.1109/TMM.2016.2558463
[21]	Jean S, Cho K, Memisevic R, Bengio Y. On using very large target vocabulary for neural machine translation[Online], available:https://arxiv.org/abs/1412.2007, March 18, 2015
[22]	Chen Y F, Li Y, Narayan R, Subramanian A, Xie X H. Gene expression inference with deep learning. Bioinformatics, 2016, 32(12):1832-1839 doi: 10.1093/bioinformatics/btw074
[23]	Leung M K K, Xiong H Y, Lee L J, Frey B J. Deep learning of the tissue regulated splicing code. Bioinformatics, 2014, 30(12):i121-i129 doi: 10.1093/bioinformatics/btu277
[24]	Xiong H Y, Alipanahi B, Lee L J, Bretschneider H, Merico D, Yuen R K C, Hua Y, Gueroussov S, Najafabadi H S, Hughes T R, Morris Q, Barash Y, Krainer A R, Jojic N, Scherer S W, Blencowe B J, Frey B J. The human splicing code reveals new insights into the genetic determinants of disease. Science, 2015, 347(6218):1254806 doi: 10.1126/science.1254806
[25]	Bengio Y, Courville A, Vincent P. Representation learning:a review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8):1798-1828 doi: 10.1109/TPAMI.2013.50
[26]	Zheng Y, Capra L, Wolfson O, Yang H. Urban computing:concepts, methodologies, and applications. ACM Transactions on Intelligent Systems and Technology, 2014, 5(3):1-18 https://uic.pure.elsevier.com/en/publications/urban-computing-concepts-methodologies-and-applications
[27]	Zheng Y. Methodologies for cross-domain data fusion:an overview. IEEE Transactions on Big Data, 2015, 1(1):16-34 doi: 10.1109/TBDATA.2015.2465959
[28]	Ngiam J, Khosla A, Kim M, Nam J, Lee H, Ng A Y. Multimodal deep learning. In:Proceedings of the 28th International Conference on Machine Learning. Bellevue, Washington, USA:ICML, 2011. 689-696 http://www.researchgate.net/publication/221345149_Multimo
[29]	Srivastava N, Salakhutdinov R R. Multimodal learning with deep boltzmann machines. In:Proceedings of the 2012 Advances in Neural Information Processing Systems. Lake Tahoe, Nevada, US:NIPS, 2012. 2222-2230
[30]	Cichocki A. Era of big data processing:a new approach via tensor networks and tensor decompositions[Online], available:https://arxiv.org/abs/1403.2048, March 9, 2014
[31]	Kuang L W, Hao F, Yang L T, Lin M, Luo C Q, Min G. A tensor-based approach for big data representation and dimensionality reduction. IEEE Transactions on Emerging Topics in Computing, 2014, 2(3):280-291 doi: 10.1109/TETC.2014.2330516
[32]	Zhang Q C, Yang L T, Chen Z K. Deep computation model for unsupervised feature fearning on big data. IEEE Transactions on Service Computing, 2014, 9(1):161-171 https://www.researchgate.net/publication/283767977_Deep_Computation_Model_for_Unsupervised_Feature_Learning_on_Big_Data
[33]	Cohen N, Sharir O, Shashua A. On the expressive power of deep learning:a tensor analysis[Online], available:https://arxiv.org/abs/1509.05009, October 30, 2017
[34]	Deng L, Yu D. Deep Learning for Signal and Information Processing. Delft:Microsoft Research Monograph, 2013. 29-48 https://www.microsoft.com/en-us/research/publication/deep-learning-for-signal-and-information-processing/?from=http%3A%2F%2Fresearch.microsoft.com%2Fapps%2Fpubs%2Fdefault.aspx%3Fid%3D192936
[35]	Li L Z, Boulware D. High-order tensor decomposition for large-scale data analysis. In:Proceedings of the 2015 IEEE International Congress on Big Data. Washington, DC, USA:IEEE, 2015. 665-668 http://ieeexplore.ieee.org/document/7207288/
[36]	Zhou G, Cichocki A, Xie S. Decomposition of big tensors with low multilinearrank[Online], available:https://arxiv.org/pdf/1412.1885.pdf, December 29, 2014
[37]	Jang J-S R. ANFIS:adaptive-network-based fuzzy inference system. IEEE Transactions on Systems, Man, Cybernetics, 1993, 23(3):665-685 doi: 10.1109/21.256541
[38]	Wang F Y, Kim H. Implementing adaptive fuzzy logic controllers with neural networks:a design paradigm. Journal of Intelligent and Fuzzy Systems, 1995, 3(2):165-180 https://www.sciencedirect.com/science/article/pii/S0165011498000980
[39]	Wang G, Shi H. TMLNN:triple-valued or multiple-valued logic neural network. IEEE Transactions on Neural Networks, 1998, 9(6):1099-1117 doi: 10.1109/72.728355
[40]	Rosca M, Lakshminarayanan B, Warde-Farley D, Mohamed S. Variational approaches for auto-encoding generative adversarial networks[Online], available:https://arxiv.org/abs/1706.04987, June 15, 2017
[41]	Lv Y, Duan Y, Kang W, Li Z, Wang F. Traffic flow prediction with big data:a deep learning approach. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(2):865-873 http://ieeexplore.ieee.org/document/6894591/
[42]	Goodfellow I, Courville A, Bengio Y. Scaling up spike-and-slab models for unsupervised feature learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8):1902-1914 doi: 10.1109/TPAMI.2012.273
[43]	Chang C H. Deep and shallow architecture of multilayer neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(10):2477-2486 doi: 10.1109/TNNLS.2014.2387439
[44]	Xie K, Wang X, Liu X L, Wen J G, Cao J N. Interference-aware cooperative communication in multi-radio multi-channel wireless networks. IEEE Transactions on Computers, 2016, 65(5):1528-1542 doi: 10.1109/TC.2015.2448089
[45]	Salakhutdinov R. Learning Deep Generative Models[Ph.D. dissertation], University of Toronto, Toronto, Canada, 2009. doi: 10.1146/annurev-statistics-010814-020120
[46]	Wu X D, Yu K, Ding W, Wang H, Zhu X Q. Online feature selection with streaming features. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(5):1178-1192 doi: 10.1109/TPAMI.2012.197
[47]	Yang L T, Kuang L W, Chen J J, Hao F, Luo C Q. A holistic approach to distributed dimensionality reduction of big data. IEEE Transactions on Cloud Computing, 2015, PP(99):1-14 http://ieeexplore.ieee.org/xpl/abstractAuthors.jsp?reload=true&arnumber=7134729
[48]	Bengio Y, Yao L, Alain G, Vincent P. Generalized denoising auto-encoders as generative models. In:Proceedings of the 26th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, USA:ACM, 2013. 899-907 http://www.oalib.com/paper/4038830
[49]	朱煜, 赵江坤, 王逸宁, 郑兵兵.基于深度学习的人体行为识别算法综述.自动化学报, 2016, 42(6):848-857 http://www.aas.net.cn/CN/abstract/abstract18875.shtml Zhu Yu, Zhao Jiang-Kun, Wang Yi-Ning, Zheng Bing-Bing. A review of human action recognition based on deep learning. Acta Automatica Sinica, 2016, 42(6):848-857 http://www.aas.net.cn/CN/abstract/abstract18875.shtml
[50]	Visin F, Kastner K, Cho K, Matteucci M, Courville A, Bengio Y. ReNet:a recurrent neural network based alternative to convolutional networks[Online], available:https://arxiv.org/abs/1505.00393, July 23, 2015
[51]	Weninger F, Bergmann J, Schuller B W. Introducing CURRENNT:the munich open-source CUDA recurrent neural network toolkit. Journal of Machine Learning Research, 2015, 16(3):547-551 https://hgpu.org/?p=12965
[52]	Wang F, Tax David M J. Survey on the attention based RNN model and its applications in computer vision[Online], available:https://arxiv.org/abs/1601.06823, January 25, 2016
[53]	段艳杰, 吕宜生, 张杰, 赵学亮, 王飞跃.深度学习在控制领域的研究现状与展望.自动化学报, 2016, 42(5):643-654 http://www.aas.net.cn/CN/abstract/abstract18852.shtml Duan Yan-Jie, Lv Yi-Sheng, Zhang Jie, Zhao Xue-Liang, Wang Fei-Yue. Deep learning for control:the state of the art and prospects. Acta Automatica Sinica, 2016, 42(5):643-654 http://www.aas.net.cn/CN/abstract/abstract18852.shtml
[54]	Yu D, Deng L, Seide F. The deep tensor neural network with applications to large vocabulary speech recognition. IEEE Transactions on Audio, Speech, and Language Processing, 2013, 21(2):388-396 doi: 10.1109/TASL.2012.2227738
[55]	Palzer D, Hutchinson B. The tensor deep stacking network toolkit. In:Proceedings of the 2015 International Joint Conference on Neural Networks. Killarney, Ireland:IEEE, 2015. 1-5 http://ieeexplore.ieee.org/xpls/icp.jsp?arnumber=7280297
[56]	Deng L, Yu D. Deep learning:methods and applications. Foundations and Trends in Signal Processing, 2014, 7(3-4):197-387 doi: 10.1561/2000000039
[57]	Zhang Q, Yang L T, Chen Z. Deep computation model for unsupervised feature learning on big data. IEEE Transactions on Services Computing, 2016, 9(1):161-171 http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=7317793&filter%3DAND%28p_IS_Number%3A7393924%29
[58]	Hutchinson B, Deng L, Yu D. Tensor deep stacking networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8):1944-1957 doi: 10.1109/TPAMI.2012.268
[59]	Shan S L, Khalil-Hani M, Bakhteri R. Bounded activation functions for enhanced training stability of deep neural networks on visual pattern recognition problems. Neurocomputing, 2016, 216:718-734 doi: 10.1016/j.neucom.2016.08.037
[60]	Chandra B, Sharma Rajesh K. Fast learning in deep neural networks. Neurocomputing, 2016, 171:1205-1215 doi: 10.1016/j.neucom.2015.07.093
[61]	Neyshabur B, Salakhutdinov R R, Srebro N. Path-SGD:path-normalized optimization in deep neural networks. In:Proceedings of the 2015 Advances in Neural Information Processing Systems. Montreal, Canada:Cornell University Library, 2015. 2422-2430 http://dl.acm.org/citation.cfm?id=2969510
[62]	Burda Y, Grosse R, Salakhutdinov R. Importance weighted autoencoders[Online], available:https://arxiv.org/abs/1509.00519, November 7, 2016
[63]	Le Q V, Jaitly N, Hinton G E. A simple way to initialize recurrent networks of rectified linear units[Online], available:https://arxiv.org/abs/1504.00941, April 7, 2015
[64]	Sutskever I, Martens J, Dahl G, Hinton G. On the importance of momentum and initialization in deep learning. In:Proceedings of the 30th International Conference on International Conference on Machine Learning. Atlanta, GA, USA:ACM, 2013. Ⅲ-1139-Ⅲ-1147
[65]	Kingma D P, Ba J L. Adam:a method for stochastic optimization. In:Proceedings of the 3rd International Conference for Learning Representations. San Diego, CA, USA:Cornell University Library, 2015. 1-15 http://arxiv.org/abs/1412.6980
[66]	Martens J, Grosse R B. Optimizing neural networks with kronecker-factored approximate curvature. In:Proceedings of the 2015 Neural and Evolutionary Computing. San Diego, CA, USA:Cornell University Library, 2015. 2408-2417 http://dl.acm.org/citation.cfm?id=3045374
[67]	Shinozaki T, Watanabe S. Structure discovery of deep neural network based on evolutionary algorithms. In:Proceedings of the 2015 IEEE International Conference on Acoustics, Speech, and Signal Processing. Brisbane, Australia:IEEE, 2015. 4979-4983 http://ieeexplore.ieee.org/document/7178918/
[68]	Jaitly N, Hinton G. Learning a better representation of speech soundwaves using restricted boltzmann machines. In:Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Prague, Czech Republic:IEEE, 2011. 5884-5887 http://ieeexplore.ieee.org/document/5947700/
[69]	Dahl G E, Sainath T N, Hinton G E. Improving deep neural networks for LVCSR using rectified linear units and dropout. In:Proceedings of the 2013 IEEE International Conference on Acoustics, Speech, and Signal Processing. Vancouver, BC, Canada:IEEE, 2013. 8609-8613 http://ieeexplore.ieee.org/document/6639346/
[70]	Maas A L, Hannun A Y, Ng A Y. Rectifier nonlinearities improve neural network acoustic models. In:Proceedings of the 30th International Conference on Machine Learning. Atlanta, USA:IEEE, 2013. 1-6 http://www.mendeley.com/catalog/rectifier-nonlinearities-improve-neural-network-acoustic-models/
[71]	Miao Y J, Metze F, Rawat S. Deep maxout networks for low-resource speech recognition. In:Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. Olomouc, Czech Republic:IEEE, 2013. 398-403 http://ieeexplore.ieee.org/abstract/document/6707763/
[72]	Zhang X H, Trmal J, Povey D, Khudanpur S. Improving deep neural network acoustic models using generalized maxout networks. In:Proceedings of the 2014 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP). Florence, Italy:IEEE, 2014. 215-219 http://ieeexplore.ieee.org/document/6853589/
[73]	Srivastava N, Hinton G E, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout:a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 2014, 15(1):1929-1958 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.669.8604
[74]	陈国良, 毛睿, 陆克中.大数据并行计算框架.科学通报, 2015, 60(5-6):566-569 http://d.wanfangdata.com.cn/Periodical/wxhlkj201701020 Chen Guo-Liang, Mao Rui, Lu Ke-Zhong. Parallel computing framework of big data. Chinese Science Bulletin, 2015, 60(5-6):566-569 http://d.wanfangdata.com.cn/Periodical/wxhlkj201701020
[75]	Chetlur S, Woolley C, Vandermersch P, Cohen J, Tran J, Catanzaro B, Shelhamer E. cuDNN:efficient primitives for deep learning[Online], available:https://arxiv.org/abs/1410.0759, December 18, 2014
[76]	Dean J, Corrado G, Monga R, Chen K, Devin M, Mao M, Ranzato M, Senior A, Tuker P, Yang K, Le Q V, Ng A Y. Large scale distributed deep networks. In:Proceedings of the 2012 Neural Information Processing Systems. Lake Tahoe, Nevada, USA:NIPS, 2012. 1223-1231
[77]	Jia Y Q, Shelhamer E, Donahue J, Darrel T. Caffe:convolutional architecture for fast feature embedding. In:Proceedings of the 22nd ACM International Conference on Multimedia. Orlando, Florida, USA:ACM, 2014. 675-678 http://dl.acm.org/citation.cfm?id=2654889
[78]	Coates A, Huval B, Wang T, Wu D, Catanzaro B, Andrew N. Deep learning with COTS HPC systems. In:Proceedings of the 30th International Conference on Machine Learning. Atlanta, Georgia, USA:ICML, 2013. 16-21 http://dl.acm.org/citation.cfm?id=3042817.3043086
[79]	Yadan O, Adams K, Taigman Y, Ranzato M. Multi-GPU training of convNets[Online], available:https://arxiv.org/abs/1312.5853, February 18, 2013
[80]	Yu K. Large-scale deep learning at Baidu. In:Proceedings of the 22nd ACM International Conference on Information and Knowledge Management. San Francisco, CA, USA:ACM, 2013. 2211-2212 http://dl.acm.org/citation.cfm?id=2514699
[81]	Chen Y J, Luo T, Liu S L, Zhang S J, He L Q, Wang J, Li L, Chen T S, Xu Z W, Sun N H, Temam O. DaDianNao:A machine-learning supercomputer. In:Proceedings of the 47th IEEE/ACM International Symposium on Microarchitecture. Cambridge, United Kingdom:IEEE, 2014. http://dl.acm.org/citation.cfm?id=2742217
[82]	Luo T, Liu S L, Li L, Wang Y Q, Zhang S J, Chen T S, Xu Z W, Temam O, Chen Y J. DaDianNao:A neural network supercomputer. IEEE Transactions on Computers, 2017, 66(1):73-86 doi: 10.1109/TC.2016.2574353
[83]	Markoff J. How many computers to identify a cat? 16, 000. New York Times, 2012. 6-25
[84]	He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In:Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, Nevada, USA:IEEE, 2016. 770-778 http://ieeexplore.ieee.org/document/7780459/
[85]	Ren M Y, Kiros R, Zemel R S. Exploring models and data for image question answering. In:Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Canada:MIT, 2015. 2953-2961 http://dl.acm.org/citation.cfm?id=2969570

施引文献

资源附件(0)

访问统计

图(7)

计量

文章访问数: 4603
HTML全文浏览量: 973
PDF下载量: 3030
被引次数: 0

姓名
邮箱
手机号码
标题
留言内容
验证码

留言板

深度学习认知计算综述

doi: 10.16383/j.aas.2017.c160690

通讯作者:
安吉尧湖南大学教授.2012年获得湖南大学博士学位.主要研究方向为信息物理系统, 并行与分布式计算, 计算智能.本文通信作者.E-mail:anbobcn@aliyun.com

计量

Review on Deep-learning-based Cognitive Computing

Corresponding author: AN Ji-Yao Professor at Hunan University. He received his Ph. D. degree from Hunan University in 2012. His research interest covers cyber-physical systems, parallel and distributed computing, and computing intelligence. Corresponding author of this paper

计量

目录

留言板

深度学习认知计算综述

doi: 10.16383/j.aas.2017.c160690

通讯作者: 安吉尧 湖南大学教授.2012年获得湖南大学博士学位.主要研究方向为信息物理系统, 并行与分布式计算, 计算智能.本文通信作者.E-mail:anbobcn@aliyun.com

计量

出版历程

Review on Deep-learning-based Cognitive Computing

Corresponding author: AN Ji-Yao Professor at Hunan University. He received his Ph. D. degree from Hunan University in 2012. His research interest covers cyber-physical systems, parallel and distributed computing, and computing intelligence. Corresponding author of this paper

计量

出版历程

目录

通讯作者:
安吉尧湖南大学教授.2012年获得湖南大学博士学位.主要研究方向为信息物理系统, 并行与分布式计算, 计算智能.本文通信作者.E-mail:anbobcn@aliyun.com