基于深度学习的表面缺陷检测方法综述

陶显; 侯伟; 徐德

doi:10.16383/j.aas.c190811

基于深度学习的表面缺陷检测方法综述

doi: 10.16383/j.aas.c190811

陶显^1,,
侯伟^{1, 2,},
徐德^{1, 2,}

1.
中国科学院自动化研究所精密感知与控制研究中心北京 100190
2.
中国科学院大学人工智能学院北京 101408

基金项目: 国家自然科学基金(61703399, 61703398, 61973302, 61673383)资助

详细信息

作者简介:
陶显：中国科学院自动化研究所副研究员. 2016年获得中国科学院自动化研究所博士学位. 主要研究方向为机器视觉, 缺陷检测和深度学习. 本文通信作者.E-mail: taoxian2013@ia.ac.cn

侯伟：中国科学院大学人工智能学院博士研究生. 2009年和2014年分别获得兰州大学学士和硕士学位. 主要研究方向为缺陷检测, 计算机视觉, 图像处理和机器学习.E-mail: houwei2018@ia.ac.cn

徐德：中国科学院自动化研究所研究员. 1985年和1990年分别获得山东工业大学学士和硕士学位. 2001年获得浙江大学博士学位. 主要研究方向为机器人视觉测量, 视觉控制, 智能控制, 视觉定位, 显微视觉, 微装配.E-mail: de.xu@ia.ac.cn

计量
- 文章访问数: 12853
- HTML全文浏览量: 9418
- PDF下载量: 4916
- 被引次数: 0
出版历程
- 收稿日期: 2019-11-27
- 录用日期: 2020-03-25
- 网络出版日期: 2021-03-04
- 刊出日期: 2021-05-20

A Survey of Surface Defect Detection Methods Based on Deep Learning

TAO Xian^1
,,
HOU Wei^{1, 2
,},
XU De^{1, 2
,}

1.
Research Center of Precision Sensing and Control, Institute of Automation, Chinese Academy of Sciences, Beijing 100190
2.
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 101408

Funds: Supported by National Natural Science Foundation of China (61703399, 61703398, 61973302, 61673383)

More Information

Author Bio:
TAO Xian　Associate professor at the Institute of Automation, Chinese Academy of Sciences. He received his Ph.D. degree at the Institute of Automation, Chinese Academy of Sciences in 2016. His research interest covers machine vision, defect detection, and deep learning. Corresponding author of this paper

HOU Wei　Ph.D. candidate at the School of Artificial Intelligence, University of Chinese Academy of Science. He received his bachelor degree and master degree from Lanzhou University in 2009 and 2014, respectively. His research interest covers defect detection, computer vision, image processing, and machine learning

XU De　Professor at the Institute of Automation, Chinese Academy of Sciences. He received his bachelor degree and master degree from Shandong University of Technology in 1985 and 1990, respectively, and received his Ph.D. degree from Zhejiang University in 2001. His research interest covers robotics and automation such as visual measurement, visual control, intelligent control, visual positioning, microscopic vision, and microassembly

摘要

摘要:
近年来, 基于深度学习的表面缺陷检测技术广泛应用在各种工业场景中. 本文对近年来基于深度学习的表面缺陷检测方法进行了梳理, 根据数据标签的不同将其分为全监督学习模型方法、无监督学习模型方法和其他方法三大类, 并对各种典型方法进一步细分归类和对比分析, 总结了每种方法的优缺点和应用场景. 本文探讨了表面缺陷检测中三个关键问题, 介绍了工业表面缺陷常用数据集. 最后, 对表面缺陷检测的未来发展趋势进行了展望.
- 深度学习 /
- 表面缺陷检测 /
- 机器视觉 /
- 卷积神经网络
Abstract:
In recent years, surface defect detection techniques based on deep learning have been widely used in various industrial scenarios. This paper reviews the latest works on deep learning based surface defect detection methods. They are classified into three categories: full-supervised learning model method, unsupervised learning model method and other methods. The typical methods are further subdivided and compared. The advantages and disadvantages of these methods and their application scenarios are summarized. This paper analyzes three key issues in surface defect detection and introduces common data sets for industrial surface defects. Finally, the future development trend of surface defect detection is predicted.
- Deep learning /
- surface defect detection /
- machine vision /
- convolutional neural network (CNN)

HTML全文

图 1 复杂工业环境下的表面缺陷图像

Fig. 1 Images of surface defects in complex industrial environment ((a) Scratch image of dark field image of optical component; (b) Surface defect of building bridge; (c) Strip surface defect; (d) Unmanned aerial vehicle insulator defect)

下载: 全尺寸图片幻灯片

图 2 缺陷检测的问题定义

Fig. 2 Definition of defect detection problem

下载: 全尺寸图片幻灯片

图 3 缺陷检测方法框架图

Fig. 3 Framework of defect detection methods

下载: 全尺寸图片幻灯片

图 4 基于滑动窗口的裂纹定位

Fig. 4 Crack location based on sliding window

下载: 全尺寸图片幻灯片

表 1 商用基于深度学习的缺陷检测软件

Table 1 Commercial deep learning based defect detection software

软件名称	公司	国家	年份
VIDI	已被康耐视收购	瑞士	2016
Halcon17 以上版本	MVTec	德国	2017
SuaKIT 数优−AI 深度学习缺陷检测软件	已被康耐视收购	韩国	2017
ALFA 深度学习外观缺陷检测软件	东莞埃法智能科技有限公司	中国	2018
AiDitron 人工智能软件	杭州谛创科技有限公司	中国	2018

下载: 导出CSV

表 2 分类网络各子方法优缺点对比

Table 2 Comparison of advantages and disadvantages of each sub-method of classification network

代表子方法	优点	缺点
直接分类	结构经典, 也是其他分类网络子方法的基础, 可参考诸多现成网络	缺陷在图像中需要占一定比例, 否则其特征容易被池化掉, 同时一般一幅图像中只容许存在一种类别的缺陷 (多标签分类除外)
定位 ROI 后分类	获取 ROI 的缺陷信息	需借助其他方法获取 ROI
多类别分类	一定程度上解决样本不平衡问题	网络采用二级训练
滑动窗口	在大图中实现缺陷的粗定位	滑动窗口尺寸需要准确选择, 且只能获得较粗位置, 遍历滑动速度慢
热力图	得到较为精准的缺陷区域	缺陷精确定位效果依赖网络分类性能
多任务学习	联合其他网络同时获取缺陷精确位置和类别, 也能减少所需训练样本数目	网络结构相对复杂, 在添加分割分支时, 需要逐像素的标签
做特征提取器	获取有效的缺陷特征	依赖其他分类器才能获得最终分类结果

下载: 导出CSV

表 3 传统图像处理与基于深度学习的缺陷检测方法的比较

Table 3 Comparison between traditional image processing and deep learning-based defect detection methods

对比项目	传统基于图像处理的方法	深度学习方法
方法	1) 结构法: 边缘、骨架、形态学等	基于卷积神经网络 CNN
	2) 统计法: 直方图、局部二值化特征 LBP、纹理特征、灰度共生矩阵 GLCM 等
	3) 滤波法: 空间滤波、频域滤波 (傅里叶、gabor、小波) 等
	4) 模型法: 随机场模型、反散射模型和分形体等
本质	人工设计特征 + 分类器 (或规则)	从大量数据中自动学习特征
所需条件	相对苛刻的成像环境要求, 缺陷和非缺陷区域之间的高对比度, 少噪声	足够的学习数据和高性能运算单元
适应性	差 (成像环境变化或缺陷类型变化时往往需要更改阈值或重新设计算法)	相对强 (能够应对一定的工业检测环境变化)

下载: 导出CSV

表 4 工业表面缺陷检测常用数据集

Table 4 Common data sets for industrial surface defect detection

方法	应用场景	数据集名称	链接
分类	钢材表面	NEU-CLS^[51]	http://faculty.neu.edu.cn/yunhyan/NEU_surface_defect_database.html
	太阳能板	elpv-dataset^[109]	https://github.com/zae-bayern/elpv-dataset
	金属表面	KolektorSDD^[40]	http://www.vicos.si/Downloads/KolektorSDD
	木材表面	wood defect database^[137]	http://www.ee.oulu.fi/olli/Projects/Lumber.Grading.html
定位	钢材表面	NEU-DET^[51]	http://faculty.neu.edu.cn/yunhyan/NEU_surface_defect_database.html
定位	铸件X射线图像	GDXray Casting^[138]	https://domingomery.ing.puc.cl/material/gdxray/
分割	磁瓦表面	Magnetic-tile-defect-datasets^[67]	https://github.com/abin24/Magnetic-tile-defect-datasets.
	钢轨表面	RSDDs dataset^[139]	http://icn.bjtu.edu.cn/Visint/resources/RSDDs.aspx
	地面裂纹	Crack_Dataset^[140]	https://drive.google.com/drive/folders/1cplcUBmgHfD82YQTWnn1dssK2Z_xRpjx
	桥梁裂缝	Bridge Cracks^[141]	https://github.com/maweifei/BridgeCrack_Image_Data
孪生网络	PCB 板	PCB Dataset^[90]	https://github.com/tangsanli5201/DeepPCB
无监督学习	多种材质缺陷	MVTec AD^[142]	http://www.mvtec.com/company/research/datasets
无监督学习	扫描隧道显微镜成像 SEM 材料表面	NanoTWICE^[143]	http://www.mi.imati.cnr.it/ettore/NanoTWICE/
弱监督学习	纹理缺陷	DAGM 2007^[144]	https://hci.iwr.uni-heidelberg.de/node/3616

下载: 导出CSV

表 5 典型缺陷分类方法在DAGM数据集上性能比较

Table 5 Effect comparison of defect classification methods on DAGM

方法	基础网络描述	是否预训练	分类准确率 (%)	发表时间
Weimer^[145]	滑动窗口分类网络, 窗口大小 32×32	否	99.2	2016
Wang^[146]	基于滑动窗口和多类别分类相结合的网络, 128×128大小的滑动窗口, 搭建的11层 CNN 网络	否	99.8	2017
Wu^[147]	基于多尺度滑动窗口和分类网络, 8 层网络, 包括 inception 结构	否	98.6	2017
Yu^[65]	两阶段 FCN 网络	是	95.9934	2017
Racki^[39]	多任务分类网络, 包括分类分支和分割分支	否	99.655	2018

下载: 导出CSV

表 6 典型缺陷定位方法在GDXray casting数据集上性能比较

Table 6 Effect comparison of defect location methods on GDXray casting

方法	mAP	使用GPU评估的时间/单幅图像 (s)
滑动窗口	0.461	0.231
Faster R-CNN (VGG-16)^[43]	0.865	0.438
Faster R-CNN (ResNet-101)^[43]	0.921	0.512
R-FCN (ResNet-101)^[148]	0.875	0.375
SSD (VGG-16)^[44]	0.697	0.025
SSD (ResNet-101)^[44]	0.762	0.051

下载: 导出CSV

参考文献(148)

[1]	Jing J F, Liu S M, Li P F, Zhang L. The fabric defect detection based on CIE Lab* color space using 2-D Gabor filter. The Journal of the Textile Institute, 2016, 107(10): 1305−1313 doi: 10.1080/00405000.2015.1102458
[2]	Chen T J, Wang Y N, Xiao C Y, Wu Q M J. A machine vision apparatus and method for can-end inspection. IEEE Transactions on Instrumentation and Measurement, 2016, 65(9): 2055−2066 doi: 10.1109/TIM.2016.2566442
[3]	Tao X, Zhang Z T, Zhang F, Xu D. A novel and effective surface flaw inspection instrument for large-aperture optical elements. IEEE Transactions on Instrumentation and Measurement, 2015, 64(9): 2530−2540 doi: 10.1109/TIM.2015.2415092
[4]	中国产业信息网. 随着现代工业自动化技术日趋成熟到2020年全球机器视觉行业市场规模将达到125亿美元2025年将超过192亿美元. [Online], available: http://www.chyxx.com/industry/201901/705852.html, October 20, 2019 China Industry Information Network. With the maturity of modern industrial automation technology, the global machine vision industry market will reach US$ 12.5 billion by 2020, and it will exceed US$ 19.2 billion by 2025. [Online], available: http://www.chyxx.com/industry/201901/705852.html, October 20, 2019
[5]	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, USA: Curran Associates Inc., 2012. 1097−1105
[6]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409.1556, 2014.
[7]	Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015. 1−9
[8]	He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 770−778
[9]	Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 4700−4708
[10]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 7132−7141
[11]	Zhang X Y, Zhou X Y, Lin M X, Sun J. ShuffleNet: An extremely efficient convolutional neural network for mobile devices. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 6848−6856
[12]	Howard A G, Zhu M L, Chen B, Kalenichenko D, Wang W J, Weyand T, et al. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv: 1704.04861, 2017.
[13]	Soukup D, Huber-Mörk R. Convolutional neural networks for steel surface defect detection from photometric stereo images. In: Proceedings of the 10th International Symposium on Visual Computing. Las Vegas, USA: Springer, 2014. 668−677
[14]	Park J K, Kwon B K, Park J H, Kang D J. Machine learning-based imaging system for surface defect inspection. International Journal of Precision Engineering and Manufacturing-Green Technology, 2016, 3(3): 303−310 doi: 10.1007/s40684-016-0039-x
[15]	Kyeong K, Kim H. Classification of mixed-type defect patterns in wafer bin maps using convolutional neural networks. IEEE Transactions on Semiconductor Manufacturing, 2018, 31(3): 395−402 doi: 10.1109/TSM.2018.2841416
[16]	Deitsch S, Christlein V, Berger S, Buerhop-Lutz C, Maier A, Gallwitz F, et al. Automatic classification of defective photovoltaic module cells in electroluminescence images. Solar Energy, 2019, 185: 455−468 doi: 10.1016/j.solener.2019.02.067
[17]	Alcantarilla P F, Bartoli A, Davison A J. KAZE features. In: Proceedings of the 12th European Conference on Computer Vision. Florence, Italy: Springer, 2012. 214−227
[18]	Lowe D G. Object recognition from local scale-invariant features. In: Proceedings of the 7th IEEE International Conference on Computer Vision. Kerkyra, Greece: IEEE, 1999. 1150−1157
[19]	Bay H, Ess A, Tuytelaars T, Van Gool L. Speeded-up robust features (SURF). Computer Vision and Image Understanding, 2008, 110(3): 346−359 doi: 10.1016/j.cviu.2007.09.014
[20]	Liang Q K, Zhu W, Sun W, Yu Z, Wang Y N, Zhang D. In-line inspection solution for codes on complex backgrounds for the plastic container industry. Measurement, 2019, 148: Article No. 106965 doi: 10.1016/j.measurement.2019.106965
[21]	Zhang Z F, Wen G R, Chen S B. Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding. Journal of Manufacturing Processes, 2019, 45: 208−216 doi: 10.1016/j.jmapro.2019.06.023
[22]	Ma L Y, Xie W, Zhang Y. Blister defect detection based on convolutional neural network for polymer lithium-ion battery. Applied Sciences, 2019, 9(6): Article No. 1085 doi: 10.3390/app9061085
[23]	Deng Y S, Luo A C, Dai M J. Building an automatic defect verification system using deep neural network for PCB defect classification. In: Proceedings of the 4th International Conference on Frontiers of Signal Processing (ICFSP). Poitiers, France: IEEE, 2018. 145−149
[24]	Shang L D, Yang Q S, Wang J N, Li S B, Lei W M. Detection of rail surface defects based on CNN image recognition and classification. In: Proceedings of the 20th International Conference on Advanced Communication Technology (ICACT). Chuncheon-si Gangwon-do, Korea (South): IEEE, 2018. 45−51
[25]	Chen J W, Liu Z G, Wang H R, Núñez A, Han Z W. Automatic defect detection of fasteners on the catenary support device using deep convolutional neural network. IEEE Transactions on Instrumentation and Measurement, 2018, 67(2): 257−269 doi: 10.1109/TIM.2017.2775345
[26]	Li W, Field K G, Morgan D. Automated defect analysis in electron microscopic images. NPJ Computational Materials, 2018, 4(1): Article No. 36 doi: 10.1038/s41524-018-0093-8
[27]	Xie Q, Li D W, Xu J X, Yu Z G, Wang J. Automatic detection and classification of sewer defects via hierarchical deep learning. IEEE Transactions on Automation Science and Engineering, 2019, 16(4): 1836−1847 doi: 10.1109/TASE.2019.2900170
[28]	Nagata F, Tokuno K, Nakashima K, Otsuka A, Ikeda T, Ochi H, et al. Fusion method of convolutional neural network and support vector machine for high accuracy anomaly detection. In: Proceedings of the 2019 IEEE International Conference on Mechatronics and Automation (ICMA). Tianjin, China: IEEE, 2019. 970−975
[29]	Cha Y J, Choi W, Büyüköztürk O. Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering, 2017, 32(5): 361−378 doi: 10.1111/mice.12263
[30]	Chen F C, Jahanshahi M R. NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Transactions on Industrial Electronics, 2018, 65(5): 4392−4400 doi: 10.1109/TIE.2017.2764844
[31]	Park S, Bang S, Kim H, Kim H. Patch-based crack detection in black box images using convolutional neural networks. Journal of Computing in Civil Engineering, 2019, 33(3): Article No. 04019017
[32]	Feng C, Liu M Y, Kao C C, Lee T Y. Deep active learning for civil infrastructure defect detection and classification. In: Proceedings of the 2017 International Workshop on Computing in Civil Engineering (IWCCE). Seattle, Washington, USA: American Society of Civil Engineers (ASCE), 2017. 298−306
[33]	Ren R X, Hung T, Tan K C. A generic deep-learning-based approach for automated surface inspection. IEEE Transactions on Cybernetics, 2018, 48(3): 929−940 doi: 10.1109/TCYB.2017.2668395
[34]	Otsu N. A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62−66 doi: 10.1109/TSMC.1979.4310076
[35]	Zhou B L, Khosla A, Lapedriza A, Oliva A, Torralba A. Learning deep features for discriminative localization. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 2921−2929
[36]	Selvaraju R R, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual explanations from deep networks via gradient-based localization. In: Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 618−626
[37]	Lin H, Li B, Wang X G, Shu Y F, Niu S L. Automated defect inspection of LED chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 2019, 30(6): 2525−2534 doi: 10.1007/s10845-018-1415-x
[38]	Zhou F, Liu G H, Xu F, Deng H. A generic automated surface defect detection based on a bilinear model. Applied Sciences, 2019, 9(15): Article No. 3159 doi: 10.3390/app9153159
[39]	Racki D, Tomazevic D, Skocaj D. A compact convolutional neural network for textured surface anomaly detection. In: Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). Lake Tahoe, NV, USA: IEEE, 2018. 1331−1339
[40]	Tabernik D, Šela S, Skvarč J, Skočaj D. Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 2020, 31(3): 759−776 doi: 10.1007/s10845-019-01476-x
[41]	Zhao Z B, Xu G Z, Qi Y C, Liu N, Zhang T F. Multi-patch deep features for power line insulator status classification from aerial images. In: Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN). Vancouver, BC, Canada: IEEE, 2016. 3187−3194
[42]	Malekzadeh T, Abdollahzadeh M, Nejati H, Cheung N M. Aircraft fuselage defect detection using deep neural networks. arXiv preprint arXiv: 1712.09213, 2020.
[43]	Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137−1149
[44]	Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, et al. SSD: Single shot multibox detector. In: Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands: Springer, 2016. 21−37
[45]	Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 779−788
[46]	Cha Y J, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(9): 731−747 doi: 10.1111/mice.12334
[47]	Zhong J P, Liu Z G, Han Z W, Han Y, Zhang W X. A CNN-based defect inspection method for catenary split pins in high-speed railway. IEEE Transactions on Instrumentation and Measurement, 2019, 68(8): 2849−2860 doi: 10.1109/TIM.2018.2871353
[48]	Tao X, Zhang D P, Wang Z H, Liu X L, Zhang H Y, Xu D. Detection of power line insulator defects using aerial images analyzed with convolutional neural networks. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020, 50(4): 1486−1498 doi: 10.1109/TSMC.2018.2871750
[49]	Xue Y D, Li Y C. A fast detection method via region-based fully convolutional neural networks for shield tunnel lining defects. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(8): 638−654 doi: 10.1111/mice.12367
[50]	Ding R W, Dai L H, Li G P, Liu H. TDD-net: A tiny defect detection network for printed circuit boards. CAAI Transactions on Intelligence Technology, 2019, 4(2): 110−116 doi: 10.1049/trit.2019.0019
[51]	He Y, Song K C, Meng Q G, Yan Y H. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 2020, 69(4): 1493−1504 doi: 10.1109/TIM.2019.2915404
[52]	Cheng J C P, Wang M Z. Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Automation in Construction, 2018, 95: 155−171 doi: 10.1016/j.autcon.2018.08.006
[53]	Lei H W, Wang B, Wu H H, Wang A H. Defect detection for polymeric polarizer based on faster R-CNN. Journal of Information Hiding and Multimedia Signal Processing, 2018, 9(6): 1414−1420
[54]	Zhao Z B, Zhen Z, Zhang L, Qi Y C, Kong Y H, Zhang K. Insulator detection method in inspection image based on improved faster R-CNN. Energies, 2019, 12(7): Article No. 1204 doi: 10.3390/en12071204
[55]	Neuhauser F M, Bachmann G, Hora P. Surface defect classification and detection on extruded aluminum profiles using convolutional neural networks. International Journal of Material Forming, 2020, 13(4): 591−603 doi: 10.1007/s12289-019-01496-1
[56]	Sun X H, Gu J N, Huang R, Zou R, Palomares B G. Surface defects recognition of wheel hub based on improved faster R-CNN. Electronics, 2019, 8(5): Article No. 481 doi: 10.3390/electronics8050481
[57]	Li Y T, Huang H S, Xie Q S, Yao L G, Chen Q P. Research on a surface defect detection algorithm based on MobileNet-SSD. Applied Sciences, 2018, 8(9): Article No. 1678 doi: 10.3390/app8091678
[58]	Liu Z G, Liu K, Zhong J P, Han Z W, Zhang W X. A high-precision positioning approach for catenary support components with multiscale difference. IEEE Transactions on Instrumentation and Measurement, 2020, 69(3): 700−711 doi: 10.1109/TIM.2019.2905905
[59]	Zhang C B, Chang C C, Jamshidi M. Concrete bridge surface damage detection using a single-stage detector. Computer-Aided Civil and Infrastructure Engineering, 2020, 35(4): 389−409 doi: 10.1111/mice.12500
[60]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015. 3431−3440
[61]	He K M, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 2961−2969
[62]	Ronneberger O, Fischer P, Brox, T. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer, 2015. 234−241
[63]	Badrinarayanan V, Kendall A, Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481−2495 doi: 10.1109/TPAMI.2016.2644615
[64]	Wang R, Guo Q, Lu S M, Zhang C M. Tire defect detection using fully convolutional network. IEEE Access, 2019, 7: 43502−43510 doi: 10.1109/ACCESS.2019.2908483
[65]	Yu Z Y, Wu X J, Gu X D. Fully convolutional networks for surface defect inspection in industrial environment. In: Proceedings of the 11th International Conference on Computer Vision Systems. Shenzhen, China: Springer, 2017. 417−426
[66]	Dung C V, Anh L D. Autonomous concrete crack detection using deep fully convolutional neural network. Automation in Construction, 2019, 99: 52−58 doi: 10.1016/j.autcon.2018.11.028
[67]	Huang Y B, Qiu C Y, Guo Y, Wang X N, Yuan K. Surface defect saliency of magnetic tile. In: Proceedings of the 14th International Conference on Automation Science and Engineering (CASE). Munich, Germany: IEEE, 2018. 612−617
[68]	Li S Y, Zhao X F, Zhou G Y. Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 2019, 34(7): 616−634 doi: 10.1111/mice.12433
[69]	Liu E Y, Chen K P, Xiang Z Y, Zhang J. Conductive particle detection via deep learning for ACF bonding in TFT-LCD manufacturing. Journal of Intelligent Manufacturing, 2020, 31(4): 1037−1049 doi: 10.1007/s10845-019-01494-9
[70]	Dong Y N, Wang J, Wang Z F, Zhang X, Gao Y, Sui Q M, et al. A deep-learning-based multiple defect detection method for tunnel lining damages. IEEE Access, 2019, 7: 182643−182657 doi: 10.1109/ACCESS.2019.2931074
[71]	Roberts G, Haile S Y, Sainju R, Edwards D J, Hutchinson B, Zhu Y Y. Deep learning for semantic segmentation of defects in advanced STEM images of steels. Scientific Reports, 2019, 9(1): Article No. 12744 doi: 10.1038/s41598-019-49105-0
[72]	Zou Q, Zhang Z, Li Q Q, Qi X B, Wang Q, Wang S. DeepCrack: Learning hierarchical convolutional features for crack detection. IEEE Transactions on Image Processing, 2019, 28(3): 1498−1512 doi: 10.1109/TIP.2018.2878966
[73]	Chaurasia A, Culurciello E. LinkNet: Exploiting encoder representations for efficient semantic segmentation. In: Proceedings of the 2017 IEEE Visual Communications and Image Processing (VCIP). St. Petersburg, FL, USA: IEEE, 2017. 1−4
[74]	Chen L C, Zhu Y K, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018. 801−818
[75]	Zhao H S, Shi J P, Qi X J, Wang X G, Jia J Y. Pyramid scene parsing network. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017. 2881−2890
[76]	Tao X, Zhang D P, Ma W Z, Liu X L, Xu D. Automatic metallic surface defect detection and recognition with convolutional neural networks. Applied Sciences, 2018, 8(9): Article No. 1575 doi: 10.3390/app8091575
[77]	Anantrasirichai N, Bull D. DefectNET: Multi-class fault detection on highly-imbalanced datasets. arXiv preprint arXiv: 1904.00863, 2019.
[78]	Choi W, Cha Y J. SDDNet: Real-time crack segmentation. IEEE Transactions on Industrial Electronics, 2020, 67(9): 8016−825 doi: 10.1109/TIE.2019.2945265
[79]	Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Quebec, Canada: MIT Press, 2014. 2672−2680
[80]	Yuan Z C, Zhang Z T, Su H, Zhang L, Shen F, Zhang F. Vision-based defect detection for mobile phone cover glass using deep neural networks. International Journal of Precision Engineering and Manufacturing, 2018, 19(6): 801−810 doi: 10.1007/s12541-018-0096-x
[81]	Mei Q P, Gül M. A conditional wasserstein generative adversarial network for pixel-level crack detection using video extracted images. arXiv preprint arXiv: 1907.06014, 2019.
[82]	Singh J, Shekhar S. Road damage detection and classification in smartphone captured images using mask R-CNN. arXiv preprint arXiv: 1811.04535, 2018.
[83]	Ferguson M K, Ronay A K, Lee Y T T, Law K H. Detection and segmentation of manufacturing defects with convolutional neural networks and transfer learning. Smart and Sustainable Manufacturing Systems, 2018, 2(1): 137−164.
[84]	Taheritanjani S, Schoenfeld R, Bruegge B. Automatic damage detection of fasteners in overhaul processes. In: Proceedings of the 15th International Conference on Automation Science and Engineering (CASE). Vancouver, BC, Canada: IEEE, 2019, 1289−1295
[85]	Liong S T, Gan Y S, Huang Y C, Yuan C A, Chang H C. Automatic defect segmentation on leather with deep learning. arXiv preprint arXiv: 1903.12139, 2019.
[86]	Kim M S, Park T, Park P. Classification of steel surface defect using convolutional neural network with few images. In: Proceedings of the 12th Asian Control Conference (ASCC). Kitakyushu-shi, Japan, : IEEE, 2019. 1398−1401
[87]	Wu S L, Wu Y B, Cao D H, Zheng C Y. A fast button surface defect detection method based on Siamese network with imbalanced samples. Multimedia Tools and Applications, 2019, 78(24): 34627−34648 doi: 10.1007/s11042-019-08042-w
[88]	Liu J B, Huang Y P, Zou Q, Tian M, Wang S C, Zhao X X, et al. Learning visual similarity for inspecting defective railway fasteners. IEEE Sensors Journal, 2019, 19(16): 6844−6857 doi: 10.1109/JSEN.2019.2911015
[89]	Staar B, Lütjen M, Freitag M. Anomaly detection with convolutional neural networks for industrial surface inspection. Procedia CIRP, 2019, 79: 484−489 doi: 10.1016/j.procir.2019.02.123
[90]	Tang S L, He F, Huang X L, Yang J. Online PCB defect detector on a new PCB defect dataset. arXiv preprint arXiv: 1902.06197, 2019.
[91]	Liu R, Yao M. Surface defect detection based on deep convolutional Siamese networks and defect saliency. Acta Microscopica, 2019, 28(2): 199−207
[92]	Mei S, Yang H, Yin Z P. An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Transactions on Instrumentation and Measurement, 2018, 67(6): 1266−1277 doi: 10.1109/TIM.2018.2795178
[93]	Haselmann M, Gruber D P, Tabatabai P. Anomaly detection using deep learning based image completion. In: Proceedings of the 17th IEEE International Conference on Machine Learning and Applications (ICMLA). Orlando, Florida, USA : IEEE, 2018. 1237−1242
[94]	Kang G Q, Gao S B, Yu L, Zhang D K. Deep architecture for high-speed railway insulator surface defect detection: Denoising autoencoder with multitask learning. IEEE Transactions on Instrumentation and Measurement, 2019, 68(8): 2679−2690 doi: 10.1109/TIM.2018.2868490
[95]	Youkachen S, Ruchanurucks M, Phatrapomnant T, Kaneko H. Defect segmentation of hot-rolled steel strip surface by using convolutional auto-encoder and conventional image processing. In: Proceedings of the 10th International Conference of Information and Communication Technology for Embedded Systems (IC-ICTES). Bangkok, Thailand: IEEE, 2019. 1−5
[96]	Zhao Z X, Li B, Dong R, Zhao P. A surface defect detection method based on positive samples. In: Proceedings of the 15th Pacific Rim International Conference on Artificial Intelligence. Nanjing, China: Springer, 2018. 473−481
[97]	Bergmann P, Löwe S, Fauser M, Sattlegger D, Steger C. Improving unsupervised defect segmentation by applying structural similarity to autoencoders. arXiv preprint arXiv: 1807.02011, 2019.
[98]	Yang H, Chen Y F, Song K Y, Yin Z P. Multiscale feature-clustering-based fully convolutional autoencoder for fast accurate visual inspection of texture surface defects. IEEE Transactions on Automation Science and Engineering, 2019, 16(3): 1450−1467 doi: 10.1109/TASE.2018.2886031
[99]	Zhai W, Zhu J, Cao Y, Wang Z F. A generative adversarial network based framework for unsupervised visual surface inspection. In: Proceedings of the 2018 International Conference on Acoustics, Speech and Signal Processing (ICASSP). Calgary, AB, Canada: IEEE, 2018. 1283−1287
[100]	Hu G H, Huang J F, Wang Q H, Li J R, Xu Z J, Huang X B. Unsupervised fabric defect detection based on a deep convolutional generative adversarial network. Textile Research Journal, 2020, 90(3-4): 247−270 doi: 10.1177/0040517519862880
[101]	Schlegl T, Seeböck P, Waldstein S M, Schmidt-Erfurth U, Langs G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Proceedings of the 25th International Conference on Information Processing in Medical Imaging. Boone, NC, USA: Springer, 2017. 146−157
[102]	Lai Y T K, Hu J S. A texture generation approach for detection of novel surface defects. In: Proceedings of the 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Miyazaki, Japan: IEEE, 2018. 4357−4362
[103]	Soukup D, Pinetz T. Reliably decoding autoencoders’ latent spaces for one-class learning image inspection scenarios. In: Proceedings of the 2018 OAGM Workshop. Hall/Tyrol, Austria: Verlag der TU Graz, 2018. 90−93
[104]	Liu K, Li A M, Wen X, Chen H Y, Yang P. Steel surface defect detection using GAN and one-class classifier. In: Proceedings of the 25th International Conference on Automation and Computing (ICAC). Lancaster, United Kingdom: IEEE, 2019. 1−6
[105]	Schlegl T, Seeböck P, Waldstein S M, Langs G, Schmidt-Erfurth U. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Analysis, 2019, 54: 30−44 doi: 10.1016/j.media.2019.01.010
[106]	Akcay S, Atapour-Abarghouei A, Breckon T P. GANomaly: Semi-supervised anomaly detection via adversarial training. In: Proceedings of the 14th Asian Conference on Computer Vision. Perth, Australia: Springer, 2018. 622−637
[107]	Marino S, Beauseroy P, Smolarz A. Weakly-supervised learning approach for potato defects segmentation. Engineering Applications of Artificial Intelligence, 2019, 85: 337−346 doi: 10.1016/j.engappai.2019.06.024
[108]	Zhou Y Z, Zhu Y, Ye Q X, Qiu Q, Jiao J B. Weakly supervised instance segmentation using class peak response. In: Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 3791−3800
[109]	Mayr M, Hoffmann M, Maier A, Christlein V. Weakly supervised segmentation of cracks on solar cells using normalized Lp norm. In: Proceedings of the 2019 IEEE International Conference on Image Processing (ICIP). Taipei, China: IEEE, 2019. 1885−1889
[110]	Niu S L, Lin H, Niu T Z, Li B, Wang X G. DefectGAN: Weakly-supervised defect detection using generative adversarial network. In: Proceedings of the 15th International Conference on Automation Science and Engineering (CASE). Vancouver, BC, Canada: IEEE, 2019. 127−132
[111]	Zhu J Y, Park T, Isola P, Efros A A. Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017. 2223−2232
[112]	He D, Xu K, Zhou P, Zhou D D. Surface defect classification of steels with a new semi-supervised learning method. Optics and Lasers in Engineering, 2019, 117: 40−48 doi: 10.1016/j.optlaseng.2019.01.011
[113]	Odena A. Semi-supervised learning with generative adversarial networks. arXiv preprint arXiv: 1606.01583, 2016.
[114]	He Y, Song K C, Dong H W, Yan Y H. Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network. Optics and Lasers in Engineering, 2019, 122: 294−302 doi: 10.1016/j.optlaseng.2019.06.020
[115]	Gao Y P, Gao L, Li X Y, Yan X G. A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robotics and Computer-Integrated Manufacturing, 2020, 61: Article No. 101825 doi: 10.1016/j.rcim.2019.101825
[116]	Tao X, Wang Z H, Zhang Z T, Zhang D P, Xu D, Gong X Y, et al. Wire defect recognition of spring-wire socket using multitask convolutional neural networks. IEEE Transactions on Components, Packaging and Manufacturing Technology, 2018, 8(4): 689−698 doi: 10.1109/TCPMT.2018.2794540
[117]	Wei B, Hao K R, Tang X S, Ding Y S. A new method using the convolutional neural network with compressive sensing for fabric defect classification based on small sample sizes. Textile Research Journal, 2019, 89(17): 3539−3555 doi: 10.1177/0040517518813656
[118]	Haselmann M, Gruber D. Supervised machine learning based surface inspection by synthetizing artificial defects. In: Proceedings of the 16th IEEE international conference on machine learning and applications (ICMLA). Cancun, Mexico: IEEE, 2017. 390−395
[119]	Liu L Z, Cao D H, Wu Y B, Wei T R. Defective samples simulation through adversarial training for automatic surface inspection. Neurocomputing, 2019, 360: 230−245 doi: 10.1016/j.neucom.2019.05.080
[120]	Huang C C, Lin X P. Study on machine learning based intelligent defect detection system. In: Proceedings of the 3rd International Conference on Inventions (ICI). Taiwan, China: MATEC Web of Conferences, 2018. 1−10
[121]	Zhang H D, Chen Z Z, Zhang C Q, Xi J T, Le X J. Weld defect detection based on deep learning method. In: Proceedings of the 15th International Conference on Automation Science and Engineering (CASE). Vancouver, BC, Canada: IEEE, 2019. 1574−1579
[122]	Chou Y C, Kuo C J, Chen T T, Horng G J, Pai M Y, Wu M E, et al. Deep-learning-based defective bean inspection with GAN-structured automated labeled data augmentation in coffee industry. Applied Sciences, 2019, 9(19): Article No. 4166 doi: 10.3390/app9194166
[123]	Yang H, Mei S, Song K Y, Tao B, Yin Z P. Transfer-learning-based online Mura defect classification. IEEE Transactions on Semiconductor Manufacturing, 2018, 31(1): 116−123 doi: 10.1109/TSM.2017.2777499
[124]	Zhang C, Shi W, Li X F, Zhang H J, Liu H. Improved bare PCB defect detection approach based on deep feature learning. The Journal of Engineering, 2018, 2018(16): 1415−1420 doi: 10.1049/joe.2018.8275
[125]	Badmos O, Kopp A, Bernthaler T, Schneider G. Image-based defect detection in lithiumion battery electrode using convolutional neural networks. Journal of Intelligent Manufacturing, 2020, 31(4): 885−897 doi: 10.1007/s10845-019-01484-x
[126]	Sun J, Wang P, Luo Y K, Li W Y. Surface defects detection based on adaptive multiscale image collection and convolutional neural networks. IEEE Transactions on Instrumentation and Measurement, 2019, 68(12): 4787−4797 doi: 10.1109/TIM.2019.2899478
[127]	Kim S, Kim W, Noh Y K, Park F C. Transfer learning for automated optical inspection. In: Proceedings of the 2017 International Joint Conference on Neural Networks (IJCNN). Anchorage, USA: IEEE, 2017. 2517−2524
[128]	Krishnamoorthi R. Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv: 1806.08342, 2018.
[129]	Han S, Pool J, Tran J, Dally W J. Learning both weights and connections for efficient neural network. In: Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Quebec, Canada: MIT Press, 2015. 1135−1143
[130]	Lei J, Gao X, Feng Z L, Qiu H M, Song M L. Scale insensitive and focus driven mobile screen defect detection in industry. Neurocomputing, 2018, 294: 72−81 doi: 10.1016/j.neucom.2018.03.013
[131]	Pan Y F, Lu R S, Zhang T D. FPGA-accelerated textured surface defect segmentation based on complete period Fourier reconstruction. Journal of Real-Time Image Processing, 2020, 17(5): 1659−1673 doi: 10.1007/s11554-019-00927-1
[132]	Luo Q W, He Y G. A cost-effective and automatic surface defect inspection system for hot-rolled flat steel. Robotics and Computer-Integrated Manufacturing, 2016, 38: 16−30 doi: 10.1016/j.rcim.2015.09.008
[133]	罗菁, 董婷婷, 宋丹, 修春波. 表面缺陷检测综述. 计算机科学与探索, 2014, 8(9): 1041−1048 doi: 10.3778/j.issn.1673-9418.1405007 Luo Jing, Dong Ting-Ting, Song Dan, Xiu Chun-Bo. A review on surface defect detection. Journal of Frontiers of Computer Science and Technology, 2014, 8(9): 1041−1048 doi: 10.3778/j.issn.1673-9418.1405007
[134]	Deng J, Dong W, Socher R, Li L, Li K, Li F F. Imagenet: A large-scale hierarchical image database. In: Proceedings of the 2009 Conference on Computer Vision and Pattern Recognition. Anchorage, USA: IEEE, 2009. 248−255
[135]	Everingham M, Van Gool L, Williams C K I, Winn J, Zisserman A. The PASCAL visual object classes (VOC) challenge. International Journal of Computer Vision, 2010, 88(2): 303−338 doi: 10.1007/s11263-009-0275-4
[136]	Lin T Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, et al. Microsoft COCO: Common objects in context. In: Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland: Springer, 2014. 740−755
[137]	Silvén O, Niskanen M, Kauppinen H. Wood inspection with non-supervised clustering. Machine Vision and Applications, 2003, 13(5-6): 275−285 doi: 10.1007/s00138-002-0084-z
[138]	Ferguson M, Ak R, Lee Y T T, Law K H. Automatic localization of casting defects with convolutional neural networks. In: Proceedings of the 2017 IEEE International Conference on Big Data. Boston, MA, USA: IEEE, 2017. 1726−1735
[139]	Gan J R, Li Q Y, Wang J Z, Yu H M. A hierarchical extractor-based visual rail surface inspection system. IEEE Sensors Journal, 2017, 17(23): 7935−7944 doi: 10.1109/JSEN.2017.2761858
[140]	Yang X C, Li H, Yu Y T, Luo X C, Huang T, Yang X. Automatic pixel-level crack detection and measurement using fully convolutional network. Computer-Aided Civil and Infrastructure Engineering, 2018, 33(12): 1090−1109 doi: 10.1111/mice.12412
[141]	李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727−1742 Li Liang-Fu, Ma Wei-Fei, Li Li, Lu Cheng. Research on detection algorithm for bridge cracks based on deep learning. Acta Automatica Sinica, 2019, 45(9): 1727−1742
[142]	Bergmann P, Fauser M, Sattlegger D, Steger C. MVTec AD — A comprehensive real-world dataset for unsupervised anomaly detection. In: Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019. 9592−9600
[143]	Carrera D, Manganini F, Boracchi G, Lanzarone E. Defect detection in SEM images of nanofibrous materials. IEEE Transactions on Industrial Informatics, 2017, 13(2): 551−561 doi: 10.1109/TII.2016.2641472
[144]	DAGM 2007 Datasets. Weakly supervised learning for industrial optical inspection [Online], available: https://hci.iwr.uni-heidelberg.de/node/3616, February 27, 2018
[145]	Weimer D, Scholz-Reiter B, Shpitalni M. Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP Annals, 2016, 65(1): 417−420 doi: 10.1016/j.cirp.2016.04.072
[146]	Wang T, Chen Y, Qiao M N, Snoussi H. A fast and robust convolutional neural network-based defect detection model in product quality control. The International Journal of Advanced Manufacturing Technology, 2018, 94(9-12): 3465−3471 doi: 10.1007/s00170-017-0882-0
[147]	Wu X J, Cao K, Gu X D. A surface defect detection based on convolutional neural network. In: Proceedings of the 11th International Conference on Computer Vision Systems. Shenzhen, China: Springer, 2017. 185−194
[148]	Dai J F, Li Y, He K M, Sun J. R-FCN: Object detection via region-based fully convolutional networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: Curran Associates Inc., 2016. 379−387