基于计算机视觉的工业金属表面缺陷检测综述

伍麟; 郝鸿宇; 宋友

doi:10.16383/j.aas.c230039

基于计算机视觉的工业金属表面缺陷检测综述

doi: 10.16383/j.aas.c230039

伍麟^1,,
郝鸿宇^1,,
宋友^1,

1.
北京航空航天大学软件学院北京 100191

详细信息

作者简介:
伍麟：北京航空航天大学硕士研究生. 主要研究方向为计算机视觉, 目标检测和表面缺陷检测. E-mail: zf2021349@buaa.edu.cn

郝鸿宇：北京航空航天大学硕士研究生. 主要研究方向为计算机视觉, 图神经网络和少样本学习. E-mail: JoeyHao@buaa.edu.cn

宋友：北京航空航天大学教授. 主要研究方向为软件工程, 异常信号检测, 算法分析与设计. 本文通信作者. E-mail: songyou@buaa.edu.cn

计量
- 文章访问数: 2941
- HTML全文浏览量: 3345
- PDF下载量: 756
- 被引次数: 0
出版历程
- 收稿日期: 2023-02-06
- 录用日期: 2023-05-18
- 网络出版日期: 2023-07-03
- 刊出日期: 2024-07-23

A Review of Metal Surface Defect Detection Based on Computer Vision

WU Lin^1
,,
HAO Hong-Yu^1
,,
SONG You^1
,

1.
School of Software, Beihang University, Beijing 100191

More Information

Author Bio:
WU Lin　Master student at the School of Software, Beihang University. His research interest covers computer vision, object detection and surface defect detection

HAO Hong-Yu　Master student at the School of Software, Beihang University. His research interest covers computer vision, graph neural network and few-shot learning

SONG You　Professor at the School of Software, Beihang University. His research interest covers software engineering, anomaly signal detection, algorithm analysis and design. Corresponding author of this paper

摘要

摘要: 针对平面及三维结构金属材料的工业表面缺陷检测, 概述了视觉检测技术的基本原理和研究现状, 并总结出视觉自动检测系统的关键技术包括光学成像技术、图像预处理技术与缺陷检测器. 首先介绍了如何根据检测对象的光学特性选择合适的二维、三维光学成像技术; 其次介绍了图像降噪、特征提取、图像分割和拼接等预处理技术的重要作用; 然后根据缺陷检测器的实现原理将其分为模板匹配、图像分类、图像语义分割、目标检测和图像异常检测五类, 并对其中的经典算法进行了归纳分析. 最后, 探讨了工业场景下金属表面缺陷检测技术实施中的关键问题, 并对该技术的发展趋势进行了展望.
- 表面缺陷检测 /
- 计算机视觉 /
- 金属表面缺陷 /
- 自动化检测
Abstract: Focusing on the industrial surface defect detection of metal planar and three-dimensional structural materials, this paper summarizes the basic principle and research status of visual defect detection technology, and summarizes the key technologies of visual automatic detection system including optical imaging technology, image preprocessing technology and defect detector. Firstly, this paper introduces how to select suitable 2D and 3D optical imaging technology according to the optical characteristics of the test object. Secondly, the important functions of image denoising, feature extraction, image segmentation and image Mosaic are introduced. Then, according to the implementation principle of defect detector, it is divided into five categories: Template matching, image classification, image semantic segmentation, target detection and image anomaly detection, and the classical algorithms are summarized and analyzed. Finally, this paper discusses the key problems in the implementation of surface defect detection in the industrial scene, and looks forward to the development trend of this technology.
- Surface defect detection /
- computer vision /
- metal surface defect /
- automatic inspection system

HTML全文

图 1 金属表面缺陷检测基本流程

Fig. 1 Pipline of metal surface defect detection

下载: 全尺寸图片幻灯片

图 2 自动光学成像系统

Fig. 2 Automated optical inspection system

下载: 全尺寸图片幻灯片

图 3 表面散射模型

Fig. 3 Light scattering model on surface

下载: 全尺寸图片幻灯片

图 4 照明光路类型

Fig. 4 Types of lighting path

下载: 全尺寸图片幻灯片

图 5 二维成像与三维成像对比

Fig. 5 2D imaging versus 3D imaging

下载: 全尺寸图片幻灯片

图 6 光度立体法

Fig. 6 Photometric stereo

下载: 全尺寸图片幻灯片

图 7 结构光法

Fig. 7 Structured light illumination

下载: 全尺寸图片幻灯片

图 8 混合成像技术 ((a) ~ (c) 二维灰度图像; (d) ~ (f)具有三维深度信息表示的图像)

Fig. 8 Hybrid imaging technique ((a) ~ (c) 2D grayscale images; (d) ~ (f) Images with 3D depth information represented)

下载: 全尺寸图片幻灯片

图 9 基于图像分割的缺陷检测

Fig. 9 Defect detection based on image segmentation

下载: 全尺寸图片幻灯片

图 10 三元网络结构

Fig. 10 The structure of triplet network

下载: 全尺寸图片幻灯片

图 11 二阶段网络和一阶段网络对比

Fig. 11 Comparison of two-stage and one-stage networks

下载: 全尺寸图片幻灯片

图 12 基于二阶段网络的金属表面缺陷检测^[100], 经许可转载自文献[100], ©Sage, 2021

Fig. 12 Metal surface defect detection based on two-stage networks^[100], reproduced with permission from reference [100], ©Sage, 2021

下载: 全尺寸图片幻灯片

图 13 DETR网络结构

Fig. 13 The network architecture of DETR

下载: 全尺寸图片幻灯片

图 14 基于图像重建的缺陷检测 ((a)变分自编码机; (b) GAN结合AE; (c)基于记忆池的模型)

Fig. 14 Defect detection based on image reconstruction ((a) VAE; (b) GAN associated with Auto-Encoder; (c) Memory based model)

下载: 全尺寸图片幻灯片

图 15 基于标准化流的缺陷检测 ((a)原始图像; (b)多尺度输入; (c)图像特征分布; (d)简单分布; (e)标准分布; (f)异常分布)

Fig. 15 Defect detection based on normalizing flow ((a) Origin image; (b) Multiscale input; (c) Feature distribution; (d) Simple distribution; (e) Normalized distribution; (f) Anomalous distribution)

下载: 全尺寸图片幻灯片

图 16 (a)基于教师−学生网络的方法; (b)基于最典型嵌入表示的方法

Fig. 16 (a) Method based on teacher-student network; (b) Method based on the most typical embedding representation

下载: 全尺寸图片幻灯片

表 1 目标检测模型在NEU-DET上的表现

Table 1 Performance of object detection models on NEU-DET

Method	Backbone	Neck	$AP_{50}$
Faster R-CNN^[97]	ResNet-50	FPN	74.7
Cascade R-CNN^[99]	ResNet-50	FPN	75.8
YOLOX^[115]	CSPDarknet	PA-FPN	70.9
YOLOv4^[105]	CSPDarknet	FPN	76.4
AutoAssign^[116]	ResNet-50	FPN	76.6
AutoAssign^[116]	Swin-Tiny	FPN	78.3
DDN^[108]	ResNet-50	MFN	82.3
CA-AutoAssign^[117]	CSPDarknet	CA	82.7

下载: 导出CSV

表 2 异常检测方法对比

Table 2 Comparison of abnormal detection

Method	Detection AUROC	Segmentation AUROC	FPS
PatchCore Large^[145]	99.6	98.2	5.9
PNI^[146]	99.5	99.0	—
MemSeg^[144]	99.5	99.6	31.3
Fastflow^[141]	99.4	98.5	21.8
EfficientAD-M^[148]	99.1	96.9	269.0
EfficientAD-S^[148]	98.8	96.8	614.0
CS-Flow^[140]	98.7	—	—
Patch SVDD^[142]	92.1	95.7	2.1
VAE-Grad^[136]	89.2	—	—

下载: 导出CSV

表 3 缺陷检测方法对比

Table 3 Comparison of defect detection methods

方法	基本原理	应用场景	优缺点
模板匹配	比较模板与待检样本的差异来判断是否存在缺陷	产品高度一致的金属精密加工制成品, 例如手机外壳、汽车零件等	方法简单有效, 但需要提取制作模板, 仅适用于一致性强的产品
分类网络	直接用 CNN 提取特征, 通过 Softmax 或距离度量来预测类别	公差较大、尺寸较小的金属制品, 例如螺母、金属盖等零件	结构简单, 是其他网络的基础, 准确率依赖缺陷样本数量, 难以定位缺陷位置
目标检测	对每个提议候选框或者每个网格进行密集预测, 从背景中找出所有目标的分类和位置	适用于绝大多数缺陷类别可事先定义的工业场景	速度快, 适用范围广, 但网络结构复杂, 依赖大量缺陷样本进行训练
语义分割	通过卷积提取高阶语义特征, 然后通过上采样输出像素级的缺陷边界划分	大面积金属板、带制品, 缺陷具有成片连续区域、形态不定的场景	可以进行像素级缺陷分割, 但是依赖大量像素级标注数据, 标注成本很高
异常检测	通过自编码机、GAN、标准流等生成模型学习正常样本的表达方式, 根据重建误差、梯度或分布差异来进行缺陷检测	缺乏缺陷样本, 只有正常样本可以用于训练的场景	无需缺陷样本和标注, 可以检测未事先定义的缺陷类别, 但准确率尚达不到有监督学习的效果

下载: 导出CSV

参考文献(148)

[1]	Neogi N, Mohanta D K, Dutta P K. Review of vision-based steel surface inspection systems. EURASIP Journal on Image and Video Processing, 2014, 2014(1): Article No. 50 doi: 10.1186/1687-5281-2014-50
[2]	卢荣胜, 吴昂, 张腾达, 王永红. 自动光学(视觉)检测技术及其在缺陷检测中的应用综述. 光学学报, 2018, 38(8): Article No. 0815002 Lu Rong-Sheng, Wu Ang, Zhang Teng-Da, Wang Yong-Hong. Review on automated optical (visual) inspection and its applications in defect detection. Acta Optica Sinica, 2018, 38(8): Article No. 0815002
[3]	吕承侃, 沈飞, 张正涛, 张峰. 图像异常检测研究现状综述. 自动化学报, 2022, 48(6): 1402−1428 Lv Cheng-Kan, Shen Fei, Zhang Zheng-Tao, Zhang Feng. Review of image anomaly detection. Acta Automatica Sinica, 2022, 48(6): 1402−1428
[4]	李维创, 尹柏强. 工业金属板带材表面缺陷自动视觉检测研究进展. 电子测量与仪器学报, 2021, 35(6): 1−16 doi: 10.13382/j.jemi.B2003349 Li Wei-Chuang, Yin Bai-Qiang. Research progress of automated visual surface defect detection for industrial metal planar materials. Journal of Electronic Measurement and Instrumentation, 2021, 35(6): 1−16 doi: 10.13382/j.jemi.B2003349
[5]	陶显, 侯伟, 徐德. 基于深度学习的表面缺陷检测方法综述. 自动化学报, 2021, 47(5): 1017−1034 Tao Xian, Hou Wei, Xu De. A survey of surface defect detection methods based on deep learning. Acta Automatica Sinica, 2021, 47(5): 1017−1034
[6]	罗东亮, 蔡雨萱, 杨子豪, 章哲彦, 周瑜, 白翔. 工业缺陷检测深度学习方法综述. 中国科学: 信息科学, 2022, 52(6): 1002−1039 doi: 10.1360/SSI-2021-0336 Luo Dong-Liang, Cai Yu-Xuan, Yang Zi-Hao, Zhang Zhe-Yan, Zhou Yu, Bai Xiang. Survey on industrial defect detection with deep learning. Scientia Sinica Informationis, 2022, 52(6): 1002−1039 doi: 10.1360/SSI-2021-0336
[7]	Khan S, Naseer M, Hayat M, Zamir S W, Khan F S, Shah M. Transformers in vision: A survey. ACM Computing Surveys, 2022, 54(10s): Article No. 200
[8]	Huang Y C, Hung K C, Liu C C, Chuang T H, Chiou S J. Customized convolutional neural networks technology for machined product inspection. Applied Sciences, 2022, 12(6): Article No. 3014 doi: 10.3390/app12063014
[9]	Liu S, Wang Q, Luo Y. A review of applications of visual inspection technology based on image processing in the railway industry. Transportation Safety and Environment, 2019, 1(3): 185−204
[10]	Lu R S, Forrest A K. 3D surface topography from the specular lobe of scattered light. Optics and Lasers in Engineering, 2007, 45(10): 1018−1027 doi: 10.1016/j.optlaseng.2007.04.008
[11]	Smith B. Geometrical shadowing of a random rough surface. IEEE Transactions on Antennas and Propagation, 1967, 15(5): 668−671 doi: 10.1109/TAP.1967.1138991
[12]	Garcia-Lamont F, Cervantes J, López A, Rodriguez L. Segmentation of images by color features: A survey. Neurocomputing, 2018, 292: 1−27 doi: 10.1016/j.neucom.2018.01.091
[13]	Foucher B. Infrared machine vision: A new contender. In: Proceedings of the SPIE 3700, Thermosense XXI. Orlando, USA: SPIE, 1999. 210−213
[14]	Nayar S K, Ikeuchi K, Kanade T. Surface reflection: Physical and geometrical perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1991, 13(7): 611−634 doi: 10.1109/34.85654
[15]	Porter T F, Sylvester R A, Bouyoucas T W, Kolesar M P. Automatic strip surface defect detection system. Iron and Steel Engineer, 1988, 65(12): 17−20
[16]	Tian G Y, Lu R S, Gledhill D. Surface measurement using active vision and light scattering. Optics and Lasers in Engineering, 2007, 45(1): 131−139 doi: 10.1016/j.optlaseng.2006.03.005
[17]	卢荣胜. 自动光学检测技术的发展现状. 红外与激光工程, 2008, 37(S1): 120−123 Lu Rong-Sheng. State of the art of automated optical inspection. Infrared and Laser Engineering, 2008, 37(S1): 120−123
[18]	Rinn R, Thompson S A, Foehr R, Luecking F, Torre J. Parsytec HTS-2: Defect detection and classification through software vs. dedicated hardware. In: Proceedings of the Electronic Imaging '99. San Jose, CA, United States: SPIE, 1999. 110−121
[19]	李松, 周亚同, 张忠伟, 池越, 韩春颖. 基于双打光模板匹配的冲压件表面缺陷检测. 锻压技术, 2018, 43(11): 137−145 Li Song, Zhou Ya-Tong, Zhang Zhong-Wei, Chi Yue, Han Chun-Ying. Surface defect detection of stamping parts based on double light pattern matching. Forging & Stamping Technology, 2018, 43(11): 137−145
[20]	Landstrom A, Thurley M J. Morphology-based crack detection for steel slabs. IEEE Journal of Selected Topics in Signal Processing, 2012, 6(7): 866−875 doi: 10.1109/JSTSP.2012.2212416
[21]	Woodham R J. Photometric method for determining surface orientation from multiple images. Optical Engineering, 1980, 19(1): Article No. 191139
[22]	Berthold K P H. Shape From Shading: A Method for Obtaining the Shape of a Smooth Opaque Object from One View, Technical Report AITR-232, Artificial Intelligence Laboratory, MIT, USA, 1970.
[23]	Wang L, Xu K, Zhou P. Online detection technique of 3D defects for steel strips based on photometric stereo. In: Proceedings of the 8th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA). Macao, China: IEEE, 2016. 428−432
[24]	Delpy D T, Cope M, van der Zee P, Arridge S, Wray S, Wyatt J. Estimation of optical pathlength through tissue from direct time of flight measurement. Physics in Medicine & Biology, 1988, 33(12): 1433−1442
[25]	Bologna F, Tannous M, Romano D, Stefanini C. Automatic welding imperfections detection in a smart factory via 2-D laser scanner. Journal of Manufacturing Processes, 2022, 73: 948−960 doi: 10.1016/j.jmapro.2021.10.046
[26]	Huang C, Wang G L, Song H, Li R S, Zhang H O. Rapid surface defects detection in wire and arc additive manufacturing based on laser profilometer. Measurement, 2022, 189: Article No. 110503 doi: 10.1016/j.measurement.2021.110503
[27]	Li J L, Liu T, Wang X F. Advanced pavement distress recognition and 3D reconstruction by using GA-DenseNet and binocular stereo vision. Measurement, 2022, 201: Article No. 111760 doi: 10.1016/j.measurement.2022.111760
[28]	Li B Z, Xu Z J, Gao F, Cao Y L, Dong Q C. 3D reconstruction of high reflective welding surface based on binocular structured light stereo vision. Machines, 2022, 10(2): Article No. 159 doi: 10.3390/machines10020159
[29]	Zhou P, Xu K, Wang D D. Rail profile measurement based on line-structured light vision. IEEE Access, 2018, 6: 16423−16431 doi: 10.1109/ACCESS.2018.2813319
[30]	Guillo L, Jiang X R, Lafruit G, Guillemot C. Light Field Video Dataset Captured by A R8 Raytrix Camera (With Disparity Maps), Technical Report hal-01804578, International Organisation for Standardisation, San Diego, USA, 2018.
[31]	Saiz F A, Barandiaran I, Arbelaiz A, Graña M. Photometric stereo-based defect detection system for steel components manufacturing using a deep segmentation network. Sensors, 2022, 22(3): Article No. 882 doi: 10.3390/s22030882
[32]	Wen X, Song K C, Huang L M, Niu M H, Yan Y H. Complex surface ROI detection for steel plate fusing the gray image and 3D depth information. Optik, 2019, 198: Article No. 163313 doi: 10.1016/j.ijleo.2019.163313
[33]	Niu M H, Song K C, Huang L M, Wang Q, Yan Y H, Meng Q G. Unsupervised saliency detection of rail surface defects using stereoscopic images. IEEE Transactions on Industrial Informatics, 2021, 17(3): 2271−2281
[34]	Ren Z H, Fang F Z, Yan N, Wu Y. State of the art in defect detection based on machine vision. International Journal of Precision Engineering and Manufacturing-Green Technology, 2022, 9(2): 661−691 doi: 10.1007/s40684-021-00343-6
[35]	Niblack W. An Introduction to Digital Image Processing. Birkeroed: Strandberg Publishing Company, 1985.
[36]	Luisier F, Blu T, Unser M. A new SURE approach to image denoising: Interscale orthonormal wavelet thresholding. IEEE Transactions on Image Processing, 2007, 16(3): 593−606 doi: 10.1109/TIP.2007.891064
[37]	Gu S H, Zhang L, Zuo W M, Feng X C. Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 2862−2869
[38]	Mohanaiah P, Sathyanarayana P, GuruKumar L. Image texture feature extraction using GLCM approach. International Journal of Scientific and Research Publications, 2013, 3(5): 1−5
[39]	Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(7): 971−987 doi: 10.1109/TPAMI.2002.1017623
[40]	Dalal N, Triggs B. Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). San Diego, USA: IEEE, 2005. 886−893
[41]	Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91−110 doi: 10.1023/B:VISI.0000029664.99615.94
[42]	Lindeberg T. Scale invariant feature transform. Scholarpedia, 2012, 7(5): Article No. 10491 doi: 10.4249/scholarpedia.10491
[43]	Oren M, Papageorgiou C, Sinha P, Osuna E, Poggio T. Pedestrian detection using wavelet templates. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Juan, USA: IEEE, 1997. 193−199
[44]	Lienhart R, Maydt J. An extended set of Haar-like features for rapid object detection. In: Proceedings of the International Conference on Image Processing. Rochester, USA: IEEE, 2002.
[45]	Wang X W, Ding X Q, Liu C S. Gabor filters-based feature extraction for character recognition. Pattern Recognition, 2005, 38(3): 369−379 doi: 10.1016/j.patcog.2004.08.004
[46]	Raheja J L, Kumar S, Chaudhary A. Fabric defect detection based on GLCM and Gabor filter: A comparison. Optik, 2013, 124(23): 6469−6474 doi: 10.1016/j.ijleo.2013.05.004
[47]	Siew L H, Hodgson R M, Wood E J. Texture measures for carpet wear assessment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1988, 10(1): 92−105 doi: 10.1109/34.3870
[48]	Liu K, Wang H Y, Chen H Y, Qu E Q, Tian Y, Sun H X. Steel surface defect detection using a new Haar–Weibull-Variance model in unsupervised manner. IEEE Transactions on Instrumentation and Measurement, 2017, 66(10): 2585−2596 doi: 10.1109/TIM.2017.2712838
[49]	Pernkopf F, O'Leary P. Image acquisition techniques for automatic visual inspection of metallic surfaces. NDT & E International, 2003, 36(8): 609−617
[50]	Kaggle. Severstal: Steel defect detection [Online], available: https://www.kaggle.com/c/severstal-steel-defect-detection, March 19, 2023
[51]	Qi S, Yang J, Zhong Z. A review on industrial surface defect detection based on deep learning technology. In: Proceedings of the 3rd International Conference on Machine Learning and Machine Intelligence. Hangzhou, China: ACM, 2020. 24−30
[52]	郑凯, 李建胜. 基于深度神经网络的图像语义分割综述. 测绘与空间地理信息, 2020, 43(10): 119−125 doi: 10.3969/j.issn.1672-5867.2020.10.032 Zheng Kai, Li Jian-Sheng. An overview of image semantic segmentation based on deep learning. Geomatics & Spatial Information Technology, 2020, 43(10): 119−125 doi: 10.3969/j.issn.1672-5867.2020.10.032
[53]	Belongie S, Malik J, Puzicha J. Shape context: A new descriptor for shape matching and object recognition. In: Proceedings of the 13th International Conference on Neural Information Processing Systems. Denver, USA: MIT Press, 2000. 798−804
[54]	Qi C R, Yi L, Su H, Guibas L J. PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 652−660
[55]	Qi C R, Yi L, Su H, Guibas L J. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc., 2017. 5105−5114
[56]	Ma X, Qin C, You H X, Ran H X, Fu Y. Rethinking network design and local geometry in point cloud: A simple residual MLP framework. In: Proceedings of the 10th International Conference on Learning Representations (ICLR). OpenReview.net, 2022.
[57]	Qian G C, Li Y C, Peng H W, Mai J J, Al Kader Hammoud H A, Elhoseiny M, et al. PointNeXt: Revisiting PointNet++ with improved training and scaling strategies. In: Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS). 2022. 23192−23204
[58]	Jain P, Tyagi V. LAPB: Locally adaptive patch-based wavelet domain edge-preserving image denoising. Information Sciences, 2015, 294: 164−181 doi: 10.1016/j.ins.2014.09.060
[59]	Li A B, Ma H W, Xu S H. Three-dimensional morphology and watershed-algorithm-based method for pitting corrosion evaluation. Buildings, 2022, 12(11): Article No. 1908 doi: 10.3390/buildings12111908
[60]	郭皓然, 邵伟, 周阿维, 杨宇祥, 刘凯斌. 全局阈值自适应的高亮金属表面缺陷识别新方法. 仪器仪表学报, 2017, 38(11): 2797−2804 Guo Hao-Ran, Shao Wei, Zhou A-Wei, Yang Yu-Xiang, Liu Kai-Bin. Novel defect recognition method based on adaptive global threshold for highlight metal surface. Chinese Journal of Scientific Instrument, 2017, 38(11): 2797−2804
[61]	魏爱东. 基于脉冲涡流热成像的钢板缺陷检测研究. 电子测试, 2020, 34(7): 56−59 doi: 10.3969/j.issn.1000-8519.2020.07.020 Wei Ai-Dong. Thermal image defect extraction based on two image segmentation algorithms. Electronic Test, 2020, 34(7): 56−59 doi: 10.3969/j.issn.1000-8519.2020.07.020
[62]	Prabha P A, Bharathwaj M, Dinesh K, Prashath G H. Defect detection of industrial products using image segmentation and saliency. Journal of Physics: Conference Series, 2021, 1916: Article No. 012165
[63]	Zhang X W, Ding Y Q, Lv Y Y, Shi A Y, Liang R Y. A vision inspection system for the surface defects of strongly reflected metal based on multi-class SVM. Expert Systems With Applications, 2011, 38(5): 5930−5939 doi: 10.1016/j.eswa.2010.11.030
[64]	Ghosh D, Kaabouch N. A survey on image mosaicing techniques. Journal of Visual Communication and Image Representation, 2016, 34: 1−11 doi: 10.1016/j.jvcir.2015.10.014
[65]	Lu R S, Shi Y Q, Li Q, Yu Q P. AOI techniques for surface defect inspection. In: Proceedings of the Applied Mechanics and Materials. Trans Tech Publications Ltd, 2010. 297−302
[66]	Kong Q, Wu Z, Song Y. Online detection of external thread surface defects based on an improved template matching algorithm. Measurement, 2022, 195: Article No. 111087
[67]	Pang G, Shen C, Cao L, Hengel V A D. Deep learning for anomaly detection: A review. ACM Computing Surveys, 2021, 54(2): 1−38
[68]	Konovalenko I, Maruschak P, Brevus V, Prentkovskis O. Recognition of scratches and abrasions on metal surfaces using a classifier based on a convolutional neural network. Metals, 2021, 11(4): Article No. 549 doi: 10.3390/met11040549
[69]	Jiang Q S, Tan D P, Li Y B, Ji S M, Cai C P, Zheng Q M. Object detection and classification of metal polishing shaft surface defects based on convolutional neural network deep learning. Applied Sciences, 2019, 10(1): Article No. 87 doi: 10.3390/app10010087
[70]	Tabernik D, Šela S, Skvarč J, Skočaj D. Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 2019, 31(3): 759−776
[71]	杨洁. 基于深度学习的无监督图像异常模式检测与识别研究 [博士学位论文], 中国科学院大学, 中国, 2021. Yang Jie. Unsupervised Visual Anomaly Detection and Recognition Based on Deep Learning [Ph.D. dissertation], University of Chinese Academy of Sciences, China, 2021.
[72]	Chetverikov D, Khenokh Y. Matching for shape defect detection. In: Proceedings of the 8th International Conference on Computer Analysis of Images and Patterns (CAIP). Ljubljana, Slovenia: Springer, 1999. 367−374
[73]	Wang L, Pavlidis T. Direct gray-scale extraction of features for character recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1993, 15(10): 1053−1067 doi: 10.1109/34.254062
[74]	Chui H, Rangarajan A. A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding, 2003, 89(2): 114−141 doi: 10.1016/S1077-3142(03)00009-2
[75]	Borgefors G. Hierarchical chamfer matching: A parametric edge matching algorithm. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1988, 10(6): 849−865 doi: 10.1109/34.9107
[76]	Zhang H J, Hu Q. Fast image matching based-on improved SURF algorithm. In: Proceedings of the International Conference on Electronics, Communications and Control (ICECC). Ningbo, China: IEEE, 2011. 1460−1463
[77]	Crispin A J, Rankov V. Automated inspection of PCB components using a genetic algorithm template-matching approach. The International Journal of Advanced Manufacturing Technology, 2007, 35(3): 293−300
[78]	Hashemi N S, Aghdam R B, Ghiasi A S B, Fatemi P. Template matching advances and applications in image analysis. arXiv preprint arXiv: 1610.07231, 2016.
[79]	Wang H Y, Zhang J W, Tian Y, Chen H Y, Sun H X, Liu K. A simple guidance template-based defect detection method for strip steel surfaces. IEEE Transactions on Industrial Informatics, 2019, 15(5): 2798−2809 doi: 10.1109/TII.2018.2887145
[80]	Pernkopf F. Detection of surface defects on raw steel blocks using Bayesian network classifiers. Pattern Analysis and Applications, 2004, 7(3): 333−342 doi: 10.1007/s10044-004-0232-3
[81]	Aghdam S R, Amid E, Imani M F. A fast method of steel surface defect detection using decision trees applied to LBP based features. In: Proceedings of the 7th IEEE Conference on Industrial Electronics and Applications (ICIEA). Singapore: IEEE, 2012. 1447−1452
[82]	Samy M P, Foong S, Soh G S, Yeo K S. Automatic optical & laser-based defect detection and classification in brick masonry walls. In: Proceedings of the IEEE Region 10 Conference (TENCON). Singapore: IEEE, 2016. 3521−3524
[83]	Li X G, Zhu J, Shi H R, Cong Z J. Surface defect detection of seals based on K-means clustering algorithm and particle swarm optimization. Scientific Programming, 2021, 2021: Article No. 3965247
[84]	Yue B, Wang Y P, Min Y Z, Zhang Z H, Wang W R, Yong J. Rail surface defect recognition method based on AdaBoost multi-classifier combination. In: Proceedings of the Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). Lanzhou, China: IEEE, 2019. 391−396
[85]	Soukup D, Huber-Mörk R. Convolutional neural networks for steel surface defect detection from photometric stereo images. In: Proceedings of the 10th International Symposium on Visual Computing (ISVC). Las Vegas, USA: Springer, 2014. 668−677
[86]	Masci J, Meier U, Ciresan D, Schmidhuber J, Fricout G. Steel defect classification with max-pooling convolutional neural networks. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN). Brisbane, Australia: IEEE, 2012. 1−6
[87]	Staar B, Lütjen M, Freitag M. Anomaly detection with convolutional neural networks for industrial surface inspection. Procedia CIRP, 2019, 79: 484−489 doi: 10.1016/j.procir.2019.02.123
[88]	Liao S C, Shao L. Graph sampling based deep metric learning for generalizable person re-identification. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 7349−7358
[89]	Alzu'bi A, Albalas F, Al-Hadhrami T, Younis L B, Bashayreh A. Masked face recognition using deep learning: A review. Electronics, 2021, 10(21): Article No. 2666 doi: 10.3390/electronics10212666
[90]	Mordia R, Verma A K. Visual techniques for defects detection in steel products: A comparative study. Engineering Failure Analysis, 2022, 134: Article No. 106047 doi: 10.1016/j.engfailanal.2022.106047
[91]	Kim M S, Park T, Park P. Classification of steel surface defect using convolutional neural network with few images. In: Proceedings of the 12th Asian Control Conference (ASCC). Kitakyushu, Japan: IEEE, 2019. 1398−1401
[92]	Wu S L, Wu Y B, Cao D H, Zheng C Y. A fast button surface defect detection method based on Siamese network with imbalanced samples. Multimedia Tools and Applications, 2019, 78(24): 34627−34648 doi: 10.1007/s11042-019-08042-w
[93]	Li X, Yang X, Ma Z, Xue J H. Deep metric learning for few-shot image classification: A selective review. arXiv preprint arXiv: 2105.08149, 2021.
[94]	Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 580−587
[95]	He K M, Zhang X Y, Ren S Q, Sun J. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904−1916 doi: 10.1109/TPAMI.2015.2389824
[96]	Girshick R. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015. 1440−1448
[97]	Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137−1149 doi: 10.1109/TPAMI.2016.2577031
[98]	He K M, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2980−2988
[99]	Cai Z W, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 6154−6162
[100]	Guo F, Qian Y, Rizos D, Suo Z, Chen X B. Automatic rail surface defects inspection based on mask R-CNN. Transportation Research Record: Journal of the Transportation Research Board, 2021, 2675(11): 655−668 doi: 10.1177/03611981211019034
[101]	Xu Y Y, Li D W, Xie Q, Wu Q Y, Wang J. Automatic defect detection and segmentation of tunnel surface using modified mask R-CNN. Measurement, 2021, 178: Article No. 109316 doi: 10.1016/j.measurement.2021.109316
[102]	Fang J T, Tan X Y, Wang Y H. ACRM: Attention cascade R-CNN with Mix-NMS for metallic surface defect detection. In: Proceedings of the 25th International Conference on Pattern Recognition (ICPR). Milan, Italy: IEEE, 2021. 423−430
[103]	Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016. 779−788
[104]	Bochkovskiy A, Wang C Y, Liao H Y M. YOLOv4: Optimal speed and accuracy of object detection. arXiv preprint arXiv: 2004.10934, 2020.
[105]	Li M J, Wang H, Wan Z B. Surface defect detection of steel strips based on improved YOLOv4. Computers and Electrical Engineering, 2022, 102: Article No. 108208 doi: 10.1016/j.compeleceng.2022.108208
[106]	Usamentiaga R, Lema D G, Pedrayes O D, Garcia D F. Automated surface defect detection in metals: A comparative review of object detection and semantic segmentation using deep learning. IEEE Transactions on Industry Applications, 2022, 58(3): 4203−4213 doi: 10.1109/TIA.2022.3151560
[107]	Xu Y H, Wang X J, Li S Y. Track surface defect detection based on EfficientDet. In: Proceedings of the 5th International Conference on Electrical Engineering and Information Technologies for Rail Transportation (EITRT) 2021: Rail Transportation Information Processing and Operational Management Technologies. Singapore: Springer, 2022. 56−66
[108]	Wang C Y, Bochkovskiy A, Liao H Y M. Scaled-YOLOv4: Scaling cross stage partial network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville, USA: IEEE, 2021. 13024−13033
[109]	He Y, Song K C, Meng Q G, Yan Y H. An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE Transactions on Instrumentation and Measurement, 2020, 69(4): 1493−1504 doi: 10.1109/TIM.2019.2915404
[110]	Anthony A, Ho E S L, Woo W L, Gao B. A review and benchmark on state-of-the-art steel defects detection [Online], available: http://dx.doi.org/10.2139/ssrn.4121951, March 19, 2023
[111]	Song G L, Liu Y, Wang X G. Revisiting the sibling head in object detector. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 11560−11569
[112]	Lin T Y, Goyal P, Girshick R, He K M, Dollár P. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2999−3007
[113]	Ma P F, Ma J, Wang X J, Yang L C, Wang N N. Deformable convolutional networks for multi-view 3D shape classification. Electronics Letters, 2018, 54(24): 1373−1375 doi: 10.1049/el.2018.6851
[114]	Law H, Deng J. CornerNet: Detecting objects as paired keypoints. In: Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018. 765−781
[115]	Ge Z, Liu S T, Wang F, Li Z M, Sun J. YOLOX: Exceeding YOLO series in 2021. arXiv preprint arXiv: 2107.08430, 2021.
[116]	Zhu B J, Wang J F, Jiang Z K, Zong F H, Liu S T, Li Z M, et al. AutoAssign: Differentiable label assignment for dense object detection. arXiv preprint arXiv: 2007.03496, 2020.
[117]	Lu H T, Fang M Y, Qiu Y, Xu W Q. An anchor-free defect detector for complex background based on pixelwise adaptive multiscale feature fusion. IEEE Transactions on Instrumentation and Measurement, 2023, 72: Article No. 5002312
[118]	Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S. End-to-end object detection with transformers. In: Proceedings of the 16th European Conference on Computer Vision (ECCV). Glasgow, UK: Springer, 2020. 213−229
[119]	Zhu X, Su W, Lu L, Li B, Wang X, Dai J. Deformable detr: Deformable transformers for end-to-end object detection. arXiv preprint arXiv: 2010.04159, 2020.
[120]	Misra I, Girdhar R, Joulin A. An end-to-end transformer model for 3D object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Montreal, Canada: IEEE, 2021. 2886−2897
[121]	Lv X, Duan F, Jiang J, Fu X, Gan L. Deep metallic surface defect detection: The new benchmark and detection network. Sensors, 2020, 20(6): Article No. 1562
[122]	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE, 2015. 3431−3440
[123]	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Munich, Germany: Springer, 2015. 234−241
[124]	Sabet D N, Zarifi M R, Khoramdel J, Borhani Y, Najafi E. An automated visual defect segmentation for flat steel surface using deep neural networks. In: Proceedings of the 12th International Conference on Computer and Knowledge Engineering (ICCKE). Mashhad, Islamic Republic of Iran: IEEE, 2022. 423−427
[125]	Lin T Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 2117−2125
[126]	Xie S N, Girshick R, Dollár P, Tu Z W, He K M. Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 5987−5995
[127]	Koonce B. EfficientNet. Convolutional Neural Networks With Swift for Tensorflow: Image Recognition and Dataset Categorization. Berkeley: Apress, 2021. 109−123
[128]	Chen L C, Papandreou G, Kokkinos I, Murphy K, Yuille A L. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834−848 doi: 10.1109/TPAMI.2017.2699184
[129]	Zheng S, Jayasumana S, Romera-Paredes B, Vineet V, Su Z Z, Du D L, et al. Conditional random fields as recurrent neural networks. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015. 1529−1537
[130]	Ma L F, Li J. SD-GCN: Saliency-based dilated graph convolution network for pavement crack extraction from 3D point clouds. International Journal of Applied Earth Observation and Geoinformation, 2022, 111: Article No. 102836 doi: 10.1016/j.jag.2022.102836
[131]	Huang Y B, Qiu C Y, Guo Y, Wang X N, Yuan K. Surface defect saliency of magnetic tile. In: Proceedings of the 14th International Conference on Automation Science and Engineering (CASE). Munich, Germany: IEEE, 2018. 612−617
[132]	Tian H, Li F. Autoencoder-based fabric defect detection with cross-patch similarity. In: Proceedings of the 16th International Conference on Machine Vision Applications (MVA). Tokyo, Japan: IEEE, 2019. 1−6
[133]	Mei S, Yang H, Yin Z P. An unsupervised-learning-based approach for automated defect inspection on textured surfaces. IEEE Transactions on Instrumentation and Measurement, 2018, 67(6): 1266−1277 doi: 10.1109/TIM.2018.2795178
[134]	Huang C Q, Cao J K, Ye F, Li M S, Zhang Y, Lu C W. Inverse-transform Autoencoder for anomaly detection. arXiv preprint arXiv: 1911.10676, 2019.
[135]	Zimmerer D, Petersen J, Kohl S A A, Maier-Hein K H. A case for the score: Identifying image anomalies using variational autoencoder gradients. arXiv preprint arXiv: 1912.00003, 2019.
[136]	Kwon G, Prabhushankar M, Temel D, AlRegib G. Backpropagated gradient representations for anomaly detection. In: Proceedings of the 16th European Conference on Computer Vision (ECCV). Glasgow, UK: Springer, 2020. 206−226
[137]	Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial networks. Communications of the ACM, 2020, 63(11): 139−144 doi: 10.1145/3422622
[138]	Baur C, Denner S, Wiestler B, Navab N, Albarqouni S. Autoencoders for unsupervised anomaly segmentation in brain MR images: A comparative study. Medical Image Analysis, 2021, 69: Article No. 101952 doi: 10.1016/j.media.2020.101952
[139]	Schlegl T, Seeböck P, Waldstein S M, Langs G, Schmidt-Erfurth U. f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Analysis, 2019, 54: 30−44 doi: 10.1016/j.media.2019.01.010
[140]	Rudolph M, Wehrbein T, Rosenhahn B, Wandt B. Fully convolutional cross-scale-flows for image-based defect detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA: IEEE, 2022. 1829−1838
[141]	Yu J W, Zheng Y, Wang X, Li W, Wu Y S, Zhao R, et al. FastFlow: Unsupervised anomaly detection and localization via 2D normalizing flows. arXiv preprint arXiv: 2111.07677, 2021.
[142]	Yi J H, Yoon S. Patch SVDD: Patch-level SVDD for anomaly detection and segmentation. In: Proceedings of the 15th Asian Conference on Computer Vision (ACCV). Kyoto, Japan: Springer, 2021. 375−390
[143]	Tax D M J, Duin R P W. Support vector data description. Machine Learning, 2004, 54(1): 45−66 doi: 10.1023/B:MACH.0000008084.60811.49
[144]	Yang M H, Wu P, Feng H. MemSeg: A semi-supervised method for image surface defect detection using differences and commonalities. Engineering Applications of Artificial Intelligence, 2023, 119: Article No. 105835 doi: 10.1016/j.engappai.2023.105835
[145]	Roth K, Pemula L, Zepeda J, Schölkopf B, Brox T, Gehler P. Towards total recall in industrial anomaly detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). New Orleans, USA: IEEE, 2022. 14298−14308
[146]	Bae J, Lee J H, Kim S. Image anomaly detection and localization with position and neighborhood information. arXiv preprint arXiv: 2211.12634, 2022.
[147]	Bergmann P, Fauser M, Sattlegger D, Steger C. Uninformed students: Student-teacher anomaly detection with discriminative latent embeddings. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 4182−4191
[148]	Batzner K, Heckler L, König R. EfficientAD: Accurate visual anomaly detection at millisecond-level latencies. arXiv preprint arXiv: 2303.14535, 2023.