2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于轻量化重构网络的表面缺陷视觉检测

余文勇 张阳 姚海明 石绘

余文勇, 张阳, 姚海明, 石绘. 基于轻量化重构网络的表面缺陷视觉检测. 自动化学报, 2020, 41(x): 1−12 doi: 10.16383/j.aas.c200535
引用本文: 余文勇, 张阳, 姚海明, 石绘. 基于轻量化重构网络的表面缺陷视觉检测. 自动化学报, 2020, 41(x): 1−12 doi: 10.16383/j.aas.c200535
Yu Wen-Yong, Zhang Yang, Yao Hai-Ming, Shi Hui. Visual inspection of surface defects based on lightweight reconstruction network. Acta Automatica Sinica, 2020, 41(x): 1−12 doi: 10.16383/j.aas.c200535
Citation: Yu Wen-Yong, Zhang Yang, Yao Hai-Ming, Shi Hui. Visual inspection of surface defects based on lightweight reconstruction network. Acta Automatica Sinica, 2020, 41(x): 1−12 doi: 10.16383/j.aas.c200535

基于轻量化重构网络的表面缺陷视觉检测

doi: 10.16383/j.aas.c200535
基金项目: 国家自然科学基金 (51775214)资助
详细信息
    作者简介:

    余文勇:华中科技大学机械科学与工程学院副教授. 2004 年获得华中科技大学博士学位. 主要研究方向为机器视觉, 智能机器人和缺陷检测. E-mail: ywy@hust.edu.cn

    张阳:2020年获华中科技大学硕士学位. 主要研究方向为机器视觉和缺陷检测. E-mail: ywy20052006@126.com

    姚海明:主要研究方向为机器视觉和缺陷检测. E-mail: weyoung@hust.edu.cn

    石绘:武汉理工大学博士研究生, 2001年获华中科技大学硕士学位. 主要研究方向为机器视觉和缺陷检测. 本文通信作者. E-mail: clove_shi@163.com

Visual Inspection of Surface Defects Based on Lightweight Reconstruction Network

Funds: Supported by National Natural Science Foundation of China (51775214)
  • 摘要: 基于深度学习的方法在某些工业产品的表面缺陷识别和分类方面表现出优异的性能, 然而大多数工业产品缺陷样本稀缺, 而且特征差异大, 导致这类需要大量缺陷样本训练的检测方法难以适用. 本文提出一种基于重构网络的无监督缺陷检测算法(Reconstruction network for defects detection, ReNet-D), 仅使用容易大量获得的无缺陷样本数据实现对异常缺陷的检测. 本文提出的算法包括两个阶段: 图像重构网络训练阶段和表面缺陷区域检测阶段. 训练阶段通过一种轻量化结构的全卷积自编码器设计重构网络, 仅使用少量正常样本进行训练, 使得重构网络能够生成无缺陷重构图像, 进一步提出一种结合结构性损失和L1损失的函数作为重构网络的损失函数, 解决自编码器检测算法对不规则纹理表面缺陷检测效果较差的问题; 缺陷检测阶段以重构图像与待测图像的残差作为缺陷的可能区域, 通过常规图像操作即可实现缺陷的定位. 本文对所提出的ReNet-D方法的网络结构、训练像素块(patch)大小、损失函数系数等影响因素进行了详细的实验分析, 并在多个缺陷图像样本集上与其他同类算法做了对比, 结果表明ReNet-D有较强的鲁棒性和准确性. 由于ReNet-D的轻量化结构, 检测1024x1024像素大小的图像仅仅耗时2.82 ms, 适合工业在线检测.
  • 图  1  各种表面缺陷. (a) 暗缺陷. (b) 明缺陷. (c) 覆盖图像的大尺度缺陷. (d) 微小缺陷. (e) 色差小的缺陷. (f)~(g) 与纹理相似的缺陷. (h) 模糊缺陷

    Fig.  1  Various surface defects. (a) Dark defects. (b) Bright defects. (c) Large-scale defects covering the image. (d) Minor defects. (e) Defects with small color difference. (f)~(g) Defects similar to texture. (h) Fuzzy defects.

    图  2  ReNet-D算法模型

    Fig.  2  ReNet-D algorithm model

    图  3  ReNet-D网络结构

    Fig.  3  ReNet-D network structure

    图  4  残差图处理流程. (a) 输入模型的原图. (b) ReNet-D重构图. (c) 由公式(9)得到的残差图. (d) 残差图滤波. (e) 缺陷定位(红色区域)

    Fig.  4  The residual graph processing flow. (a) The original input image of the model. (b) Reconstruction map by ReNet-D. (c) The residual image obtained by formula (9). (d) Filtered residual map. (e) Defect location (red area).

    图  5  实验采用的表面缺陷数据集. (a) AITEX数据集. (b)~(e) DAGM2007数据集. (f) Kylberg Sintorn数据集

    Fig.  5  Surface defect data set used in the experiment. (a) AITEX data set. (b)~(e) DAGM2007 data set. (f) Kylberg Sintorn data set.

    图  6  不同损失函数下ReNet-D的检测结果. (a) 不规则纹理样本. (b) 规则纹理样本. (c) 样本(a)在不同损失函数下的收敛测试. (d) 样本(b)在不同损失函数下的收敛测试

    Fig.  6  ReNet-D detection results under different loss functions. (a) Irregular texture samples. (b) Regular texture samples. (c) Convergence test under loss function of sample(a). (d) Convergence test under loss function of sample(b).

    图  7  不同权重系数下ReNet-D性能比较. (a) 残差热力图对比. (b)训练LOSS曲线比较

    Fig.  7  Comparison of ReNet-D performance under different weight coefficients. (a) Comparison of residual heat maps. (b) Comparison of training LOSS curves

    图  8  不同特征提取网络下ReNet-D的残差图对比

    Fig.  8  Comparison of residual maps of ReNet-D under different feature extraction networks

    图  9  不同Patch size下ReNet-D的残差图和检测结果对比. (a) 不规则纹理样本. (b) 规则纹理样本. (c)不规则纹理收敛趋势比较. (d)规则纹理收敛趋势比较

    Fig.  9  Comparison of the residual image and detection results of ReNet-D under different patch sizes. (a) Irregular texture samples. (b) Regular texture samples. (c) Comparison of irregular texture convergence trends. (d) Comparison of regular texture convergence trends

    图  10  无监督样本的测试结果

    Fig.  10  Test results of unsupervised samples

    图  11  多种算法测试效果对比

    Fig.  11  Comparison of test results of multiple algorithms.

    表  1  计算机系统配置

    Table  1  Computer system configuration

    系统Ubantu16.04
    内存128G
    GPUNVIDIA GTX-1080Ti
    CPUIntel E5-2650v4@2.2GHz
    深度学习框架Pytorch, CUDA 9.0, CUDNN 5.1
    下载: 导出CSV

    表  2  默认网络参数

    Table  2  Default network parameters

    Patch size32x32
    Batch size256
    迭代步数1000
    损失函数权重α0.15
    下载: 导出CSV

    表  3  不同损失函数下检测结果的比较. (A-不规则纹理样本, B-规则纹理样本)

    Table  3  Comparison of test results under different loss functions. (A-irregular texture sample, B-regular texture sample)

    损失函数
    指标, 样本
    L1MSEMSE+SSIML1+SSIMSSIM
    RecallA0.510.380.50.750.59
    B0.760.700.670.710.59
    PrecisionA0.930.350.520.890.93
    B0.840.650.700.870.96
    F1-MeasureA0.660.360.510.820.72
    B0.800.670.690.780.73
    下载: 导出CSV

    表  4  不同权重系数下的检测结果比较

    Table  4  Comparison of test results under different weight coefficients

    权重系数
    指标
    00.150.250.350.450.550.650.750.851
    Recall0.720.790.620.730.650.670.520.550.720.45
    Precision0.710.690.580.280.460.530.230.890.540.62
    F1-Measure0.710.730.600.410.540.600.320.680.620.52
    下载: 导出CSV

    表  5  不同Patch size下的检测结果比较. (A-不规则纹理样本, B-规则纹理样本)

    Table  5  Comparison of test results under different patch sizes. (A-irregular texture sample, B-regular texture sample)

    Patch size
    指标,样本
    16×1632×3264×64
    RecallA0.820.670.40
    B0.830.640.53
    PrecisionA0.640.860.77
    B0.530.890.74
    F1-MeasureA0.720.760.52
    B0.640.750.62
    下载: 导出CSV

    表  6  无监督样本的测试结果

    Table  6  Test results of unsupervised samples

    指标
    样本
    RecallPrecisionF1-Measure
    油污0.710.940.80
    破损0.660.480.55
    磨痕0.630.890.70
    涂抹0.270.470.32
    胶带0.160.350.20
    下载: 导出CSV

    表  7  不同算法的检测效果比较

    Table  7  Comparison of detection effects of different algorithms

    算法
    指标, 样本
    LCAPHOTMSCDAEReNet-D
    RecallA0.6630.1550.5620.772
    B0.1170.3410.9660.707
    C0.4780.1330.2030.799
    D0.5610.6100.3580.659
    E0.6120.3180.3590.946
    F0.6410.4140.8810.948
    PrecisionA0.4360.3240.4630.884
    B0.0020.4780.4440.793
    C0.0240.1120.1430.855
    D0.6390.2990.6420.940
    E0.4120.3670.6960.824
    F0.8990.0060.9200.935
    F1-MeasureA0.5260.2100.5080.824
    B0.0040.3980.6080.732
    C0.0450.1220.1680.822
    D0.5970.4010.4600.771
    E0.4920.3410.6620.881
    F0.7480.0120.9000.941
    下载: 导出CSV

    表  8  处理耗时的比较

    Table  8  Comparison of processing time

    检测方法PhotLCAMSCDAEReNet-D
    耗时(ms)4504309746.592.82
    下载: 导出CSV
  • [1] Deng S, Cai W, Xu Q, Liang B. Defect detection of bearing surfaces based on machine vision technique. In: 2010 International Conference on Computer Application and System Modeling (ICCASM 2010), USA: IEEE Press, Vol. 4, pp. V4−548
    [2] Jian C, Gao J, Ao Y. Automatic surface defect detection for mobile phone screen glass based on machine vision. Applied Soft Computing, 2017, 52: 348−358 doi: 10.1016/j.asoc.2016.10.030
    [3] Bulnes F G, Usamentiaga R, Garcia D F, Molleda J. An efficient method for defect detection during the manufacturing of web materials. Journal of Intelligent Manufacturing, 2016, 27(2): 431−445 doi: 10.1007/s10845-014-0876-9
    [4] 金侠挺, 王耀南, 张辉, 刘理, 钟杭, 贺振东. 基于贝叶斯 CNN 和注意力网络的钢轨表面缺陷检测系统. 自动化学报, 2019, 45(12): 2312−2327

    Jin XiaTing, Wang YaoNan, Zhang Hui, Liu Li, Zhong Hang, He ZhenDong. DeepRail: automatic visual detection system for railway surface defect using Bayesian CNN and attention network. Acta Automatica Sinica, 2019, 45(12): 2312−2327
    [5] 李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727−1742

    Li Liang-Fu, Ma Wei-Fei, Li Li, Lu Cheng. Research on detection algorithm for bridge cracks based on deep learning. Acta Automatica Sinica, 2019, 45(9): 1727−1742
    [6] Chen S, Hu T, Liu G, Pu Z, Li M, Du L. (2008, May). Defect classification algorithm for IC photomask based on PCA and SVM. In: 2008 Congress on Image and Signal Processing, USA: IEEE Press, Vol. 1, pp. 491−496.
    [7] Jiexian H, Di L, Feng, Y. Detection of surface of solder on flexible circuit. Optics and Precision Engineering, 2010, 18(11): 2443−2453
    [8] Krizhevsky A, Sutskever I, Hinton, G. E. Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems 2012, pp. 1097−1105.
    [9] Napoletano P, Piccoli F, Schettini, R. Anomaly detection in nanofibrous materials by CNN-based self-similarity. Sensors, 2018, 18(1): 209 doi: 10.1109/JSEN.2017.2771313
    [10] Cha Y J, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O. Autonomous structural visual inspection using region‐based deep learning for detecting multiple damage types. Computer‐Aided Civil and Infrastructure Engineering, 2018, 33(9): 731−747 doi: 10.1111/mice.12334
    [11] Gao Y, Gao L, Li X, Yan X. A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robotics and Computer-Integrated Manufacturing, 2020, 61: 101825 doi: 10.1016/j.rcim.2019.101825
    [12] Zhao Z, Xu G, Qi Y, Liu N, Zhang T. Multi-patch deep features for power line insulator status classification from aerial images. In: International Joint Conference on Neural Networks (IJCNN) IEEE Press, 2016.pp. 3187−3194
    [13] Wang T, Chen Y, Qiao M, Snoussi H. A fast and robust convolutional neural network-based defect detection model in product quality control. The International Journal of Advanced Manufacturing Technology, 2018, 94(9-12): 3465−3471 doi: 10.1007/s00170-017-0882-0
    [14] Xu X, Zheng H, Guo Z, Wu X, Zheng Z. SDD-CNN: Small Data-Driven Convolution Neural Networks for Subtle Roller Defect Inspection. Applied Sciences, 2019, 9(7): 1364 doi: 10.3390/app9071364
    [15] Paolo N, Flavio P, Raimondo S. Anomaly Detection in Nanofibrous Materials by CNN-Based Self-Similarity. Sensors, 2018, 18(2): 209−216 doi: 10.3390/s18010209
    [16] Weimer D, Scholz-Reiter B, Shpitalni M. Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection. CIRP Annals. Manufacturing Technology, 2016, 65(1): 417−420 doi: 10.1016/j.cirp.2016.04.072
    [17] Girshick R, Donahue J, Darrell T, et al. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]. v, 2014: 580−587.
    [18] Liu W, Anguelov D, Erhan D, et al. SSD: Single Shot MultiBox Detector[C].In: European Conference on Computer Vision. Springer International Publishing, 2016: pp 21−37.
    [19] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You only look once: Unified, real-time object detection, ” in Proc. In: IEEE Conf. Comput. Vis. Pattern Recognit. Jun. 2016, pp. 779−788
    [20] Chen J, Liu Z, Wang H, Núñez A, Han, Z. Automatic defect detection of fasteners on the catenary support device using deep convolutional neural network. IEEE Transactions on Instrumentation and Measurement, 2017, 67(2): 257−269
    [21] Cha Y J, Choi W, Suh G, et al. Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types. Computer Aided Civil & Infrastructure Engineering, 2017, 00(4): 1−17
    [22] Huang Y, Qiu C, Guo Y, Wang X, Yuan, K. (2018, August). Surface defect saliency of magnetic tile. In: 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE).IEEE Press: pp. 612−617.
    [23] Lingteng, Qiu, Xiaojun, et al. A High-Efficiency Fully Convolutional Networks for Pixel-Wise Surface Defect Detection. IEEE Access, 2019: 15884−15893
    [24] Yu W, Zhang Y, Shi H. Surface Defect Inspection Under a Small Training Set Condition. In: International Conference on Intelligent Robotics and Applications. Springer, Cham. 2019, August. pp. 517−528.
    [25] Masci J, Meier U, Cireşan D, Schmidhuber J. Stacked convolutional auto-encoders for hierarchical feature extraction. In International Conference on Artificial Neural Networks. Berlin, Heidelberg. Springer, 2011, June. pp. 52−59
    [26] Li Y, Zhao W, Pan J. Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Transactions on Automation Science and Engineering, 2016, 14(2): 1256−1264
    [27] Chalapathy R, Menon A K, Chawla S. Robust, Deep and Inductive Anomaly Detection.In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases: Springer, 2017.36−51.
    [28] 袁静, 章毓晋. 融合梯度差信息的稀疏去噪自编码网络在异常行为检测中的应用. 自动化学报, 2017, 43(4): 604−610

    Yuan Jing, Zhang YuJin. Application of sparse denoising autoencoder network with gradient difference information for abnormal action detection. Acta Automatica Sinica, 2017, 43(4): 604−610
    [29] Mei S, Yang H, Yin Z. An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces. IEEE Transactions on Instrumentation and Measurement, 2018: 1−12
    [30] D. Aiger and H. Talbot, “The phase only transform for unsupervised surface defect detection, ”.In Proc. IEEE Conf. Comput. Vis. Pattern Recognit., IEEE press, Jun. 2010, pp. 295−302.
    [31] H.-D. Lin. Tiny surface defect inspection of electronic passive components using discrete cosine transform decomposition and cumulative sum techniques. Image Vis. Comput, 2008, 26(5): 603−621 doi: 10.1016/j.imavis.2007.07.009
    [32] Yang H, Chen Y, Song K, Yin Z. Multiscale Feature-Clustering-Based Fully Convolutional Autoencoder for Fast Accurate Visual Inspection of Texture Surface Defects. IEEE Transactions on Automation Science and Engineering. 2019, 1−18.
    [33] Makhzani A, Shlens J, Jaitly N, Goodfellow I, Frey B. (2015). Adversarial autoencoders. arXiv preprint arXiv: 1511.05644.
    [34] Zhao Z, Li B, Dong R, et al. A Surface Defect Detection Method Based on Positive Samples[C]. International Conference on Artificial Intelligence, Pacific Rim, Springer, Cham, 2018: 473−481.
    [35] Di He, et al. "Surface defect classification of steels with a new semi-supervised learning method." Optics and Lasers in Engineering, 2019, 117: 40−48.
    [36] Schlegl T, Seeböck P, Waldstein, S M, Schmidt-Erfurth U, Langs, G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In International Conference on Information Processing in Medical Imaging, Springer, Cham. 2017, June.pp. 146−157.
    [37] Paul Bergmann, Michael Fauser, David Sattlegger, Carsten Steger; A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), USA: IEEE Press: 2019.pp. 9592−9600
    [38] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv: 1502.03167. 2015.
    [39] Zhao H, Gallo O, Frosio I, Kautz J. Loss functions for image restoration with neural networks. IEEE Transactions on Computational Imaging, 2016, 3(1): 47−57
    [40] Bergmann P, Löwe S, Fauser M, et al. Improving unsupervised defect segmentation by applying structural similarity to autoencoders[J]. arXiv preprint arXiv: 1807.02011
    [41] Lv C, Zhang Z, Shen F, et al. A Fast Surface Defect Detection Method Based on Background Reconstruction[J].In: International Journal of Precision Engineering and Manufacturing, 2019: 1−13.
    [42] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 2004, 13(4): 600−612 doi: 10.1109/TIP.2003.819861
    [43] Silvestre-Blanes J, Albero-Albero T, Miralles I, Pérez-Llorens R, Moreno J. A Public Fabric Database for Defect Detection Methods and Results. Autex Research Journal, 2019, 19(4): 363−374 doi: 10.2478/aut-2019-0035
    [44] Jager M, Knoll C, Hamprecht F A. Weakly supervised learning of a classifier for unusual event detection. IEEE Transactions on Image Processing, 2019, 17(9): 1700−1708
    [45] G. Kylberg and I. Sintorn. (2013). Kylberg Sintorn RotationDataset.[Online]. Available: http://www.cb.uu.se/~gustaf/KylbergSintornRotation/
    [46] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2015. pp. 3431−3440.
    [47] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention Springer, Cham. 2015, October. pp. 234−241.
    [48] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. USA: IEEE Press, 2016.pp. 770−778.
    [49] Shi W, Caballero J, Huszár F, Totz J, Aitken A P, Bishop R, ... Wang, Z. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition USA: IEEE Press, 2016. pp. 1874−1883.
    [50] Huang G, Liu Z, Van Der Maaten L, Weinberger, K. Q.Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. USA: IEEE Press, 2017. pp. 4700−4708.
    [51] Tsai D M, Huang T Y. Automated surface inspection for statistical textures. Image and Vision computing, 2003, 21(4): 307−323 doi: 10.1016/S0262-8856(03)00007-6
  • 加载中
计量
  • 文章访问数:  510
  • HTML全文浏览量:  130
  • 被引次数: 0
出版历程

目录

    /

    返回文章
    返回