2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于轻量化重构网络的表面缺陷视觉检测

余文勇 张阳 姚海明 石绘

余文勇, 张阳, 姚海明, 石绘. 基于轻量化重构网络的表面缺陷视觉检测. 自动化学报, 2022, 48(9): 2175−2186 doi: 10.16383/j.aas.c200535
引用本文: 余文勇, 张阳, 姚海明, 石绘. 基于轻量化重构网络的表面缺陷视觉检测. 自动化学报, 2022, 48(9): 2175−2186 doi: 10.16383/j.aas.c200535
Yu Wen-Yong, Zhang Yang, Yao Hai-Ming, Shi Hui. Visual inspection of surface defects based on lightweight reconstruction network. Acta Automatica Sinica, 2022, 48(9): 2175−2186 doi: 10.16383/j.aas.c200535
Citation: Yu Wen-Yong, Zhang Yang, Yao Hai-Ming, Shi Hui. Visual inspection of surface defects based on lightweight reconstruction network. Acta Automatica Sinica, 2022, 48(9): 2175−2186 doi: 10.16383/j.aas.c200535

基于轻量化重构网络的表面缺陷视觉检测

doi: 10.16383/j.aas.c200535
基金项目: 国家自然科学基金 (51775214)资助
详细信息
    作者简介:

    余文勇:华中科技大学机械科学与工程学院副教授. 2004 年获得华中科技大学博士学位. 主要研究方向为机器视觉, 智能机器人和缺陷检测. E-mail: ywy@hust.edu.cn

    张阳:华中科技大学机械科学与工程学院博士研究生. 2020年获华中科技大学硕士学位. 主要研究方向为机器视觉和缺陷检测. E-mail: ywy20052006@126.com

    姚海明:华中科技大学机械科学与工程学院博士研究生. 主要研究方向为机器视觉和缺陷检测. E-mail: weyoung@hust.edu.cn

    石绘:武汉理工大学博士研究生. 2001年获华中科技大学硕士学位. 主要研究方向为机器视觉和缺陷检测. 本文通信作者. E-mail: clove_shi@163.com

Visual Inspection of Surface Defects Based on Lightweight Reconstruction Network

Funds: Supported by National Natural Science Foundation of China (51775214)
More Information
    Author Bio:

    YU Wen-Yong Associate professor at the School of Mechanical Science and Engineering, Huazhong University of Science and Technology. He received his Ph.D. degree from Huazhong University of Science and Technology in 2004. His research interest covers machine vision, intelligent robot and defect inspection

    ZHANG Yang Ph.D. candidate at the School of Mechanical Science and Engineering, Huazhong University of Science and Technology. He received his master degree from Huazhong University Science and Technology in 2020. His research interest covers machine vision and defect detection

    YAO Hai-Ming Ph.D. candidate at the School of Mechanical Science and Engineering, Huazhong University of Science and Technology. His research interest covers machine vision and defect detection

    SHI Hui Ph.D. candidate at the School of Mechanical and Electronic Engineering, Wuhan University of Technology. She received her master degree from Huazhong University of Science and Technology in 2001. Her research interest covers machine vision and defect detection. Corresponding author of this paper

  • 摘要: 基于深度学习的方法在某些工业产品的表面缺陷识别和分类方面表现出优异的性能, 然而大多数工业产品缺陷样本稀缺, 而且特征差异大, 导致这类需要大量缺陷样本训练的检测方法难以适用. 提出一种基于重构网络的无监督缺陷检测算法, 仅使用容易大量获得的无缺陷样本数据实现对异常缺陷的检测. 提出的算法包括两个阶段: 图像重构网络训练阶段和表面缺陷区域检测阶段. 训练阶段通过一种轻量化结构的全卷积自编码器设计重构网络, 仅使用少量正常样本进行训练, 使得重构网络能够生成无缺陷重构图像, 进一步提出一种结合结构性损失和L1损失的函数作为重构网络的损失函数, 解决自编码器检测算法对不规则纹理表面缺陷检测效果较差的问题; 缺陷检测阶段以重构图像与待测图像的残差作为缺陷的可能区域, 通过常规图像操作即可实现缺陷的定位. 对所提出的重构网络的无监督缺陷检测算法的网络结构、训练像素块大小、损失函数系数等影响因素进行了详细的实验分析, 并在多个缺陷图像样本集上与其他同类算法做了对比, 结果表明重构网络的无监督缺陷检测算法有较强的鲁棒性和准确性. 由于重构网络的无监督缺陷检测算法的轻量化结构, 检测1024 × 1024像素图像仅仅耗时2.82 ms, 适合工业在线检测.
  • 图  1  各种表面缺陷

    Fig.  1  Various surface defects

    图  2  ReNet-D算法模型

    Fig.  2  ReNet-D algorithm model

    图  3  ReNet-D网络结构

    Fig.  3  ReNet-D network structure

    图  4  残差图处理流程

    Fig.  4  The residual graph processing flow location

    图  5  实验采用的表面缺陷数据集

    Fig.  5  Surface defect data set used in the experiment

    图  6  不同损失函数下ReNet-D的检测结果

    Fig.  6  ReNet-D detection results under different loss functions

    图  7  不同权重系数下ReNet-D性能比较

    Fig.  7  Comparison of ReNet-D performances under different weight coefficients

    图  8  不同特征提取网络下ReNet-D的残差图对比

    Fig.  8  Comparison of residual maps of ReNet-D under different feature extraction networks

    图  9  不同图像块下ReNet-D的残差图和检测结果对比

    Fig.  9  Comparison of the residual image and detection results of ReNet-D under different patch sizes of texture samples

    图  10  无监督样本的测试结果

    Fig.  10  Test results of unsupervised samples

    图  11  多种算法测试效果对比

    Fig.  11  Comparison of test results of multiple algorithms

    表  1  计算机系统配置

    Table  1  Computer system configuration

    系统Ubantu 16.04
    内存128 GB
    图形处理器NVIDIA GTX-1080 Ti
    中央处理器Intel E5-2650 v4@2.2 GHz
    深度学习框架Pytorch, CUDA 9.0, CUDNN 5.1
    下载: 导出CSV

    表  2  默认网络参数

    Table  2  Default network parameters

    像素块尺寸32$\, \times \,$32
    样本数256
    迭代步数1000
    损失函数权重$\alpha $0.15
    下载: 导出CSV

    表  3  不同损失函数下检测结果的比较

    Table  3  Comparison of test results under different loss functions

    指标 样本L1MSEMSE + SSIML1 + SSIMSSIM
    召回率不规则纹理样本0.510.380.50.750.59
    规则纹理样本0.760.700.670.710.59
    精准率不规则纹理样本0.930.350.520.890.93
    规则纹理样本0.840.650.700.870.96
    加权调和平均不规则纹理样本0.660.360.510.820.72
    规则纹理样本0.800.670.690.780.73
    下载: 导出CSV

    表  4  不同权重系数下的检测结果比较

    Table  4  Comparison of test results under different weight coefficients

    权重系数
    00.150.250.350.450.550.650.750.851
    召回率0.720.790.620.730.650.670.520.550.720.45
    精准率0.710.690.580.280.460.530.230.890.540.62
    加权调和平均0.710.730.600.410.540.600.320.680.620.52
    下载: 导出CSV

    表  5  不同尺寸像素块的检测结果比较

    Table  5  Comparison of test results under different patch sizes texture samples

    指标 样本16 × 1632 × 3264 × 64
    召回率不规则纹理样本0.820.670.40
    规则纹理样本0.830.640.53
    精准率不规则纹理样本0.640.860.77
    规则纹理样本0.530.890.74
    加权调和平均不规则纹理样本0.720.760.52
    规则纹理样本0.640.750.62
    下载: 导出CSV

    表  6  无监督样本的测试结果

    Table  6  Test results of unsupervised samples

    样本
    召回率精准率加权调和平均
    油污0.710.940.80
    破损0.660.480.55
    磨痕0.630.890.70
    涂抹0.270.470.32
    胶带0.160.350.20
    下载: 导出CSV

    表  7  不同算法的检测效果比较

    Table  7  Comparison of detection effects of different algorithms

    指标 样本LCAPHOTMSCDAEReNet-D
    召回率图11(a)0.6630.1550.5620.772
    图11(b)0.1170.3410.9660.707
    图11(c)0.4780.1330.2030.799
    图11(d)0.5610.6100.3580.659
    图11(e)0.6120.3180.3590.946
    图11(f)0.6410.4140.8810.948
    精准率图11(a)0.4360.3240.4630.884
    图11(b)0.0020.4780.4440.793
    图11(c)0.0240.1120.1430.855
    图11(d)0.6390.2990.6420.940
    图11(e)0.4120.3670.6960.824
    图11(f)0.8990.0060.9200.935
    加权调和平均图11(a)0.5260.2100.5080.824
    图11(b)0.0040.3980.6080.732
    图11(c)0.0450.1220.1680.822
    图11(d)0.5970.4010.4600.771
    图11(e)0.4920.3410.6620.881
    图11(f)0.7480.0120.9000.941
    下载: 导出CSV

    表  8  处理耗时的比较 (ms)

    Table  8  Comparison of processing time (ms)

    检测方法PHOTLCAMSCDAEReNet-D
    耗时4504309746.592.82
    下载: 导出CSV
  • [1] Deng S, Cai W, Xu Q, Liang B. Defect detection of bearing surfaces based on machine vision technique. In: Proceedings of the 2010 International Conference on Computer Application and System Modeling (ICCASM 2010). Taiyuan, China: IEEE Press, 2010. 4−548
    [2] Jian, C., Gao, J., & Ao, Y. Automatic surface defect detection for mobile phone screen glass based on machine vision. Applied Soft Computing, 2017, 52: 348-358. doi: 10.1016/j.asoc.2016.10.030
    [3] Bulnes, F. G., Usamentiaga, R., Garcia, D. F., & Molleda, J. An efficient method for defect detection during the manufacturing of web materials. Journal of Intelligent Manufacturing, 2016.27(2): 431-445. doi: 10.1007/s10845-014-0876-9
    [4] 金侠挺, 王耀南, 张辉, 刘理, 钟杭, 贺振东. 基于贝叶斯 CNN 和注意力网络的钢轨表面缺陷检测系统. 自动化学报, 2019, 45(12): 2312-2327

    Jin XiaTing, Wang YaoNan, Zhang Hui, Liu Li, Zhong Hang, He ZhenDong. DeepRail: automatic visual detection system for railway surface defect using Bayesian CNN and attention network. Acta Automatica Sinica, 2019, 45(12): 2312-2327
    [5] 李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727-1742

    Li Liang-Fu, Ma Wei-Fei, Li Li, Lu Cheng. Research on detection algorithm for bridge cracks based on deep learning. Acta Automatica Sinica, 2019, 45(9): 1727-1742.
    [6] Chen S, Hu T, Liu G, Pu Z, Li M, Du L. Defect classification algorithm for IC photomask based on PCA and SVM. In: Proceedings of the 2008 Congress on Image and Signal Processing. Sanya, China: IEEE Press, 2008. 491−496
    [7] Jiexian, H., Di, L., & Feng, Y. Detection of surface of solder on flexible circuit[J]. Optics and Precision Engineering, 2010, 18(11):2443-2453.
    [8] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. In: Proceedings of the Advances in Neural Information Processing Systems. Toronto, Canada: 2012. 1097−1105
    [9] Napoletano, P., Piccoli, F., & Schettini, R. (2018). Anomaly detection in nanofibrous materials by CNN-based self-similarity. Sensors, 2018, 18(1): 209. doi: 10.1109/JSEN.2017.2771313
    [10] Cha, Y. J., Choi, W., Suh, G., Mahmoudkhani, S., & Büyüköztürk, O. Autonomous structural visual inspection using region‐based deep learning for detecting multiple damage types. Computer‐Aided Civil and Infrastructure Engineering, 2018, 33(9):731-747. doi: 10.1111/mice.12334
    [11] Gao, Y., Gao, L., Li, X., & Yan, X. A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robotics and Computer-Integrated Manufacturing, 2020, 61, 101825. doi: 10.1016/j.rcim.2019.101825
    [12] Zhao Z, Xu G, Qi Y, Liu N, Zhang T. Multi-patch deep features for power line insulator status classification from aerial images. In: Proceedings of the International Joint Conference on Neural Networks. Vancouver, Canada: IEEE Press, 2016. 3187− 3194
    [13] Wang, T., Chen, Y., Qiao, M., & Snoussi, H. A fast and robust convolutional neural network-based defect detection model in product quality control. The International Journal of Advanced Manufacturing Technology, 2018.94(9-12), 3465-3471. doi: 10.1007/s00170-017-0882-0
    [14] Xu, X., Zheng, H., Guo, Z., Wu, X., & Zheng, Z. SDD-CNN: Small Data-Driven Convolution Neural Networks for Subtle Roller Defect Inspection. Applied Sciences, 2019. 9(7), 1364. doi: 10.3390/app9071364
    [15] Paolo N, Flavio P, Raimondo S. Anomaly Detection in Nanofibrous Materials by CNN-Based Self-Similarity[J]. Sensors, 2018, 18(2):209-216. doi: 10.3390/s18010209
    [16] Weimer D, Scholz-Reiter B, Shpitalni M. Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection[J]. CIRP Annals. Manufacturing Technology, 2016.65(1):417-420. doi: 10.1016/j.cirp.2016.04.072
    [17] Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. IEEE Computer Society, 2014: 580−587
    [18] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, et al. SSD: Single shot multi-box detector. In: Proceedings of the European Conference on Computer Vision. Amsterdam, Netherlands: Springer International Publishing, 2016. 21−37
    [19] Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Conference on Computer Vision and Pattern Recognition. Seattle, USA: 2016. 779−788
    [20] Chen, J., Liu, Z., Wang, H., Núñez, A., & Han, Z. Automatic defect detection of fasteners on the catenary support device using deep convolutional neural network. IEEE Transactions on Instrumentation and Measurement.2017, 67(2), 257-269.
    [21] Cha Y J, Choi W, Suh G, et al. Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types[J]. Computer Aided Civil & Infrastructure Engineering, 2017, 00(4):1-17
    [22] Huang Y, Qiu C, Guo Y, Wang X, Yuan K. Surface defect saliency of magnetic tile. In: Proceedings of the 2018 IEEE 14th International Conference on Automation Science and Engineering. Munich, Germany: 2018. 612−617
    [23] Lingteng, Qiu, Xiaojun, et al. A High-Efficiency Fully Convolutional Networks for Pixel-Wise Surface Defect Detection[J]. IEEE Access, 2019: 15884-15893
    [24] Yu W, Zhang Y, Shi H. Surface defect inspection under a small training set condition. In: Proceedings of the International Conference on Intelligent Robotics and Applications. Shenyang, China: 2019, 517−528
    [25] Masci J, Meier U, Cireşan D, Schmidhuber J. Stacked convolutional auto-encoders for hierarchical feature extraction. In: Proceedings of the International Conference on Artificial Neural Networks. Berlin, Germany: Springer, 2011. 52−59
    [26] Li, Y., Zhao, W., & Pan, J. Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Transactions on Automation Science and Engineering, 2016, 14(2), 1256-1264.
    [27] Chalapathy R, Menon A K, Chawla S. Robust, deep and inductive anomaly detection. In: Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Skopje, Macedonia: Springer, 2017. 36−51
    [28] 袁静, 章毓晋. 融合梯度差信息的稀疏去噪自编码网络在异常行为检测中的应用. 自动化学报, 2017, 43(4): 604-610

    Yuan Jing, Zhang YuJin. Application of sparse denoising autoencoder network with gradient difference information for abnormal action detection. Acta Automatica Sinica, 2017, 43(4):604-610.
    [29] Mei S, Yang H, Yin Z. An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces[J]. IEEE Transactions on Instrumentation and Measurement, 2018:1-12.
    [30] Aiger D, Talbot H. The phase only transform for unsupervised surface defect detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE Press, 2010. 295−302
    [31] H.-D. Lin, “Tiny surface defect inspection of electronic passive components using discrete cosine transform decomposition and cumulative sum techniques, ” Image Vis. Comput, 2008, vol. 26, no. 5, pp. 603–621 doi: 10.1016/j.imavis.2007.07.009
    [32] Yang H, Chen Y, Song K, Yin Z. Multi-scale feature-clustering-based fully convolutional autoencoder for fast accurate visual inspection of texture surface defects. IEEE Transactions on Automation Science and Engineering. 2019, 1−18
    [33] Makhzani A, Shlens J, Jaitly N, Goodfellow I, Frey B. Adversarial autoencoders. arXiv preprint, 2015, arXiv: 1511.05644
    [34] Zhao Z, Li B, Dong R, Zhao P. A surface defect detection method based on positive samples. In: Proceedings of the International Conference on Artificial Intelligence. Nanjing, China: 2018. 473−481
    [35] He D, Xu K, Zhou P, Zhou D. Surface defect classification of steels with a new semi-supervised learning method. Optics and Lasers in Engineering, 2019, 117: 40−48
    [36] Schlegl T, Seeböck P, Waldstein S M, Schmidt-Erfurth U, Langs G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Proceedings of the International Conference on Information Processing in Medical Imaging. Boone, USA: 2017. 146−157
    [37] Paul B, Michael F, David S, Carsten S. A comprehensive real-world dataset for unsupervised anomaly detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: 2019. 9592−9600
    [38] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv prepr-int, 2015, arXiv: 1502.03167
    [39] Zhao, H., Gallo, O., Frosio, I., & Kautz, J. Loss functions for image restoration with neural networks. IEEE Transactions on Computational Imaging, 2016. 3(1), 47-57
    [40] Bergmann P, Löwe S, Fauser M, Sattlegger D, Steger C. Improving unsupervised defect segmentation by applying structural similarity to autoencoders. In: Proceedings of the 14th International Conference on Computer Vision Theory and Applicati-ons. Prague, Czecg Republic. 2019: 372−380
    [41] Lv C, Zhang Z, Shen F, Zhang F, Su H. A fast surface defect detection method based on background reconstruction. Internation-al Journal of Precision Engineering and Manufacturing, 2019: 1−13
    [42] Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 2004, 13(4):600-612. doi: 10.1109/TIP.2003.819861
    [43] Silvestre-Blanes, J., Albero-Albero, T., Miralles, I., Pérez-Llorens, R., & Moreno, J. A Public Fabric Database for Defect Detection Methods and Results. Autex Research Journal, 2019, 19(4):363-374. doi: 10.2478/aut-2019-0035
    [44] Jager, M., Knoll, C., & Hamprecht, F. A.Weakly supervised learning of a classifier for unusual event detection. IEEE Transactions on Image Processing, 2019, 17(9):1700-1708.
    [45] Kylberg G, Sintorn I. Sintorn K. Rotationdataset[Online]. Available: http://www.cb.uu.se/~gustaf/KylbergSintor-nRotation/ Fe-bruary 25, 2013.
    [46] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: 2015. 3431−3440
    [47] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-assisted Intervention. Munich, Germany: 2015. 234− 241
    [48] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 770−778
    [49] Shi W, Caballero J, Huszár F, Totz J, Aitken A P, Bishop R, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 1874−1883
    [50] Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 4700−4708
    [51] Tsai, D. M., & Huang, T. Y. Automated surface inspection for statistical textures. Image and Vision computing, 2003.21(4):307-323. doi: 10.1016/S0262-8856(03)00007-6
  • 加载中
图(11) / 表(8)
计量
  • 文章访问数:  3348
  • HTML全文浏览量:  700
  • PDF下载量:  671
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-07-11
  • 网络出版日期:  2022-09-16
  • 刊出日期:  2022-09-16

目录

    /

    返回文章
    返回