-
摘要: 基于深度学习的方法在某些工业产品的表面缺陷识别和分类方面表现出优异的性能, 然而大多数工业产品缺陷样本稀缺, 而且特征差异大, 导致这类需要大量缺陷样本训练的检测方法难以适用. 提出一种基于重构网络的无监督缺陷检测算法, 仅使用容易大量获得的无缺陷样本数据实现对异常缺陷的检测. 提出的算法包括两个阶段: 图像重构网络训练阶段和表面缺陷区域检测阶段. 训练阶段通过一种轻量化结构的全卷积自编码器设计重构网络, 仅使用少量正常样本进行训练, 使得重构网络能够生成无缺陷重构图像, 进一步提出一种结合结构性损失和L1损失的函数作为重构网络的损失函数, 解决自编码器检测算法对不规则纹理表面缺陷检测效果较差的问题; 缺陷检测阶段以重构图像与待测图像的残差作为缺陷的可能区域, 通过常规图像操作即可实现缺陷的定位. 对所提出的重构网络的无监督缺陷检测算法的网络结构、训练像素块大小、损失函数系数等影响因素进行了详细的实验分析, 并在多个缺陷图像样本集上与其他同类算法做了对比, 结果表明重构网络的无监督缺陷检测算法有较强的鲁棒性和准确性. 由于重构网络的无监督缺陷检测算法的轻量化结构, 检测1024 × 1024像素图像仅仅耗时2.82 ms, 适合工业在线检测.Abstract: Deep learning-based methods show excellent performance in identifying and classifying surface defects of certain industrial products. However, most industrial product defect samples are scarce and feature differences are large, making it difficult to apply this type of detection method that requires a large number of defect samples. This paper proposes an image reconstruction-based unsupervised defect detection algorithm reconstruction network for defects detection, which uses only non-defective sample data that is easily available in large quantities to detect abnormal defects. The algorithm proposed in this paper includes two stages: image reconstruction network training stage and surface defect area detection stage. In the training phase, the reconstruction network is designed by a fully convolutional self-encoder with a lightweight structure, and only a small number of normal samples are used for training, so that the reconstruction network can generate defect-free reconstruction images, and a combination of structural loss and L1 is further proposed. The loss function is used as the loss function of the reconstructed network to solve the problem of poor detection of irregular texture surface defects by the self-encoder detection algorithm; the residual area of the reconstructed image and the image to be tested is used as a possible defect area in the defect detection stage. The final inspection result can be obtained through conventional image operations. In this paper, the network structure, training patch size, loss function coefficient and other influencing factors of the proposed reconstruction network for defects detection method are analyzed in detail, and compared with other similar algorithms on several defect image sample sets. The results show that reconstruction network for defects detection has strong robustness and accuracy. Due to the lightweight structure of reconstruction network for defects detection, it takes only 2.82 ms to detect 1024 × 1024 pixel images, which is suitable for industrial online detection.
-
Key words:
- Defect detection /
- deep learning /
- small samples /
- fully convolutional auto encoder /
- loss function
-
图 6 不同损失函数下ReNet-D的检测结果
Fig. 6 ReNet-D detection results under different loss functions
图 7 不同权重系数下ReNet-D性能比较
Fig. 7 Comparison of ReNet-D performances under different weight coefficients
图 8 不同特征提取网络下ReNet-D的残差图对比
Fig. 8 Comparison of residual maps of ReNet-D under different feature extraction networks
图 9 不同图像块下ReNet-D的残差图和检测结果对比
Fig. 9 Comparison of the residual image and detection results of ReNet-D under different patch sizes of texture samples
表 1 计算机系统配置
Table 1 Computer system configuration
系统 Ubantu 16.04 内存 128 GB 图形处理器 NVIDIA GTX-1080 Ti 中央处理器 Intel E5-2650 v4@2.2 GHz 深度学习框架 Pytorch, CUDA 9.0, CUDNN 5.1 
下载: 导出CSV
表 2 默认网络参数
Table 2 Default network parameters
像素块尺寸 32$\, \times \,$32 样本数 256 迭代步数 1000 损失函数权重$\alpha $ 0.15
下载: 导出CSV
表 3 不同损失函数下检测结果的比较
Table 3 Comparison of test results under different loss functions
指标 样本 L1 MSE MSE + SSIM L1 + SSIM SSIM 召回率 不规则纹理样本 0.51 0.38 0.5 0.75 0.59 规则纹理样本 0.76 0.70 0.67 0.71 0.59 精准率 不规则纹理样本 0.93 0.35 0.52 0.89 0.93 规则纹理样本 0.84 0.65 0.70 0.87 0.96 加权调和平均 不规则纹理样本 0.66 0.36 0.51 0.82 0.72 规则纹理样本 0.80 0.67 0.69 0.78 0.73
下载: 导出CSV
表 4 不同权重系数下的检测结果比较
Table 4 Comparison of test results under different weight coefficients
权重系数 0 0.15 0.25 0.35 0.45 0.55 0.65 0.75 0.85 1 召回率 0.72 0.79 0.62 0.73 0.65 0.67 0.52 0.55 0.72 0.45 精准率 0.71 0.69 0.58 0.28 0.46 0.53 0.23 0.89 0.54 0.62 加权调和平均 0.71 0.73 0.60 0.41 0.54 0.60 0.32 0.68 0.62 0.52
下载: 导出CSV
表 5 不同尺寸像素块的检测结果比较
Table 5 Comparison of test results under different patch sizes texture samples
指标 样本 16 × 16 32 × 32 64 × 64 召回率 不规则纹理样本 0.82 0.67 0.40 规则纹理样本 0.83 0.64 0.53 精准率 不规则纹理样本 0.64 0.86 0.77 规则纹理样本 0.53 0.89 0.74 加权调和平均 不规则纹理样本 0.72 0.76 0.52 规则纹理样本 0.64 0.75 0.62
下载: 导出CSV
表 6 无监督样本的测试结果
Table 6 Test results of unsupervised samples
样本 召回率 精准率 加权调和平均 油污 0.71 0.94 0.80 破损 0.66 0.48 0.55 磨痕 0.63 0.89 0.70 涂抹 0.27 0.47 0.32 胶带 0.16 0.35 0.20
下载: 导出CSV
表 7 不同算法的检测效果比较
Table 7 Comparison of detection effects of different algorithms
指标 样本 LCA PHOT MSCDAE ReNet-D 召回率 图11(a) 0.663 0.155 0.562 0.772 图11(b) 0.117 0.341 0.966 0.707 图11(c) 0.478 0.133 0.203 0.799 图11(d) 0.561 0.610 0.358 0.659 图11(e) 0.612 0.318 0.359 0.946 图11(f) 0.641 0.414 0.881 0.948 精准率 图11(a) 0.436 0.324 0.463 0.884 图11(b) 0.002 0.478 0.444 0.793 图11(c) 0.024 0.112 0.143 0.855 图11(d) 0.639 0.299 0.642 0.940 图11(e) 0.412 0.367 0.696 0.824 图11(f) 0.899 0.006 0.920 0.935 加权调和平均 图11(a) 0.526 0.210 0.508 0.824 图11(b) 0.004 0.398 0.608 0.732 图11(c) 0.045 0.122 0.168 0.822 图11(d) 0.597 0.401 0.460 0.771 图11(e) 0.492 0.341 0.662 0.881 图11(f) 0.748 0.012 0.900 0.941
下载: 导出CSV
表 8 处理耗时的比较 (ms)
Table 8 Comparison of processing time (ms)
检测方法 PHOT LCA MSCDAE ReNet-D 耗时 450 430 9746.59 2.82
下载: 导出CSV
-
[1] Deng S, Cai W, Xu Q, Liang B. Defect detection of bearing surfaces based on machine vision technique. In: Proceedings of the 2010 International Conference on Computer Application and System Modeling (ICCASM 2010). Taiyuan, China: IEEE Press, 2010. 4−548 [2] Jian, C., Gao, J., & Ao, Y. Automatic surface defect detection for mobile phone screen glass based on machine vision. Applied Soft Computing, 2017, 52: 348-358. doi: 10.1016/j.asoc.2016.10.030 [3] Bulnes, F. G., Usamentiaga, R., Garcia, D. F., & Molleda, J. An efficient method for defect detection during the manufacturing of web materials. Journal of Intelligent Manufacturing, 2016.27(2): 431-445. doi: 10.1007/s10845-014-0876-9 [4] 金侠挺, 王耀南, 张辉, 刘理, 钟杭, 贺振东. 基于贝叶斯 CNN 和注意力网络的钢轨表面缺陷检测系统. 自动化学报, 2019, 45(12): 2312-2327Jin XiaTing, Wang YaoNan, Zhang Hui, Liu Li, Zhong Hang, He ZhenDong. DeepRail: automatic visual detection system for railway surface defect using Bayesian CNN and attention network. Acta Automatica Sinica, 2019, 45(12): 2312-2327 [5] 李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727-1742Li Liang-Fu, Ma Wei-Fei, Li Li, Lu Cheng. Research on detection algorithm for bridge cracks based on deep learning. Acta Automatica Sinica, 2019, 45(9): 1727-1742. [6] Chen S, Hu T, Liu G, Pu Z, Li M, Du L. Defect classification algorithm for IC photomask based on PCA and SVM. In: Proceedings of the 2008 Congress on Image and Signal Processing. Sanya, China: IEEE Press, 2008. 491−496 [7] Jiexian, H., Di, L., & Feng, Y. Detection of surface of solder on flexible circuit[J]. Optics and Precision Engineering, 2010, 18(11):2443-2453. [8] Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks. In: Proceedings of the Advances in Neural Information Processing Systems. Toronto, Canada: 2012. 1097−1105 [9] Napoletano, P., Piccoli, F., & Schettini, R. (2018). Anomaly detection in nanofibrous materials by CNN-based self-similarity. Sensors, 2018, 18(1): 209. doi: 10.1109/JSEN.2017.2771313 [10] Cha, Y. J., Choi, W., Suh, G., Mahmoudkhani, S., & Büyüköztürk, O. Autonomous structural visual inspection using region‐based deep learning for detecting multiple damage types. Computer‐Aided Civil and Infrastructure Engineering, 2018, 33(9):731-747. doi: 10.1111/mice.12334 [11] Gao, Y., Gao, L., Li, X., & Yan, X. A semi-supervised convolutional neural network-based method for steel surface defect recognition. Robotics and Computer-Integrated Manufacturing, 2020, 61, 101825. doi: 10.1016/j.rcim.2019.101825 [12] Zhao Z, Xu G, Qi Y, Liu N, Zhang T. Multi-patch deep features for power line insulator status classification from aerial images. In: Proceedings of the International Joint Conference on Neural Networks. Vancouver, Canada: IEEE Press, 2016. 3187− 3194 [13] Wang, T., Chen, Y., Qiao, M., & Snoussi, H. A fast and robust convolutional neural network-based defect detection model in product quality control. The International Journal of Advanced Manufacturing Technology, 2018.94(9-12), 3465-3471. doi: 10.1007/s00170-017-0882-0 [14] Xu, X., Zheng, H., Guo, Z., Wu, X., & Zheng, Z. SDD-CNN: Small Data-Driven Convolution Neural Networks for Subtle Roller Defect Inspection. Applied Sciences, 2019. 9(7), 1364. doi: 10.3390/app9071364 [15] Paolo N, Flavio P, Raimondo S. Anomaly Detection in Nanofibrous Materials by CNN-Based Self-Similarity[J]. Sensors, 2018, 18(2):209-216. doi: 10.3390/s18010209 [16] Weimer D, Scholz-Reiter B, Shpitalni M. Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection[J]. CIRP Annals. Manufacturing Technology, 2016.65(1):417-420. doi: 10.1016/j.cirp.2016.04.072 [17] Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. IEEE Computer Society, 2014: 580−587 [18] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, et al. SSD: Single shot multi-box detector. In: Proceedings of the European Conference on Computer Vision. Amsterdam, Netherlands: Springer International Publishing, 2016. 21−37 [19] Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Conference on Computer Vision and Pattern Recognition. Seattle, USA: 2016. 779−788 [20] Chen, J., Liu, Z., Wang, H., Núñez, A., & Han, Z. Automatic defect detection of fasteners on the catenary support device using deep convolutional neural network. IEEE Transactions on Instrumentation and Measurement.2017, 67(2), 257-269. [21] Cha Y J, Choi W, Suh G, et al. Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types[J]. Computer Aided Civil & Infrastructure Engineering, 2017, 00(4):1-17 [22] Huang Y, Qiu C, Guo Y, Wang X, Yuan K. Surface defect saliency of magnetic tile. In: Proceedings of the 2018 IEEE 14th International Conference on Automation Science and Engineering. Munich, Germany: 2018. 612−617 [23] Lingteng, Qiu, Xiaojun, et al. A High-Efficiency Fully Convolutional Networks for Pixel-Wise Surface Defect Detection[J]. IEEE Access, 2019: 15884-15893 [24] Yu W, Zhang Y, Shi H. Surface defect inspection under a small training set condition. In: Proceedings of the International Conference on Intelligent Robotics and Applications. Shenyang, China: 2019, 517−528 [25] Masci J, Meier U, Cireşan D, Schmidhuber J. Stacked convolutional auto-encoders for hierarchical feature extraction. In: Proceedings of the International Conference on Artificial Neural Networks. Berlin, Germany: Springer, 2011. 52−59 [26] Li, Y., Zhao, W., & Pan, J. Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Transactions on Automation Science and Engineering, 2016, 14(2), 1256-1264. [27] Chalapathy R, Menon A K, Chawla S. Robust, deep and inductive anomaly detection. In: Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Skopje, Macedonia: Springer, 2017. 36−51 [28] 袁静, 章毓晋. 融合梯度差信息的稀疏去噪自编码网络在异常行为检测中的应用. 自动化学报, 2017, 43(4): 604-610Yuan Jing, Zhang YuJin. Application of sparse denoising autoencoder network with gradient difference information for abnormal action detection. Acta Automatica Sinica, 2017, 43(4):604-610. [29] Mei S, Yang H, Yin Z. An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces[J]. IEEE Transactions on Instrumentation and Measurement, 2018:1-12. [30] Aiger D, Talbot H. The phase only transform for unsupervised surface defect detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, USA: IEEE Press, 2010. 295−302 [31] H.-D. Lin, “Tiny surface defect inspection of electronic passive components using discrete cosine transform decomposition and cumulative sum techniques, ” Image Vis. Comput, 2008, vol. 26, no. 5, pp. 603–621 doi: 10.1016/j.imavis.2007.07.009 [32] Yang H, Chen Y, Song K, Yin Z. Multi-scale feature-clustering-based fully convolutional autoencoder for fast accurate visual inspection of texture surface defects. IEEE Transactions on Automation Science and Engineering. 2019, 1−18 [33] Makhzani A, Shlens J, Jaitly N, Goodfellow I, Frey B. Adversarial autoencoders. arXiv preprint, 2015, arXiv: 1511.05644 [34] Zhao Z, Li B, Dong R, Zhao P. A surface defect detection method based on positive samples. In: Proceedings of the International Conference on Artificial Intelligence. Nanjing, China: 2018. 473−481 [35] He D, Xu K, Zhou P, Zhou D. Surface defect classification of steels with a new semi-supervised learning method. Optics and Lasers in Engineering, 2019, 117: 40−48 [36] Schlegl T, Seeböck P, Waldstein S M, Schmidt-Erfurth U, Langs G. Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: Proceedings of the International Conference on Information Processing in Medical Imaging. Boone, USA: 2017. 146−157 [37] Paul B, Michael F, David S, Carsten S. A comprehensive real-world dataset for unsupervised anomaly detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: 2019. 9592−9600 [38] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv prepr-int, 2015, arXiv: 1502.03167 [39] Zhao, H., Gallo, O., Frosio, I., & Kautz, J. Loss functions for image restoration with neural networks. IEEE Transactions on Computational Imaging, 2016. 3(1), 47-57 [40] Bergmann P, Löwe S, Fauser M, Sattlegger D, Steger C. Improving unsupervised defect segmentation by applying structural similarity to autoencoders. In: Proceedings of the 14th International Conference on Computer Vision Theory and Applicati-ons. Prague, Czecg Republic. 2019: 372−380 [41] Lv C, Zhang Z, Shen F, Zhang F, Su H. A fast surface defect detection method based on background reconstruction. Internation-al Journal of Precision Engineering and Manufacturing, 2019: 1−13 [42] Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 2004, 13(4):600-612. doi: 10.1109/TIP.2003.819861 [43] Silvestre-Blanes, J., Albero-Albero, T., Miralles, I., Pérez-Llorens, R., & Moreno, J. A Public Fabric Database for Defect Detection Methods and Results. Autex Research Journal, 2019, 19(4):363-374. doi: 10.2478/aut-2019-0035 [44] Jager, M., Knoll, C., & Hamprecht, F. A.Weakly supervised learning of a classifier for unusual event detection. IEEE Transactions on Image Processing, 2019, 17(9):1700-1708. [45] Kylberg G, Sintorn I. Sintorn K. Rotationdataset[Online]. Available: http://www.cb.uu.se/~gustaf/KylbergSintor-nRotation/ Fe-bruary 25, 2013. [46] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: 2015. 3431−3440 [47] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-assisted Intervention. Munich, Germany: 2015. 234− 241 [48] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 770−778 [49] Shi W, Caballero J, Huszár F, Totz J, Aitken A P, Bishop R, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 1874−1883 [50] Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 4700−4708 [51] Tsai, D. M., & Huang, T. Y. Automated surface inspection for statistical textures. Image and Vision computing, 2003.21(4):307-323. doi: 10.1016/S0262-8856(03)00007-6 -
图(11) / 表(8)
计量
- 文章访问数: 3386
- HTML全文浏览量: 715
- PDF下载量: 679
- 被引次数: 0
