-
摘要: 基于深度学习的表面缺陷检测技术是工业上的一项重要应用, 而缺陷图像数据集质量对缺陷检测性能有重要影响. 为解决实际工业生产过程中缺陷样本获取成本高、缺陷数据量少的痛点, 提出了一种基于去噪扩散概率模型(Denoising Diffusion Probabilistic Model, DDPM)的缺陷生成方法. 该方法在训练过程中加强了模型对缺陷部位和无缺陷背景的差异化学习. 在生成过程中通过缺陷控制模块对生成缺陷的类别、形态、显著性等特征进行精准控制, 通过背景融合模块, 能将缺陷在不同的无缺陷背景上进行迁移, 大大降低新背景上缺陷样本的获取难度. 实验验证了该模型的缺陷控制和缺陷迁移能力, 其生成结果能有效扩充训练数据集, 提升下游缺陷检测任务的准确率.Abstract: Surface defect detection technology based on deep learning is an important application in industry and the quality of defect image dataset has a significant impact on defect detection performance.A defect image generation method based on denoising diffusion probabilistic model (DDPM) is designed to address the pain points of high cost of obtaining defect samples and low amount of defect data in actual industrial production processes.This method enhances the model's differential learning of defect locations and defect free backgrounds during the training process.Through the defect control module during the generation process, this method accurately controls the category, morphology, saliency and other features of generated defects.Through the background fusion module, defects can be migrated on different defect free backgrounds, which greatly reducing the difficulty of obtaining defect samples on new backgrounds.The experiment has verified the defect control and defect migration capabilities of the model, and its generated results can effectively expand the training dataset and improve the accuracy of downstream defect detection tasks.
-
Key words:
- Data augmentation /
- dataset expansion /
- defect image generation /
- deep learning
-
表 1 评价指标统计
Table 1 Statistics of evaluation metrics
评价指标 Pix2pix StyleGAN2 DFMGAN RePaint模型 所提方法 IS↑ 1.388 1.368 1.301 1.474 1.541 FID↓ 59.056 124.748 96.783 72.750 57.650 KID↓ 0.024 0.098 0.053 0.044 0.020 MS-SSIM↓ 0.189 0.161 0.174 0.187 0.159 PSNR↓ 28.308 28.284 28.357 28.273 28.223 * RePaint模型使用裂纹掩码生成缺陷效果差,改为区块掩码
* 箭头标明评价指标得分更好的方向表 2 分类结果统计
Table 2 Statistics of classification results
测试集 Pix2pix StyleGAN2 DFMGAN RePaint模型 所提方法 缺陷检出率 总正确率 缺陷检出率 总正确率 缺陷检出率 总正确率 缺陷检出率 总正确率 缺陷检出率 总正确率 D1 30.77 65.38 26.92 63.46 7.69 53.85 13.46 54.81 88.46 94.23 D2 54.46 77.07 49.94 74.97 32.26 66.13 45.76 70.99 88.32 93.76 *RePaint模型使用裂纹掩码生成缺陷效果差, 改为区块掩码 -
[1] 张辉, 张邹铨, 陈煜嵘, 吴天月, 钟杭, 王耀南. 工业铸件缺陷无损检测技术的应用进展与展望. 自动化学报, 2022, 48(4): 935−956Zhang Hui, Zhang Zou-Quan, Chen Yu-Rong, Wu Tian-Yue, Zhong Hang, Wang Yao-Nan. Application advance and prospect of nondestructive testing technology for industrial casting defects. Acta Automatica Sinica, 2022, 48(4): 935−956 [2] 罗东亮, 蔡雨萱, 杨子豪, 章哲彦, 周瑜, 白翔. 工业缺陷检测深度学习方法综述. 中国科学:信息科学, 2022, 52(6): 1002−1039 doi: 10.1360/SSI-2021-0336Luo Dong-Liang, Cai Yu-Xuan, Yang Zi-Hao, Zhang Zhe-Yan, Zhou Yu, Bai Xiang. Survey on industrial defect detection with deep learning. Scientia Sinica Informationis, 2022, 52(6): 1002−1039 doi: 10.1360/SSI-2021-0336 [3] 陶晓天, 何博侠, 张鹏辉, 田德旭. 基于深度学习的航天密封圈表面缺陷检测. 仪器仪表学报, 2021, 42(1): 199−206Tao Xiao-Tian, He Bo-Xia, Zhang Peng-Hui, Tian De-Xu. Surface defect detection of aerospace sealing rings based on deep learning. Chinese Journal of Scientific Instrument, 2021, 42(1): 199−206 [4] 田娟秀, 刘国才, 谷珊珊, 鞠忠建, 刘劲光, 顾冬冬. 医学图像分析深度学习方法研究与挑战. 自动化学报, 2018, 44(3): 401−424Tian Juan-Xiu, Liu Guo-Cai, Gu Shan-Shan, Ju Zhong-Jian, Liu Jin-Guang, Gu Dong-Dong. Deep learning in medical image analysis and its challenges. Acta Automatica Sinica, 2018, 44(3): 401−424 [5] 王国力, 孙宇, 魏本征. 医学图像图深度学习分割算法综述. 计算机工程与应用, 2022, 58(12): 37−50 doi: 10.3778/j.issn.1002-8331.2112-0225Wang Guo-Li, Sun Yu, Wei Ben-Zheng. Systematic review on graph deep learning in medical image segmentation. Computer Engineering and Applications, 2022, 58(12): 37−50 doi: 10.3778/j.issn.1002-8331.2112-0225 [6] 李书林, 冯朝路, 于鲲, 刘鑫, 江鑫, 赵大哲. 基于深度学习的心脏磁共振影像超分辨率前沿进展. 中国图象图形学报, 2022, 27(3): 704−721 doi: 10.11834/j.issn.1006-8961.2022.3.zgtxtxxb-a202203005Li Shu-Lin, Feng Chao-Lu, Yu Kun, Liu Xin, Jiang Xin, Zhao Da-Zhe. Critical review of human cardiac magnetic resonance image super resolution reconstruction based on deep learning method. Journal of Image and Graphics, 2022, 27(3): 704−721 doi: 10.11834/j.issn.1006-8961.2022.3.zgtxtxxb-a202203005 [7] Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016. 779−788 [8] Liu W, Anguelov D, Erhan D, et al. Ssd: single shot multibox detector. In: Computer Vision–ECCV 2016: 14th European Conference. Amsterdam, The Netherlands: Springer International Publishing, 2016. 21−37 [9] Ren S, He K, Girshick R, et al. Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2017, 39(6): 1137−1149 [10] Redmon J, Farhadi A. Yolov3: an incremental improvement. arXiv: 1804.02767, 2018. [11] Santos C F G D, Papa J P. Avoiding overfitting: a survey on regularization methods for convolutional neural networks. ACM Computing Surveys (CSUR), 2022, 54(10s): 1−25 [12] 陶显, 侯伟, 徐德. 基于深度学习的表面缺陷检测方法综述. 自动化学报, 2021, 47(05): 1017−1034Tao Xian, Hou Wei, Xu De. A survey of surface defect detection methods based on deep learning. Acta Automatica Sinica, 2021, 47(05): 1017−1034 [13] Niu S, Li B, Wang X, et al. Defect image sample generation with GAN for improving defect recognition. IEEE Transactions on Automation Science and Engineering, 2020, 17(3): 1611−1622 [14] 伍麟, 郝鸿宇, 宋友. 基于计算机视觉的工业金属表面缺陷检测综述. 自动化学报, DOI: 10.16383/j.aas.c230039Wu Lin, Hao Hong-Yu, Song You. A review of metal surface defect detection based on computer vision. Acta Automatica Sinica, DOI: 10.16383/j.aas.c230039 [15] Rippel O, Müller M, Merhof D. GAN-based defect synthesis for anomaly detection in fabrics. In: 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA). Vienna, Austria: IEEE, 2020. 1: 534−540 [16] Zhang G, Cui K, Hung T Y, et al. Defect-GAN: high-fidelity defect synthesis for automated defect inspection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa, HI, USA: IEEE, 2021. 2524−2534 [17] Zhang H, Pan D, Liu J, et al. A novel MAS-GAN-based data synthesis method for object surface defect detection. Neurocomputing, 2022, 499: 106−114 doi: 10.1016/j.neucom.2022.05.021 [18] Wang R, Hoppe S, Monari E, et al. Defect transfer GAN: diverse defect synthesis for data augmentation. arXiv: 2302.08366, 2023. [19] 丁鹏, 卢文壮, 刘杰, 袁志响. 基于生成对抗网络的叶片表面缺陷图像数据增强. 组合机床与自动化加工技术, 2022(7): 18−21Ding Peng, Lu Wen-Jie, Liu Jie, Yuan Zhi-Xiang. image data augmentation of blade surface defects based on generative adversarial network. Modular Machine Tool & Automatic Manufacturing Technique, 2022(7): 18−21 [20] Dhariwal P, Nichol A. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 2021, 34: 8780−8794 [21] Carlini N, Hayes J, Nasr M, et al. Extracting training data from diffusion models. In: 32nd USENIX Security Symposium (USENIX Security 23). ANAHEIM, CA, USA, 2023. 5253−5270 [22] Jain S, Seth G, Paruthi A, et al. Synthetic data augmentation for surface defect detection and classification using deep learning. Journal of Intelligent Manufacturing, 20221−14 [23] Zhang H, Cisse M, Dauphin Y N, et al. Mixup: beyond empirical risk minimization. arXiv: 1710.09412, 2017. [24] Zhang L, Deng Z, Kawaguchi K, et al. How does mixup help with robustness and generalization?. arXiv: 2010.04819, 2020. [25] Chou H P, Chang S C, Pan J Y, et al. Remix: rebalanced mixup. In: Proceedings of Computer Vision–ECCV 2020 Workshops. Glasgow, UK: Springer International Publishing, 2020. 95−110 [26] Ren X, Lin W, Yang X, et al. Data augmentation in defect detection of sanitary ceramics in small and non-iid datasets. IEEE Transactions on Neural Networks and Learning Systems, 20228669−8678 [27] Isola P, Zhu J Y, Zhou T, et al. Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, 2017. 1125−1134 [28] Duan Y, Hong Y, Niu L, et al. Few-shot defect image generation via defect-aware feature manipulation. In: Proceedings of the AAAI Conference on Artificial Intelligence. Washington DC, USA: AAAI, 2023. 37(1): 571−578 [29] Sohl-Dickstein J, Weiss E, Maheswaranathan N, et al. Deep unsupervised learning using nonequilibrium thermodynamics. In: 32nd International Conference on Machine Learning. PMLR, 2015. 2256−2265 [30] Ho J, Jain A, Abbeel P. Denoising diffusion probabilistic models. Advances in neural information processing systems, 2020, 33: 6840−6851 [31] Ho J, Salimans T. Classifier-free diffusion guidance. arXiv: 2207.12598, 2022. [32] Saharia C, Ho J, Chan W, et al. Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(4): 4713−4726 [33] Lugmayr A, Danelljan M, Romero A, et al. RePaint: Inpainting using denoising diffusion probabilistic models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, LA, USA: IEEE, 2022. 11461−11471 [34] Wang W, Bao J, Zhou W, et al. Semantic image synthesis via diffusion models. arXiv: 2207.00050, 2022. [35] Nichol A, Dhariwal P, Ramesh A, et al. Glide: towards photorealistic image generation and editing with text-guided diffusion models. arXiv: 2112.10741, 2021. [36] Ramesh A, Dhariwal P, Nichol A, et al. Hierarchical text-conditional image generation with clip latents. arXiv: 2204.06125, 2022, 1(2): 3. [37] Rombach R, Blattmann A, Lorenz D, et al. High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. New Orleans, LA, USA: IEEE, 2022. 10684−10695 [38] Song Y, Sohl-Dickstein J, Kingma D P, et al. Score-based generative modeling through stochastic differential equations. arXiv: 2011.13456, 2020. [39] Tabernik D, Šela S, Skvarč J, et al. Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 2020, 31(3): 759−776 doi: 10.1007/s10845-019-01476-x [40] Matthias Wieler, Tobias Hahn, Fred. A. Hamprecht. (2007) Weakly supervised learning for industrial optical inspection. [Dataset]. [41] Song K, Yan Y. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science, 2013, 285(21): 858−864 [42] Karras T, Laine S, Aittala M, et al. Analyzing and improving the image quality of stylegan. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Seattle, WA, USA: IEEE, 2020. 8110−8119 [43] Barratt S, Sharma R. A note on the Inception Score. arXiv: 1801.01973, 2018. [44] Heusel M, Ramsauer H, Unterthiner T, et al. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 2017, 30: [45] Bińkowski M, Sutherland D J, Arbel M, et al. Demystifying mmd gans. arXiv: 1801.01401, 2018. [46] Wang Z, Simoncelli E P, Bovik A C. Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers. IEEE, 2003. 2: 1398−1402 [47] Huynh-Thu Q, Ghanbari M. Scope of validity of PSNR in image/video quality assessment. Electronics letters, 2008, 44(13): 800−801 doi: 10.1049/el:20080522 -
计量
- 文章访问数: 18
- HTML全文浏览量: 8
- 被引次数: 0