2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于深度学习的纹理布匹瑕疵检测方法

许玉格 钟铭 吴宗泽 任志刚 刘伟生

许玉格, 钟铭, 吴宗泽, 任志刚, 刘伟生. 基于深度学习的纹理布匹瑕疵检测方法. 自动化学报, 2023, 49(4): 857−871 doi: 10.16383/j.aas.c200148
引用本文: 许玉格, 钟铭, 吴宗泽, 任志刚, 刘伟生. 基于深度学习的纹理布匹瑕疵检测方法. 自动化学报, 2023, 49(4): 857−871 doi: 10.16383/j.aas.c200148
Xu Yu-Ge, Zhong Ming, Wu Zong-Ze, Ren Zhi-Gang, Liu Wei-Sheng. Detection of detecting textured fabric defects based on deep learning. Acta Automatica Sinica, 2023, 49(4): 857−871 doi: 10.16383/j.aas.c200148
Citation: Xu Yu-Ge, Zhong Ming, Wu Zong-Ze, Ren Zhi-Gang, Liu Wei-Sheng. Detection of detecting textured fabric defects based on deep learning. Acta Automatica Sinica, 2023, 49(4): 857−871 doi: 10.16383/j.aas.c200148

基于深度学习的纹理布匹瑕疵检测方法

doi: 10.16383/j.aas.c200148
基金项目: 国家自然科学基金(61703114, 61673126, U1701261, 51675108)资助
详细信息
    作者简介:

    许玉格:华南理工大学自动化科学与工程学院副教授. 主要研究方向为机器学习与智能计算. E-mail: xuyuge@scut.edu.cn

    钟铭:华南理工大学自动化科学与工程学院硕士研究生. 主要研究方向为深度学习, 计算机视觉. E-mail: tdlming@163.com

    吴宗泽:深圳大学机电与控制工程学院教授. 主要研究方向为自动控制, 信号处理, 大数据, 知识自动化, 人工智能. 本文通信作者. E-mail: zzwu@szu.edu.cn

    任志刚:2016年获得浙江大学控制理论与控制工程专业博士学位. 主要研究方向为最优控制, 知识自动化, 人工智能. E-mail: renzhigang@gdut.edu.cn

    刘伟生:2019年获得西安交通大学电子与信息工程学院硕士学位. 主要研究方向为深度学习, 工业质检. E-mail: liuweisheng1992@outlook.com

Detection of Detecting Textured Fabric Defects Based on Deep Learning

Funds: Supported by National Natural Science Foundation of China (61703114, 61673126, U1701261, 51675108)
More Information
    Author Bio:

    XU Yu-Ge Associate professor at the School of Automation Science and Engineering, South China University of Technology. Her research interest covers machine learning and intelligent computing

    ZHONG Ming Master student at the School of Automation Science and Engineering, South China University of Technology. His research interest covers deep learning and computer vision

    WU Zong-Ze Professor at theSchool of Electromechanical and Control Engineering, Shenzhen University. His research interest covers automation control, signal processing, big data, knowledge automation, and artificial intelligence. Corresponding author of this paper

    REN Zhi-Gang Received his Ph.D. degree in control theory and control engineering from Zhejiang University in 2016.His research interest covers optimal control, knowle-dge automation, and artificial intelligence

    LIU Wei-Sheng Received his master degree from the School of Electronic and Information Engineering, Xi'an Jiaotong University in 2019. His research interest covers deep learning and industrial quality inspection

  • 摘要: 布匹瑕疵检测是纺织工业中产品质量评估的关键环节, 实现快速、准确、高效的布匹瑕疵检测对于提升纺织工业的产能具有重要意义. 在实际布匹生产过程中, 布匹瑕疵在形状、大小及数量分布上存在不平衡问题, 且纹理布匹复杂的纹理信息会掩盖瑕疵的特征, 加大布匹瑕疵检测难度. 本文提出基于深度卷积神经网络的分类不平衡纹理布匹瑕疵检测方法(Detecting defects in imbalanced texture fabric based on deep convolutional neural network, ITF-DCNN), 首先建立一种基于通道叠加的ResNet50卷积神经网络模型(ResNet50+)对布匹瑕疵特征进行优化提取; 其次提出一种冗余特征过滤的特征金字塔网络(Filter-feature pyramid network, F-FPN)对特征图中的背景特征进行过滤, 增强其中瑕疵特征的语义信息; 最后构造针对瑕疵数量进行加权的MFL (Multi focal loss)损失函数, 减轻数据集不平衡对模型的影响, 降低模型对于少数类瑕疵的不敏感性. 通过实验对比, 提出的方法能有效提升布匹瑕疵检测的准确率及定位精度, 同时降低了布匹瑕疵检测的误检率和漏检率, 明显优于当前主流的布匹瑕疵检测算法.
  • 图  1  纹理布匹瑕疵样本图片

    Fig.  1  Samples of jacquard fabric defects

    图  2  纹理布匹瑕疵形状分布

    Fig.  2  Shape distribution of jacquard fabric defects

    图  3  ITF-DCNN模型的整体结构图

    Fig.  3  Structure of proposed model

    图  4  ResNet50网络结构图

    Fig.  4  Model structure of ResNet50

    图  5  残差模块

    Fig.  5  Model structure of residual block

    图  6  特征图过滤方式

    Fig.  6  Methods to filtering feature maps

    图  7  模型的损失收敛曲线图

    Fig.  7  The loss curves of models

    图  8  ITF-DCNN模型检测结果图

    Fig.  8  Experimental results of the proposed method ITF-DCNN

    图  9  MFL在少数类上的检测结果图

    Fig.  9  Experimental result of MFL based on minority classes

    图  10  模板图片

    Fig.  10  Template images

    图  11  FPN和F-FPN的泛化性实验对比图

    Fig.  11  Comparison of FPN and F-FPN generalization experiments

    表  1  增强前后数据集中的样本分布

    Table  1  Samples distribution of the dataset before and after data augmentation

    瑕疵类别
    沾污花毛虫粘破洞蜡斑网折其他正常总计
    训练集增强前243239820812273775227566118
    训练集增强后972815948344902923062061102624476
    验证集增强前14133126254420623
    验证集增强后56213448226221416722490
    下载: 导出CSV

    表  2  数据集增强前后模型准确率对比实验结果 (%)

    Table  2  Experimental results of model on accuracy before and after dataset enhancement (%)

    瑕疵类别
    沾污花毛虫粘破洞蜡斑网折其他正常总计
    数据集增强前88.2483.3687.5689.3683.7888.2189.6598.6688.61
    数据集增强后90.5685.5190.3591.4287.6489.2490.0299.8190.57
    下载: 导出CSV

    表  3  数据集增强前后模型mAP对比实验结果 (%)

    Table  3  Experimental results of model on mAP before and after dataset enhancement (%)

    瑕疵类别
    沾污花毛虫粘破洞蜡斑网折其他正常总计
    数据集增强前69.0658.5181.5083.4433.3363.7045.5162.15
    数据集增强后70.0459.1283.2383.5435.7863.7047.3163.25
    下载: 导出CSV

    表  4  不同模型在布匹瑕疵数据集上的实验结果 (%)

    Table  4  Experimental results of different models on the jacquard fabric defect dataset (%)

    检测器主干网络mAP准确率误检率漏检率
    Faster R-CNNResNet5065.5687.4012.601.42
    Cascade R-CNNResNet5063.7790.559.452.85
    RetinaNetResNet5065.6053.8646.130.20
    Faster R-CNNResNet10163.8588.7211.282.24
    Cascade R-CNNResNet10164.6090.359.651.83
    RetinaNetResNet10166.5256.2343.770.12
    GLCM64.6335.376.87
    Gabor83.8716.131.67
    GMM81.3218.681.77
    PTIT[38]92.567.440.94
    CAE-SGAN[41]85.0114.992.65
    SurfNet[42]84.8215.181.79
    ITF-DCNNResNet5073.4197.562.441.65
    ITF-DCNNResNet10173.9297.662.341.14
    下载: 导出CSV

    表  5  改进后的ResNet50+ 网络性能对比实验 (%)

    Table  5  Experimental performance result of ResNet50+ (%)

    mAP准确率误检率漏检率
    ResNet5063.7790.559.452.85
    ResNet50+I63.6891.768.242.66
    ResNet50+C64.1492.317.693.43
    ResNet50+64.7292.787.222.91
    下载: 导出CSV

    表  6  F-FPN性能验证实验结果 (%)

    Table  6  Experimental performance result of F-FPN (%)

    mAP准确率误检率漏检率
    Top-Down FPN63.7790.559.452.85
    PANet65.6992.237.772.56
    加性F-FPN70.3193.656.531.95
    卷积F-FPN71.4296.723.281.25
    下载: 导出CSV

    表  7  MFL的性能验证实验结果

    Table  7  Experimental performance result of MFL

    损失函数$ \alpha $$ \gamma $$ \omega $mAP (%)准确率 (%)误检率 (%)漏检率 (%)
    CE 63.77 90.55 9.45 2.85
    FL 0.25 5.0 52.53 70.23 29.77 9.56
    FL 0.25 2.0 65.62 92.86 7.14 2.02
    FL 0.25 1.0 64.88 91.91 8.09 2.12
    FL 0.50 0.5 64.74 91.55 8.45 2.33
    FL 0.75 0.2 59.27 83.02 16.98 8.50
    FL 0.75 0.1 58.11 80.85 19.15 7.56
    FL 0.75 0.0 58.01 81.22 18.78 7.66
    MFL 1.0 0.618 68.21 94.39 5.61 1.44
    MFL 2.0 0.618 70.12 95.32 4.68 1.68
    MFL 5.0 0.618 68.11 94.50 5.50 1.56
    MFL 2.0 0.100 67.22 93.68 6.32 2.26
    MFL 2.0 0.300 69.03 94.88 5.12 1.56
    MFL 2.0 1.000 69.22 95.17 4.83 1.29
    MFL 2.0 2.000 68.81 94.35 5.65 1.68
    MFL 2.0 5.000 64.38 92.41 7.59 2.42
    下载: 导出CSV

    表  8  采用F-FPN的模型在不同模板上的泛化性分析 (%)

    Table  8  Generalization analysis of models using F-FPN on different templates (%)

    模板1模板2模板3模板4模板5模板6模板7模板8模板9模板10均值
    准确率95.8796.7999.6793.5691.7491.1193.6699.1298.2393.6595.34
    mAP69.1269.7375.3768.9767.4668.1268.2475.9676.8268.0270.78
    下载: 导出CSV

    表  9  采用FPN的模型在不同模板上的泛化性分析 (%)

    Table  9  Generalization analysis of models using FPN on different templates (%)

    模板1模板2模板3模板4模板5模板6模板7模板8模板9模板10均值
    准确率91.6391.0492.3890.3988.3488.1291.2591.7591.4289.1190.54
    mAP62.4361.7565.8662.0161.9961.0862.5165.6366.7461.4663.07
    下载: 导出CSV
  • [1] 张慧, 王坤峰, 王飞跃. 深度学习在目标视觉检测中的应用进展与展望. 自动化学报, 2017, 43(8): 1289-1305

    Zhang Hui, Wang Kun-Feng, Wang Fei-Yue. Advances and perspectives on applications of deep learning in visual object detection. Acta Automatica Sinica, 2017, 43(8): 1289-1305
    [2] Kumar A. Computer-vision-based fabric defect detection: A survey. IEEE Transactions on Industrial Electronics, 2008, 55(1): 348-363 doi: 10.1109/TIE.1930.896476
    [3] Chan C H, Pang G K H. Fabric defect detection by Fourier analysis. IEEE Transactions on Industry Applications, 2000, 36(5): 1267-1276 doi: 10.1109/28.871274
    [4] Hanbay K, Talu M F, Özgüven Ö F. Fabric defect detection systems and methods-a systematic literature review. Optik, 2016, 127(24): 11960-11973 doi: 10.1016/j.ijleo.2016.09.110
    [5] Chetverikov D, Hanbury A. Finding defects in texture using regularity and local orientation. Pattern Recognition, 2002, 35(10): 2165-2180 doi: 10.1016/S0031-3203(01)00188-1
    [6] Ngan H Y T, Pang G K H, Yung N H C. Automated fabric defect detection-a review. Image and Vision Computing, 2011, 29(7): 442-458 doi: 10.1016/j.imavis.2011.02.002
    [7] Yapi D, Allili M S, Baaziz N. Automatic fabric defect detection using learning-based local textural distributions in the contourlet domain. IEEE Transactions on Automation Science and Engineering, 2018, 15(3): 1014-1026 doi: 10.1109/TASE.2017.2696748
    [8] Ravandi S A H, Toriumi K. Fourier transform analysis of plain weave fabric appearance. Textile Research Journal, 1995, 65(11): 676-683 doi: 10.1177/004051759506501108
    [9] Escofet J, Garcia-Verela M S M, Abril H C, Torrecilla E. Inspection of fabric resistance to abrasion by Fourier analysis. In: Proceedings of the SPIE 3490, Optics in Computing'98. Bruges, Belgium: SPIE, 1998. 207−210
    [10] Zachevsky I, Zeevi Y Y. Model-based color natural stochastic textures processing and classification. In: Proceedings of the IEEE Global Conference on Signal and Information Processing (GlobalSIP). Orlando, USA: IEEE, 2015. 1357−1361
    [11] 李敏, 崔树芹, 谢治平. 高斯混合模型在印花织物疵点检测中的应用. 纺织学报, 2015, 36(8): 94-98 doi: 10.13475/j.fzxb.20140504105

    Li Min, Cui Shu-Qin, Xie Zhi-Ping. Application of Gaussian mixture model on defect detection of print fabric. Journal of Textile Research, 2015, 36(8): 94-98 doi: 10.13475/j.fzxb.20140504105
    [12] Allili M S, Baaziz N, Mejri M. Texture modeling using contourlets and finite mixtures of generalized Gaussian distributions and applications. IEEE Transactions on Multimedia, 2014, 16(3): 772-784 doi: 10.1109/TMM.2014.2298832
    [13] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA: Curran Associates Inc., 2012. 1097−1105
    [14] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations. San Diego, USA: ICLR, 2015.
    [15] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016. 2818−2826
    [16] Szegedy C, Ioffe S, Vanhoucke V, Alemi A A. Inception-v4, inception-ResNet and the impact of residual connections on learning. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI, 2017. 4278−4284
    [17] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning. Lille, France: JMLR.org, 2015. 448−456
    [18] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE, 2015. 1−9
    [19] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016. 770−778
    [20] Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 2261−2269
    [21] Cai Z W, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 6154−6162
    [22] Tan M X, Pang R M, Le Q V. EfficientDet: Scalable and efficient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2019. 10778−10787
    [23] Lin T Y, Dollár P, Girshick R, He K M, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 936−944
    [24] 周波, 李俊峰. 结合目标检测的人体行为识别. 自动化学报, 2020, 46(9): 1961-1970 doi: 10.16383/j.aas.c180848

    Zhou Bo, Li Jun-Feng. Human action recognition combined with object detection. Acta Automatica Sinica, 2020, 46(9): 1961-1970 doi: 10.16383/j.aas.c180848
    [25] 范家伟, 张如如, 陆萌, 何佳雯, 康霄阳, 柴文俊, 等. 深度学习方法在糖尿病视网膜病变诊断中的应用. 自动化学报, 2021, 47(5): 985-1004

    Fan Jia-Wei, Zhang Ru-Ru, Lu Meng, He Jia-Wen, Kang Xiao-Yang, Chai Wen-Jun, et al. Applications of deep learning techniques for diabetic retinal diagnosis. Acta Automatica Sinica, 2021, 47(5): 985-1004
    [26] 陈文帅, 任志刚, 吴宗泽, 付敏跃. 基于深度学习的极性电子元器件目标检测与方向识别方法. 自动化学报, 2021, 47(7): 1701-1709 doi: 10.16383/j.aas.c190037

    Chen Wen-Shuai, Ren Zhi-Gang, Wu Zong-Ze, Fu Min-Yue. Detecting object and direction for polar electronic components via deep learning. Acta Automatica Sinica, 2021, 47(7): 1701-1709 doi: 10.16383/j.aas.c190037
    [27] 吴高昌, 刘强, 柴天佑, 秦泗钊. 基于时序图像深度学习的电熔镁炉异常工况诊断. 自动化学报, 2019, 45(8): 1475-1485

    Wu Gao-Chang, Liu Qiang, Chai Tian-You, Qin S J. Abnormal condition diagnosis through deep learning of image sequences for fused magnesium furnaces. Acta Automatica Sinica, 2019, 45(8): 1475-1485
    [28] 张芳, 王萌, 肖志涛, 吴骏, 耿磊, 童军, 等. 基于全卷积神经网络与低秩稀疏分解的显著性检测. 自动化学报, 2019, 45(11): 2148-2158

    Zhang Fang, Wang Meng, Xiao Zhi-Tao, Wu Jun, Geng Lei, Tong Jun, et al. Saliency detection via full convolution neural network and low rank sparse decomposition. Acta Automatica Sinica, 2019, 45(11): 2148-2158
    [29] 李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727-1742 doi: 10.16383/j.aas.2018.c170052

    Li Liang-Fu, Ma Wei-Fei, Li Li, Lu Cheng. Research on detection algorithm for bridge cracks based on deep learning. Acta Automatica Sinica, 2019, 45(9): 1727-1742 doi: 10.16383/j.aas.2018.c170052
    [30] Lin T Y, Goyal P, Girshick R, He K M, Piotr D. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2999−3007
    [31] Yu J H, Jiang Y N, Wang Z Y, Cao Z M, Huang T. UnitBox: An advanced object detection network. In: Proceedings of the 24th ACM International Conference on Multimedia. Amsterdam, The Netherlands: ACM, 2016. 516−520
    [32] Çelik H İ, Dülger L C, Topalbekiroğlu M. Fabric defect detection using linear filtering and morphological operations. Indian Journal of Fibre and Textile Research, 2014, 39(3): 254-259
    [33] Ng M K, Ngan H Y T, Yuan X M, Zhang W X. Patterned fabric inspection and visualization by the method of image decomposition. IEEE Transactions on Automation Science and Engineering, 2014, 11(3): 943-947 doi: 10.1109/TASE.2014.2314240
    [34] Ren R X, Hung T, Tan K C. A generic deep-learning-based approach for automated surface inspection. IEEE Transactions on Cybernetics, 2018, 48(3): 929-940 doi: 10.1109/TCYB.2017.2668395
    [35] Çelik H İ, Dülger L C, Topalbekiroğlu M. Development of a machine vision system: Real-time fabric defect detection and classification with neural networks. The Journal of The Textile Institute, 2014, 105(6): 575-585 doi: 10.1080/00405000.2013.827393
    [36] Bissi L, Baruffa G, Placidi P, Ricci E, Scorzoni A, Valigi P. Automated defect detection in uniform and structured fabrics using Gabor filters and PCA. Journal of Visual Communication and Image Representation, 2013, 24(7): 838-845 doi: 10.1016/j.jvcir.2013.05.011
    [37] Li Y D, Zhao W G, Pan J H. Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Transactions on Automation Science and Engineering, 2017, 14(2): 1256-1264 doi: 10.1109/TASE.2016.2520955
    [38] Jing J F, Ma H, Zhang H H. Automatic fabric defect detection using a deep convolutional neural network. Coloration Technology, 2019, 135(3): 213-223 doi: 10.1111/cote.12394
    [39] Raheja J L, Ajay B, Chaudhary A. Real time fabric defect detection system on an embedded DSP platform. Optik, 2013, 124(21): 5280-5284 doi: 10.1016/j.ijleo.2013.03.038
    [40] Tao X, Zhang D P, Ma W Z, Liu X L, Xu D. Automatic metallic surface defect detection and recognition with convolutional neural networks. Applied Sciences, 2018, 8(9): Article No. 1575
    [41] He D, Xu K, Zhou P, Zhou D D. Surface defect classification of steels with a new semi-supervised learning method. Optics and Lasers in Engineering, 2019, 117: 40-48 doi: 10.1016/j.optlaseng.2019.01.011
    [42] Arikan S, Varanasi K, Stricker D. Surface defect classification in real-time using convolutional neural networks. arXiv preprint arXiv: 1904.04671, 2019.
  • 加载中
图(11) / 表(9)
计量
  • 文章访问数:  3343
  • HTML全文浏览量:  956
  • PDF下载量:  463
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-03-20
  • 录用日期:  2020-06-19
  • 网络出版日期:  2023-02-03
  • 刊出日期:  2023-04-20

目录

    /

    返回文章
    返回