-
摘要: 布匹瑕疵检测是纺织工业中产品质量评估的关键环节, 实现快速、准确、高效的布匹瑕疵检测对于提升纺织工业的产能具有重要意义. 在实际布匹生产过程中, 布匹瑕疵在形状、大小及数量分布上存在不平衡问题, 且纹理布匹复杂的纹理信息会掩盖瑕疵的特征, 加大布匹瑕疵检测难度. 本文提出基于深度卷积神经网络的分类不平衡纹理布匹瑕疵检测方法(Detecting defects in imbalanced texture fabric based on deep convolutional neural network, ITF-DCNN), 首先建立一种基于通道叠加的ResNet50卷积神经网络模型(ResNet50+)对布匹瑕疵特征进行优化提取; 其次提出一种冗余特征过滤的特征金字塔网络(Filter-feature pyramid network, F-FPN)对特征图中的背景特征进行过滤, 增强其中瑕疵特征的语义信息; 最后构造针对瑕疵数量进行加权的MFL (Multi focal loss)损失函数, 减轻数据集不平衡对模型的影响, 降低模型对于少数类瑕疵的不敏感性. 通过实验对比, 提出的方法能有效提升布匹瑕疵检测的准确率及定位精度, 同时降低了布匹瑕疵检测的误检率和漏检率, 明显优于当前主流的布匹瑕疵检测算法.Abstract: Fabric defect detection is a key part of product quality assessment in the textile industry. Achieving fast, accurate and efficient fabric defect detection is of great significance for improving the productivity of the textile industry. In the production process of fabric, imbalance exists in the shape, size and quantity distribution of fabric defects, and the complex texture information of the jacquard fabric will cover the characteristics of the defect, which makes it difficult to detect fabric defects. This paper proposes a method for detecting defects in imbalanced texture fabric based on deep convolutional neural network (ITF-DCNN). First, an improved ResNet50 convolutional neural network model (ResNet50+) based on channel concatenate is established to optimize the fabric defect features. Second, F-FPN (filter-feature pyramid network) method for filtering redundant feature is proposed to filter the background features in the feature maps and enhance the semantic information of defect features. Finally, a MFL (multi focal loss) function weighted with the number of defects is construct to reduce the impact of imbalance on the model, and reduce the model's insensitivity to a small number of defects. Experiments shows the proposed method effectively improves the accuracy of fabric defect detection and the accuracy of defect positioning, while reducing the false detection rate and missed detection rate of defect detection, which is significantly higher than the mainstream fabric defect detection algorithm.
-
表 1 增强前后数据集中的样本分布
Table 1 Samples distribution of the dataset before and after data augmentation
瑕疵类别 沾污 花毛 虫粘 破洞 蜡斑 网折 其他 正常 总计 训练集增强前 2432 398 208 122 73 77 52 2756 6118 训练集增强后 9728 1594 834 490 292 306 206 11026 24476 验证集增强前 141 33 12 6 2 5 4 420 623 验证集增强后 562 134 48 22 6 22 14 1672 2490 表 2 数据集增强前后模型准确率对比实验结果 (%)
Table 2 Experimental results of model on accuracy before and after dataset enhancement (%)
瑕疵类别 沾污 花毛 虫粘 破洞 蜡斑 网折 其他 正常 总计 数据集增强前 88.24 83.36 87.56 89.36 83.78 88.21 89.65 98.66 88.61 数据集增强后 90.56 85.51 90.35 91.42 87.64 89.24 90.02 99.81 90.57 表 3 数据集增强前后模型mAP对比实验结果 (%)
Table 3 Experimental results of model on mAP before and after dataset enhancement (%)
瑕疵类别 沾污 花毛 虫粘 破洞 蜡斑 网折 其他 正常 总计 数据集增强前 69.06 58.51 81.50 83.44 33.33 63.70 45.51 — 62.15 数据集增强后 70.04 59.12 83.23 83.54 35.78 63.70 47.31 — 63.25 表 4 不同模型在布匹瑕疵数据集上的实验结果 (%)
Table 4 Experimental results of different models on the jacquard fabric defect dataset (%)
检测器 主干网络 mAP 准确率 误检率 漏检率 Faster R-CNN ResNet50 65.56 87.40 12.60 1.42 Cascade R-CNN ResNet50 63.77 90.55 9.45 2.85 RetinaNet ResNet50 65.60 53.86 46.13 0.20 Faster R-CNN ResNet101 63.85 88.72 11.28 2.24 Cascade R-CNN ResNet101 64.60 90.35 9.65 1.83 RetinaNet ResNet101 66.52 56.23 43.77 0.12 GLCM — — 64.63 35.37 6.87 Gabor — — 83.87 16.13 1.67 GMM — — 81.32 18.68 1.77 PTIT[38] — — 92.56 7.44 0.94 CAE-SGAN[41] — — 85.01 14.99 2.65 SurfNet[42] — — 84.82 15.18 1.79 ITF-DCNN ResNet50 73.41 97.56 2.44 1.65 ITF-DCNN ResNet101 73.92 97.66 2.34 1.14 表 5 改进后的ResNet50+ 网络性能对比实验 (%)
Table 5 Experimental performance result of ResNet50+ (%)
mAP 准确率 误检率 漏检率 ResNet50 63.77 90.55 9.45 2.85 ResNet50+I 63.68 91.76 8.24 2.66 ResNet50+C 64.14 92.31 7.69 3.43 ResNet50+ 64.72 92.78 7.22 2.91 表 6 F-FPN性能验证实验结果 (%)
Table 6 Experimental performance result of F-FPN (%)
mAP 准确率 误检率 漏检率 Top-Down FPN 63.77 90.55 9.45 2.85 PANet 65.69 92.23 7.77 2.56 加性F-FPN 70.31 93.65 6.53 1.95 卷积F-FPN 71.42 96.72 3.28 1.25 表 7 MFL的性能验证实验结果
Table 7 Experimental performance result of MFL
损失函数 $ \alpha $ $ \gamma $ $ \omega $ mAP (%) 准确率 (%) 误检率 (%) 漏检率 (%) CE — — — 63.77 90.55 9.45 2.85 FL 0.25 5.0 — 52.53 70.23 29.77 9.56 FL 0.25 2.0 — 65.62 92.86 7.14 2.02 FL 0.25 1.0 — 64.88 91.91 8.09 2.12 FL 0.50 0.5 — 64.74 91.55 8.45 2.33 FL 0.75 0.2 — 59.27 83.02 16.98 8.50 FL 0.75 0.1 — 58.11 80.85 19.15 7.56 FL 0.75 0.0 — 58.01 81.22 18.78 7.66 MFL — 1.0 0.618 68.21 94.39 5.61 1.44 MFL — 2.0 0.618 70.12 95.32 4.68 1.68 MFL — 5.0 0.618 68.11 94.50 5.50 1.56 MFL — 2.0 0.100 67.22 93.68 6.32 2.26 MFL — 2.0 0.300 69.03 94.88 5.12 1.56 MFL — 2.0 1.000 69.22 95.17 4.83 1.29 MFL — 2.0 2.000 68.81 94.35 5.65 1.68 MFL — 2.0 5.000 64.38 92.41 7.59 2.42 表 8 采用F-FPN的模型在不同模板上的泛化性分析 (%)
Table 8 Generalization analysis of models using F-FPN on different templates (%)
模板1 模板2 模板3 模板4 模板5 模板6 模板7 模板8 模板9 模板10 均值 准确率 95.87 96.79 99.67 93.56 91.74 91.11 93.66 99.12 98.23 93.65 95.34 mAP 69.12 69.73 75.37 68.97 67.46 68.12 68.24 75.96 76.82 68.02 70.78 表 9 采用FPN的模型在不同模板上的泛化性分析 (%)
Table 9 Generalization analysis of models using FPN on different templates (%)
模板1 模板2 模板3 模板4 模板5 模板6 模板7 模板8 模板9 模板10 均值 准确率 91.63 91.04 92.38 90.39 88.34 88.12 91.25 91.75 91.42 89.11 90.54 mAP 62.43 61.75 65.86 62.01 61.99 61.08 62.51 65.63 66.74 61.46 63.07 -
[1] 张慧, 王坤峰, 王飞跃. 深度学习在目标视觉检测中的应用进展与展望. 自动化学报, 2017, 43(8): 1289-1305Zhang Hui, Wang Kun-Feng, Wang Fei-Yue. Advances and perspectives on applications of deep learning in visual object detection. Acta Automatica Sinica, 2017, 43(8): 1289-1305 [2] Kumar A. Computer-vision-based fabric defect detection: A survey. IEEE Transactions on Industrial Electronics, 2008, 55(1): 348-363 doi: 10.1109/TIE.1930.896476 [3] Chan C H, Pang G K H. Fabric defect detection by Fourier analysis. IEEE Transactions on Industry Applications, 2000, 36(5): 1267-1276 doi: 10.1109/28.871274 [4] Hanbay K, Talu M F, Özgüven Ö F. Fabric defect detection systems and methods-a systematic literature review. Optik, 2016, 127(24): 11960-11973 doi: 10.1016/j.ijleo.2016.09.110 [5] Chetverikov D, Hanbury A. Finding defects in texture using regularity and local orientation. Pattern Recognition, 2002, 35(10): 2165-2180 doi: 10.1016/S0031-3203(01)00188-1 [6] Ngan H Y T, Pang G K H, Yung N H C. Automated fabric defect detection-a review. Image and Vision Computing, 2011, 29(7): 442-458 doi: 10.1016/j.imavis.2011.02.002 [7] Yapi D, Allili M S, Baaziz N. Automatic fabric defect detection using learning-based local textural distributions in the contourlet domain. IEEE Transactions on Automation Science and Engineering, 2018, 15(3): 1014-1026 doi: 10.1109/TASE.2017.2696748 [8] Ravandi S A H, Toriumi K. Fourier transform analysis of plain weave fabric appearance. Textile Research Journal, 1995, 65(11): 676-683 doi: 10.1177/004051759506501108 [9] Escofet J, Garcia-Verela M S M, Abril H C, Torrecilla E. Inspection of fabric resistance to abrasion by Fourier analysis. In: Proceedings of the SPIE 3490, Optics in Computing'98. Bruges, Belgium: SPIE, 1998. 207−210 [10] Zachevsky I, Zeevi Y Y. Model-based color natural stochastic textures processing and classification. In: Proceedings of the IEEE Global Conference on Signal and Information Processing (GlobalSIP). Orlando, USA: IEEE, 2015. 1357−1361 [11] 李敏, 崔树芹, 谢治平. 高斯混合模型在印花织物疵点检测中的应用. 纺织学报, 2015, 36(8): 94-98 doi: 10.13475/j.fzxb.20140504105Li Min, Cui Shu-Qin, Xie Zhi-Ping. Application of Gaussian mixture model on defect detection of print fabric. Journal of Textile Research, 2015, 36(8): 94-98 doi: 10.13475/j.fzxb.20140504105 [12] Allili M S, Baaziz N, Mejri M. Texture modeling using contourlets and finite mixtures of generalized Gaussian distributions and applications. IEEE Transactions on Multimedia, 2014, 16(3): 772-784 doi: 10.1109/TMM.2014.2298832 [13] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In: Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, USA: Curran Associates Inc., 2012. 1097−1105 [14] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the 3rd International Conference on Learning Representations. San Diego, USA: ICLR, 2015. [15] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016. 2818−2826 [16] Szegedy C, Ioffe S, Vanhoucke V, Alemi A A. Inception-v4, inception-ResNet and the impact of residual connections on learning. In: Proceedings of the 31st AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI, 2017. 4278−4284 [17] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning. Lille, France: JMLR.org, 2015. 448−456 [18] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, USA: IEEE, 2015. 1−9 [19] He K M, Zhang X Y, Ren S Q, Sun J. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016. 770−778 [20] Huang G, Liu Z, Van Der Maaten L, Weinberger K Q. Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 2261−2269 [21] Cai Z W, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 6154−6162 [22] Tan M X, Pang R M, Le Q V. EfficientDet: Scalable and efficient object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2019. 10778−10787 [23] Lin T Y, Dollár P, Girshick R, He K M, Hariharan B, Belongie S. Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017. 936−944 [24] 周波, 李俊峰. 结合目标检测的人体行为识别. 自动化学报, 2020, 46(9): 1961-1970 doi: 10.16383/j.aas.c180848Zhou Bo, Li Jun-Feng. Human action recognition combined with object detection. Acta Automatica Sinica, 2020, 46(9): 1961-1970 doi: 10.16383/j.aas.c180848 [25] 范家伟, 张如如, 陆萌, 何佳雯, 康霄阳, 柴文俊, 等. 深度学习方法在糖尿病视网膜病变诊断中的应用. 自动化学报, 2021, 47(5): 985-1004Fan Jia-Wei, Zhang Ru-Ru, Lu Meng, He Jia-Wen, Kang Xiao-Yang, Chai Wen-Jun, et al. Applications of deep learning techniques for diabetic retinal diagnosis. Acta Automatica Sinica, 2021, 47(5): 985-1004 [26] 陈文帅, 任志刚, 吴宗泽, 付敏跃. 基于深度学习的极性电子元器件目标检测与方向识别方法. 自动化学报, 2021, 47(7): 1701-1709 doi: 10.16383/j.aas.c190037Chen Wen-Shuai, Ren Zhi-Gang, Wu Zong-Ze, Fu Min-Yue. Detecting object and direction for polar electronic components via deep learning. Acta Automatica Sinica, 2021, 47(7): 1701-1709 doi: 10.16383/j.aas.c190037 [27] 吴高昌, 刘强, 柴天佑, 秦泗钊. 基于时序图像深度学习的电熔镁炉异常工况诊断. 自动化学报, 2019, 45(8): 1475-1485Wu Gao-Chang, Liu Qiang, Chai Tian-You, Qin S J. Abnormal condition diagnosis through deep learning of image sequences for fused magnesium furnaces. Acta Automatica Sinica, 2019, 45(8): 1475-1485 [28] 张芳, 王萌, 肖志涛, 吴骏, 耿磊, 童军, 等. 基于全卷积神经网络与低秩稀疏分解的显著性检测. 自动化学报, 2019, 45(11): 2148-2158Zhang Fang, Wang Meng, Xiao Zhi-Tao, Wu Jun, Geng Lei, Tong Jun, et al. Saliency detection via full convolution neural network and low rank sparse decomposition. Acta Automatica Sinica, 2019, 45(11): 2148-2158 [29] 李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727-1742 doi: 10.16383/j.aas.2018.c170052Li Liang-Fu, Ma Wei-Fei, Li Li, Lu Cheng. Research on detection algorithm for bridge cracks based on deep learning. Acta Automatica Sinica, 2019, 45(9): 1727-1742 doi: 10.16383/j.aas.2018.c170052 [30] Lin T Y, Goyal P, Girshick R, He K M, Piotr D. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2999−3007 [31] Yu J H, Jiang Y N, Wang Z Y, Cao Z M, Huang T. UnitBox: An advanced object detection network. In: Proceedings of the 24th ACM International Conference on Multimedia. Amsterdam, The Netherlands: ACM, 2016. 516−520 [32] Çelik H İ, Dülger L C, Topalbekiroğlu M. Fabric defect detection using linear filtering and morphological operations. Indian Journal of Fibre and Textile Research, 2014, 39(3): 254-259 [33] Ng M K, Ngan H Y T, Yuan X M, Zhang W X. Patterned fabric inspection and visualization by the method of image decomposition. IEEE Transactions on Automation Science and Engineering, 2014, 11(3): 943-947 doi: 10.1109/TASE.2014.2314240 [34] Ren R X, Hung T, Tan K C. A generic deep-learning-based approach for automated surface inspection. IEEE Transactions on Cybernetics, 2018, 48(3): 929-940 doi: 10.1109/TCYB.2017.2668395 [35] Çelik H İ, Dülger L C, Topalbekiroğlu M. Development of a machine vision system: Real-time fabric defect detection and classification with neural networks. The Journal of The Textile Institute, 2014, 105(6): 575-585 doi: 10.1080/00405000.2013.827393 [36] Bissi L, Baruffa G, Placidi P, Ricci E, Scorzoni A, Valigi P. Automated defect detection in uniform and structured fabrics using Gabor filters and PCA. Journal of Visual Communication and Image Representation, 2013, 24(7): 838-845 doi: 10.1016/j.jvcir.2013.05.011 [37] Li Y D, Zhao W G, Pan J H. Deformable patterned fabric defect detection with fisher criterion-based deep learning. IEEE Transactions on Automation Science and Engineering, 2017, 14(2): 1256-1264 doi: 10.1109/TASE.2016.2520955 [38] Jing J F, Ma H, Zhang H H. Automatic fabric defect detection using a deep convolutional neural network. Coloration Technology, 2019, 135(3): 213-223 doi: 10.1111/cote.12394 [39] Raheja J L, Ajay B, Chaudhary A. Real time fabric defect detection system on an embedded DSP platform. Optik, 2013, 124(21): 5280-5284 doi: 10.1016/j.ijleo.2013.03.038 [40] Tao X, Zhang D P, Ma W Z, Liu X L, Xu D. Automatic metallic surface defect detection and recognition with convolutional neural networks. Applied Sciences, 2018, 8(9): Article No. 1575 [41] He D, Xu K, Zhou P, Zhou D D. Surface defect classification of steels with a new semi-supervised learning method. Optics and Lasers in Engineering, 2019, 117: 40-48 doi: 10.1016/j.optlaseng.2019.01.011 [42] Arikan S, Varanasi K, Stricker D. Surface defect classification in real-time using convolutional neural networks. arXiv preprint arXiv: 1904.04671, 2019.