2.793

2018影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

无纺布疵点实时检测技术与系统设计

邓泽林 刘行 董云龙 袁烨

邓泽林, 刘行, 董云龙, 袁烨. 无纺布疵点实时检测技术与系统设计. 自动化学报, 2020, 46(x): 1−11 doi: 10.16383/j.aas.c200446
引用本文: 邓泽林, 刘行, 董云龙, 袁烨. 无纺布疵点实时检测技术与系统设计. 自动化学报, 2020, 46(x): 1−11 doi: 10.16383/j.aas.c200446
Deng Ze-Lin, Liu Xing, Dong Yun-Long, Yuan Ye. Non-woven fabric real-time defects detection method and framework design. Acta Automatica Sinica, 2020, 46(x): 1−11 doi: 10.16383/j.aas.c200446
Citation: Deng Ze-Lin, Liu Xing, Dong Yun-Long, Yuan Ye. Non-woven fabric real-time defects detection method and framework design. Acta Automatica Sinica, 2020, 46(x): 1−11 doi: 10.16383/j.aas.c200446

无纺布疵点实时检测技术与系统设计

doi: 10.16383/j.aas.c200446
基金项目: 国家自然科学基金(91748112)资助
详细信息
    作者简介:

    邓泽林:2018年获得华中科技大学数学与统计学院学士学位, 2020年获得中国香港城市大学商学院硕士学位. 主要研究方向为深度学习在智能制造中的应用. E-mail:dengzelinhust@gmail.com

    刘行:2020年获华东交通大学电气与自动化工程学院学士学位. 华东理工大学信息科学与工程学院硕士研究生. 主要研究方向为深度学习, 智能控制与图像处理. E-mail: prevalent98@outlook.com

    董云龙:2017年获华中科技大学自动化学院学士学位, 华中科技大学人工智能与自动化学院博士. 主要研究方向为深度学习、控制理论和机器人. E-mail: dyl@hust.edu.cn

    袁烨:2008年获得中国上海交通大学自动化系自动化学士学位. 分别于2009年10月和2012年2月在英国剑桥剑桥大学工程系控制科学与工程硕士学位与博士学位. 现任中国武汉华中科技大学正教授. 曾担任加州大学伯克利分校博士后研究员, 剑桥大学达尔文学院的初级研究员. 主要研究包括信息物理系统的系统识别和控制. E-mail: yye@hust.edu.cn

Non-woven Fabric Real-time Defects Detection Method and Framework Design

Funds: Supported by National Natural Science Foundation of China (91748112)
  • 摘要: 无纺布生产过程中产生的疵点会严重影响产品质量并限制生产效率. 提高疵点检测的自动化程度对于无纺布的生产效率和质量管控至关重要. 传统疵点检测方法难以应对纹理、疵点类型以及环境变化等问题, 限制了其应用范围. 近年来基于卷积神经网络的方法在疵点检测领域得到了广泛应用, 具有泛化性强、准确度高的特点. 但是在无纺布生产过程中, 布匹宽度大、速度快的特点会产生大量图像数据, 基于卷积神经网络的方法难以实现实时检测. 针对上述难题, 本文提出了一种基于最大稳定极值区域分析与卷积神经网络协同的疵点实时检测方法, 并设计了分布式计算处理架构应对数据流过大的问题. 在实际生产部署应用中, 本文所设计的系统与算法无需使用专用计算硬件(GPU、FPGA等), 通过8台工控机与16路工业摄像头对复卷机上布宽2.8 m、速度30 m/min的无纺布进行分布式实时在线检测, 大幅度提高无纺布生产中疵点检测的自动化程度与效率. 本文所提出的系统能够实现对0.3 mm以上疵点召回率100%, 对0.1 mm丝状疵点召回率98.8%.
  • 图  1  无纺布生产过程中产生的瑕疵, 点状瑕疵用红色矩形框出, 丝状瑕疵用绿色矩形框出.

    Fig.  1  Defects generated in the production process of non-woven fabrics. The dotted defects are framed by red rectangles, and the filamentary defects are framed by green rectangles.

    图  2  在不同阈值$\tau $时MSER算法产生的候选区域, 红色点代表候选区域的中心点.

    Fig.  2  In the candidate regions generated by the MSER algorithm at different thresholds $\tau $, the red dots represent the center position of the candidate region.

    图  3  本文采用的预检测模型结构图. 输入疵点图像经过三个不同尺度的卷积后得到三个特征图, 特征图拼接后作为Dense Block的输入. Dense Block输出与全连接层和softmax层相连. 其中虚线矩形框出部分为展开Dense Block的具体形式.

    Fig.  3  The structure diagram of the pre-detection model used in this paper. The input defect image is convolved at three different scales to obtain three feature maps, and the feature maps are concatenated as the input of the Dense Block. The output of the Dense Block is connected to the fully connected layer and the softmax layer. Among them. The dashed rectangle outlines the specific form of the Dense Block.

    图  4  本文采用的精确检测模型. 其中${t_x},{t_y},{t_w},{t_h}$分别表示检测矩形框的横坐标、纵坐标、宽度与高度, ${\rm{C}}$表示疵点检测置信度.

    Fig.  4  The precise detection model used in this paper. ${t_x},{t_y},{t_w},{t_h}$ are the abscissa, ordinate, width and height of the detection rectangle, and ${\rm{C}}$ is the confidence of defect detection.

    图  5  NMS算法处理效果图. (a)为原图, (b)为网络预测效果图, 网络预测的矩形用红色矩形绘制出, (c)为经过NMS算法处理后的效果图.

    Fig.  5  NMS algorithm processing effect diagram.(a) is the original image, (b) is the network prediction effect diagram, the network prediction rectangle is drawn with a red rectangle, and (c) is the effect diagram processed by the NMS algorithm.

    图  7  无纺布疵点检测算法流程图.

    Fig.  7  Flow chart of non-woven fabric defect detection algorithm.

    图  8  无纺布疵点检测系统总体组成, 左边为设计示意图, 右边为现场实物图. 设计图中的(a)代表工业控制机集群, (b) 代表高速工业摄像头, 摄像头发出的的蓝色光线代表摄像头的视野. 下方的摄像头组用于预检测, 上方的摄像头组用于精确检测.

    Fig.  8  The overall composition of the non-woven defect detection system, the left is the design schematic diagram, and the right is the real machine. In the design diagram (a) represents the industrial control machine cluster, (b) represents the high-speed industrial camera, and the blue light from the camera represents the camera's field of view The lower camera group is used for pre-detection, and the upper camera group is used for precise detection.

    图  6  系统分布式设计结构.

    Fig.  6  System distributed design structure.

    图  9  疵点检测系统终端软件.

    Fig.  9  Defect detection system terminal software.

    图  12  通过本文算法对疵点进行检测的结果.

    Fig.  12  The result of defect detection through the algorithm of this paper.

    图  10  不同模型在不同阈值$\tau $下的精度对比.

    Fig.  10  Comparison of the accuracy of different models under different thresholds $\tau $.

    图  11  不同模型在不同阈值$\tau $下的召回率对比.

    Fig.  11  Comparison of recall rates of different models under different thresholds $\tau $.

    图  13  不同光照和纹理条件下疵点检测结果.

    Fig.  13  Defect detection results under different lighting and texture conditions.

    表  1  最大稳定极值区域算法选取候选点

    Table  1  Maximum stable extreme value region algorithm to select candidate points

    输入: 灰度图像阈值${{\rm{I}}_g}$, $\tau \in {{\mathbb{R}}^{\rm{ + }}}$.
    输出: 候选点坐标$P$.
    步骤1. 求解灰度图像的最大值${t_{\max }} = \max ({I_g})$与最小值${t_{\min }} = \min ({I_g})$, 最大稳定区域初始化为${I_s} = {\rm{O}}$, 初始化阈值$t{\rm{ = }}{t_{\min }} + 1$.
    步骤2. 取阈值$t - 1$, $t$, $t + 1$. 其中$t \in \{ {t_{min}},{t_{min}} + 1,$ ${t_{min}} + 2, \cdots ,{t_{max}}\} $, 对灰度图像${I_g}$进行阈值操作得到二值图像${B_{t - 1}} = \left( {{I_g} < t - 1} \right)$, ${B_t} = \left( {{I_g} < t} \right)$, ${B_{t + 1}} = $ $({I_g} < t + 1)$.
    步骤3. 对${B_{t - 1}}$, ${B_t}$, ${B_{t + 1}}$求联通域$\{ Q_{t - 1}^i,Q_t^i,Q_{t + 1}^i|i = 1,2, \cdots ,M\}$.
    步骤4. 计算联通域$Q_{t - 1}^i$, $Q_t^i$, $Q_{t + 1}^i$的面积$A_{t - 1}^i$, $A_t^i$, $A_{t + 1}^i$.
    步骤5. 如果联通区域随阈值的变化率$|A_{t + 1}^i - A_{t - 1}^i|/$ $|A_t^i| \leqslant \tau $, 则此区域为稳定区域, 使${I_s} = {I_s} \cup Q_t^i$.
    步骤6. 如果$t < {t_{\max }}$, 则$t$自增1, 重复步骤2-5.
    步骤7. 得到最大稳定区域${I_s}$, 对其求四邻域联通域, 其联通域的质心坐标即为候选点坐标$P$.
    下载: 导出CSV

    表  2  非极大值抑制算法

    Table  2  Non-maximum suppression algorithm

    输入: 模型预测输出$n$个外接矩形集${\mathbb{B}} = \{ {B_i}|i = 0,1,$ $\cdots ,n\}$, 及其属于疵点的置信度集合${\mathbb{C} } = \{ {C_i}|i = 0,1, \cdots ,n\}$, $IoU$的阈值$T \in (0,1)$.
    输出: 极大外接矩形集合${\mathbb{M}}$.
    步骤1. 极大外接矩形框集合${\mathbb{M}} = \emptyset $.
    步骤2. 求解疵点置信度集合${\mathbb{C}}$中的最大值${C_m}$, 及其对应的外接矩形${B_m}$.
    步骤3. 将${B_m}$存入极大外接矩形集合中${\mathbb{M}} \leftarrow {\mathbb{M}} \cup {B_m}$, 并将${B_m}$${C_m}$从集合中剔除${\mathbb{B}} \leftarrow {\mathbb{B}} - {B_m}$, ${\mathbb{C}} \leftarrow $ ${\mathbb{C}} - {C_m}$.
    步骤4. 计算矩形框集合中元${B_i} \in {\mathbb{M}}$${B_m}$之间的$IoU$, 记为$Io{U^i}$.
    步骤5. 如果$Io{U^i} > T$, 则将${B_m}$, ${C_m}$从集合中剔除${\mathbb{B}} \leftarrow {\mathbb{B}} - {B_m}$, ${\mathbb{C}} \leftarrow {\mathbb{C}} - {C_m}$.
    步骤6. 如果${\mathbb{B}} \ne \emptyset $${\mathbb{C}} \ne \emptyset $, 则重复步骤2-5.
    步骤7. 返回极大外接矩形集合${\mathbb{M}}$.
    下载: 导出CSV

    表  3  训练超参数配置

    Table  3  Training hyperparameter configuration

    参数类型取值
    输入尺寸48×48
    训练轮数100
    学习率$1{e^{ - 4}}$
    衰减因子0.99
    批次大小8
    优化器Adam[34]
    下载: 导出CSV

    表  4  硬件设备参数

    Table  4  Hardware device information

    硬件类型参数
    摄像头像素分辨率物距视角
    500w800*60020-10 m75度
    控制器处理器内存固态显卡
    i58G64GIntel HD
    交换机端口数端口参数
    8个1000M自适应RJ45端口
    下载: 导出CSV

    表  5  模型在不同阈值下的检测精度和召回率

    Table  5  The detection accuracy and recall rate of the model under different thresholds

    模型指标$\tau {\rm{ = 20}}$$\tau {\rm{ = 30}}$$\tau {\rm{ = 40}}$
    预检测精度73.0%80.5%86.4%
    召回率100.0%100.0%99.3%
    精确模型精度86.1%94.5%98.4%
    召回率100.0%100.0%99.7%
    下载: 导出CSV

    表  6  模型预测速度测试

    Table  6  Model prediction speed test

    模型48×48候选区域2400×600无纺布图像
    预检测模型3.2 ms46.8 ms
    精确检测模型13.9 ms213.3 ms
    下载: 导出CSV

    表  7  不同类型疵点的检测精度和召回率测试

    Table  7  Testing accuracy and recall rate of different types of defects

    疵点类型疵点尺寸召回率精度
    点状瑕疵0.3 mm100%98.8%
    0.1 mm-0.3 mm99.6%98.3%
    丝状瑕疵0.1 mm以上98.8%98.6%
    0.05 mm-0.1 mm96.8%97.2%
    下载: 导出CSV
  • [1] 周树照. 论无纺布的现状与发展. 山东工业技术, 2017, (20): 273

    Zhou S Z. On the status quo and development of non-woven fabrics. Shandong Industrial Technology, 2017, (20): 273
    [2] De Baar H J W. von Liebig's law of the minimum and plankton ecology (1899–1991). Progress in Oceanography, 1994, 33(4): 347−386 doi: 10.1016/0079-6611(94)90022-1
    [3] 陶显, 侯伟, 徐德. 基于深度学习的表面缺陷检测方法综述. 自动化学报, 2020, 1-19[2020-06-21].

    Tao X, Hou W, Xu D. A survey of surface defect detection methods based on deep learning. Acta Automatica Sinica, 2020, 1−19[2020-06-21].
    [4] 刘伟斌, 郑力新, 周凯汀. 采用频域滤波的织物疵点检测方法. 华侨大学学报(自然科学版), 2017, (4): 562−566

    Liu W B, Zheng L X, Zhou K T. Detection method of fabric defects based on frequency domain filtering. Journal of Huaqiao University(Natural Science), 2017, (4): 562−566
    [5] 杨亚, 薛云灿, 沙伟, 等. 基于正交小波分解的织物疵点检测. 微处理机, 2015, 36(3): 46−49 doi: 10.3969/j.issn.1002-2279.2015.03.014

    Yang Y, Xue Y C, Sha W, et al. Fabric defect detection based on orthonormal wavelet decomposition. Micro- processors, 2015, 36(3): 46−49 doi: 10.3969/j.issn.1002-2279.2015.03.014
    [6] 曹媛媛, 杨波, 等. 基于分形纹理特征和小波变换的网状纹理检测方法. 自动化学报, 2007, 33(7): 688−692

    Cao Y Y, Yang B. Netlike texture detection method using fractal texture features and wavelet transform. Acta Automatica Sinica, 2007, 33(7): 688−692
    [7] 吴秀永, 徐科, 徐金梧. 基于Gabor小波和核保局投影算法的表面缺陷自动识别方法. 自动化学报, 2010, 36(3): 438−441 doi: 10.3724/SP.J.1004.2010.00438

    Wu X Y, Xu K, Xu J W. Automatic recognition method of surface defects based on gabor wavelet and kernel locality preserving projections. Acta Automatica Sinica, 2010, 36(3): 438−441 doi: 10.3724/SP.J.1004.2010.00438
    [8] 王传桐, 胡峰, 徐启永, 等. 采用Gabor滤波簇和等距映射算法的织物疵点检测方法. 纺织学报, 2017, 38(03): 162−167

    Wang C T, Hu F, Xu Q Y, et al. Detection of fabric defects based on Gabor filters and Isomap. Journal of Textile Research, 2017, 38(03): 162−167
    [9] 孙君顶, 李欣, 盛娜, 等. 基于MBLBPV算法的布匹瑕疵检测方法. 测控技术, 2019, 38(1): 71−76

    Sun J D, Li X, Sheng N, et al. Fabric defect detection based on multi-scale block binary patterns variance. Measurement & Control Technology, 2019, 38(1): 71−76
    [10] 张沐光, 宋执环. LPMVP算法及其在故障检测中的应用. 自动化学报, 2009, 35(6): 766−772 doi: 10.3724/SP.J.1004.2009.00766

    Zhang M G, Song Z H. LPMVP algorithm and its application to fault detection. Acta Automatica Sinica, 2009, 35(6): 766−772 doi: 10.3724/SP.J.1004.2009.00766
    [11] 董蓉, 李勃, 徐晨. 应用积分图的织物瑕疵检测快速算法. 纺织学报, 2016, 37(11): 141−147

    Dong R, Li B, Xu C. Fast fabric defect detection algorithm based on integral image. Journal of Textile Research, 2016, 37(11): 141−147
    [12] 杨晓波. 基于GMFR模型的统计特征畸变织物疵点识别. 纺织学报, 2013, 34(4): 137−142 doi: 10.3969/j.issn.0253-9721.2013.04.027

    Yang X B. Fabric defect detection of statistic aberration feature based on GMRF model. Journal of Textile Research, 2013, 34(4): 137−142 doi: 10.3969/j.issn.0253-9721.2013.04.027
    [13] 李良福, 马卫飞, 李丽, 陆铖. 基于深度学习的桥梁裂缝检测算法研究. 自动化学报, 2019, 45(9): 1727−1742

    Li L F, Ma W F, Li L, et al. Research on detection algorithm for bridge cracks based on deep learning. Acta Automatica Sinica, 2019, 45(9): 1727−1742
    [14] 金侠挺, 王耀南, 张辉, 等. 基于贝叶斯CNN和注意力网络的钢轨表面缺陷检测系统. 自动化学报, 2019, 45(12): 2312−2327

    Jin X T, Wang Y N, Zhang H, et al. DeepRail: Automatic visual detection system for railway surface defect using bayesian CNN and attention network. Acta Automatica Sinica, 2019, 45(12): 2312−2327
    [15] 王孟涛, 李岳阳, 杜帅. 基于机器视觉的疵点检测方法的研究进展. 现代纺织技术, 2019, 27(05): 57−61

    Wang M T, Li Y Y, Du S. Research progress of defect detection method based on machine vision. Advanced Textile Technology, 2019, 27(05): 57−61
    [16] Zhao Y, Hao K, He H, et al. A visual long-short-term memory based integrated CNN model for fabric defect image classification. Neurocomputing, 2020, 380: 259−270 doi: 10.1016/j.neucom.2019.10.067
    [17] DAGM 2007 Datasets[Online], available: https://hci.iwr.uni-heidelberg.de/node/3616, February 27, 2018
    [18] Kampouris C, Zafeiriou S, Ghosh A, et al. Fine-grained material classification using micro-geometry and reflectance. European Conference on Computer Vision. Springer, Cham, 2016: 778−792.
    [19] Kylberg G. Kylberg Texture dataset v.1.0. centre for image analysis, Swedish University of Agricultural Sciences and Uppsala University, 2011.
    [20] Çelik H İ, Dülger L C, Topalbekiroğlu M. Development of a machine vision system: real-time fabric defect detection and classification with neural networks. The Journal of The Textile Institute, 2014, 105(6): 575−585 doi: 10.1080/00405000.2013.827393
    [21] 车翔玖, 刘华罗, 邵庆彬. 基于Fast RCNN改进的布匹瑕疵识别算法. 吉林大学学报 (工学版), 2019, 49(6): 2038−2044

    Che X J, Liu H L, Shao Q B. Fabric defect recognition algorithm based on improved Fast RCNN. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2038−2044
    [22] Liu Z, Liu X, Li C, et al. Fabric defect detection based on faster R-CNN. Ninth International Conference on Graphic and Image Processing (ICGIP 2017). International Society for Optics and Photonics, 2018, 10615: 106150A
    [23] 邵鑫玉, 华继钊. 基于机器视觉的无纺布缺陷自动检测系统. 计算机科学, 2014, 41(S1): 487−489

    Shao X Y, Hua J Z. Automatic detection system of fabric defects based on machine vision. Computer Science, 2014, 41(S1): 487−489
    [24] 李明, 景军锋, 李鹏飞. 应用GAN和Faster R-CNN的色织物缺陷识别. 西安工程大学学报, 2018, 32(06): 663−669

    Li M, Jing J F, Li P F. Yarn-dyed fabric defect detection based on GAN and Faster R-CNN. Journal of Xi’an Polytechnic University, 2018, 32(06): 663−669
    [25] Matas J, Chum O, Urban M, et al. Robust wide-baseline stereo from maximally stable extremal regions. Image and Vision Computing, 2004, 22(10): 761−767 doi: 10.1016/j.imavis.2004.02.006
    [26] 贺智明, 彭亚楠. 基于深度学习的织物疵点检测研究进展. 毛纺科技, 2019, 47(08): 83−88

    He Z M, Peng Y N. Progress in fabric defect detection based on deep learning. Wool Textile Journal, 2019, 47(08): 83−88
    [27] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4700−4708.
    [28] Deng J, Guo J, Zhou Y, et al. Retinaface: Single-stage dense face localisation in the wild. arXiv preprint arXiv: 1905.00641, 2019.
    [29] Lin T, Dollár P, Girshick R, et al. Feature pyramid networks for object detection. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 2017: 2117−2125.
    [30] Redmon J, Farhadi A. Yolov3: An incremental improvement. arXiv preprint arXiv: 1804.02767, 2018.
    [31] Zhou D, Fang J, Song X, et al. ou loss for 2D/3D object detection. 2019 International Conference on 3D Vision (3DV). IEEE, 2019: 85−94.
    [32] Zheng Z, Wang P, Liu W, et al. Distance-IoU loss: faster and better learning for bounding box regression. arXiv preprint arXiv: 1911.08287, 2019.
    [33] 童艳, 孙君亮, 王鹏宇. 实时测控数据处理集群软件测试系统设计. 自动化技术与应用, 2019, 38(06): 11 doi: 10.3969/j.issn.1003-7241.2019.06.003

    Tong Y, Sun J L, Wang P Y. Design of test system for real-time measurement and control data processing cluster. Techniques of Automation and Applications, 2019, 38(06): 11 doi: 10.3969/j.issn.1003-7241.2019.06.003
    [34] Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv: 1412.6980, 2014.
    [35] Liu W, Anguelov D, et al. Ssd: Single shot multibox detector. European Conference on Computer Vision, 2016: 21−37.
  • 加载中
计量
  • 文章访问数:  9
  • HTML全文浏览量:  6
  • 被引次数: 0
出版历程

目录

    /

    返回文章
    返回