-
摘要: 传统的图像处理算法不能很好地对桥梁裂缝进行检测，而经典的深度学习模型直接用于桥梁裂缝的检测，效果不甚理想.针对这些问题，本文提出了一种基于深度学习的桥梁裂缝检测算法.首先，利用滑动窗口算法将桥梁裂缝图像切分为较小的桥梁裂缝面元图像和桥梁背景面元图像，并根据对面元图像的分析，提出一种基于卷积神经网络（Convolutional neural networks，CNN）的DBCC（Deep bridge crack classify）分类模型，用于桥梁背景面元和桥梁裂缝面元的识别.然后，基于DBCC分类模型结合改进的窗口滑动算法对桥梁裂缝进行检测.最后，采用图像金字塔和感兴趣区域（Region of interest，ROI）结合的搜索策略对算法进行加速.实验结果表明：与传统算法相比，本文算法具有更好的识别效果和更强的泛化能力.Abstract: The traditional image processing algorithms failed to detect the bridge cracks and the effect was not ideal if the classical deep learning models were used to detect the bridge cracks directly. In order to solve these problems, an algorithm for the detection of bridge cracks based on deep learning was proposed. Firstly, the bridge images with cracks were divided into smaller bridge crack patches and bridge background patches by using the window sliding algorithm. According to the analysis of the patches, a classification model based on convolutional neural network, called DBCC (Deep bridge crack classify), was proposed and the model was used to identify the bridge background patches and bridge crack patches. Secondly, the DBCC classification model combined with an improved window sliding algorithm was used for the detection of bridge cracks. Finally, the algorithm was accelerated by using a search strategy of combining image pyramid and ROI. The experimental results show that the algorithm has better recognition effect and stronger generalization ability compared with the traditional algorithm.
-
Key words:
- Crack detection /
- deep learning /
- convolutional neural network /
- window sliding algorithm
-
-
图 1 经典深度学习模型和桥梁裂缝特点示意图
Fig. 1 Schematic diagram of classical depth learning model and bridge crack characteristics
图 2 桥梁裂缝面元数据集人工扩增方式示意图
Fig. 2 Schematic diagram of manual expansion of bridge crack surface metadata set
图 4 桥梁裂缝面元和背景面元示意图
Fig. 4 Schematic diagram of bridge crack surface and background surface
图 9 图像金字塔和ROI加速策略示意图
Fig. 9 Schematic diagram of image pyramid and ROI acceleration strategy
图 10 DBCC模型桥梁裂缝检测结果的示意图
Fig. 10 Schematic diagram of DBCC model bridge crack detection results
图 11 基于桥梁裂缝面元的裂缝提取和定位算法流程示意图
Fig. 11 Flow chart of crack extraction and location algorithm based on bridge crack surface element
图 12 CIFAR10模型和DBCC模型对于桥梁裂缝检测结果的对比
Fig. 12 Comparison between CIFAR10 model and DBCC model for bridge crack detection
图 13 概率区分阈值Tp对于桥梁裂缝识别效果的影响
Fig. 13 Effect of probability discrimination threshold Tp on bridge crack identification
图 14 基于桥梁裂缝面元的桥梁裂缝提取算法对于检测结果的影响
Fig. 14 Influence of the bridge crack extraction algorithm based on bridge crack surface element on detection
图 15 各种裂缝检测算法对于桥梁裂缝定位准确度的影响
Fig. 15 Influence of various crack detection algorithms on accuracy of bridge crack location
图 16 主流裂缝检测算法和本文算法对于桥梁裂缝检测的效果图
Fig. 16 Detection results of the main stream crack detection algorithm and our algorithm for bridge cracks image
图 17 基于本文算法进行桥梁裂缝检测的部分结果
Fig. 17 Partial results of bridge crack detection based on our algorithm
表 1 DBCC模型的输入层至第2池化层各层的具体模型构建参数
Table 1 Modeling parameters from the input layer to the second pool layer of the DBCC model
模型 输入层 卷积层1 Max-Pooling 1 卷积层2 Ave-Pooling 2 CIFAR10 32 × 32 × 3 Conv5-1-2-32 MP3-2-0-32 Conv5-1-2-32 AVE3-2-0-32 DBCC-A 16 × 16 × 3 Conv3-1-1-32 MP3-2-0-32 Conv3-1-1-32 AVE3-2-0-32 DBCC-B 16 × 16 × 3 Conv5-1-2-32 MP3-2-0-32 Conv5-1-2-32 AVE3-2-0-32 DBCC-C 16 × 16 × 3 Conv5-1-2-32 MP2-2-0-32 Conv5-1-2-32 AVE2-2-0-32 DBCC-D 16 × 16 × 3 Conv5-1-2-32 MP2-2-0-32 Conv5-1-2-64 AVE2-2-0-64 DBCC-E 16 × 16 × 3 Conv5-1-2-32-LRN MP2-2-0-32 Conv5-1-2-64 AVE2-2-0-64 DBCC 16 × 16 × 3 Conv5-1-2-32-LRN MP2-2-0-32 Conv5-1-2-64 AVE2-2-0-64 
下载: 导出CSV
表 2 DBCC模型的第3卷积层至输出层各层的具体模型构建参数
Table 2 Modeling parameters from the third volume accumulated layer to the output layer of the DBCC model
模型 卷积层3 Ave-Pooling 3 卷积层4 FC1 Dropout层 FC2 输出层 CIFAR10 Conv5-1-2-64 AVE3-2-0-64 --- 2 --- --- Softmax DBCC-A Conv3-1-1-64 AVE3-2-0-64 --- 2 --- --- Softmax DBCC-B Conv5-1-2-64 AVE3-2-0-64 --- 2 --- --- Softmax DBCC-C Conv5-1-2-64 AVE2-2-0-64 Conv2-1-0-64 2 --- --- Softmax DBCC-D Conv5-1-2-128 AVE2-2-0-128 Conv2-1-0-256 128 --- 2 Softmax DBCC-E Conv5-1-2-128 AVE2-2-0-128 Conv2-1-0-256 128 --- 2 Softmax DBCC Conv5-1-2-128 AVE2-2-0-128 Conv2-1-0-256 128 Dropout 2 Softmax
下载: 导出CSV
表 3 人工数据集扩增方法对于DBCC模型识别准确率的影响
Table 3 Effect of artificial data set amplification on recognition accuracy of DBCC model
桥梁裂缝面元的总数 有无数据集的扩增 DBCC模型正确识别数 准确率 500 无 353 70.6 500 有 489 97.8
下载: 导出CSV
表 4 各模型对于桥梁裂缝面元识别的准确率
Table 4 Accuracy of each model for bridge crack surface element identification
行号 模型 裂缝面元的总数 裂缝面元规格(像素) 正确识别的裂缝面元数(幅) 识别的准确率(%) 0 CIFAR10 1 000 8×8 34 3.4 1 CIFAR10 1 000 16 × 16 347 34.7 2 CIFAR10 1 000 32 × 32 976 97.6 3 DBCC-A 1 000 16 × 16 954 95.4 4 DBCC-B 1 000 16 × 16 958 95.8 5 DBCC-C 1 000 16 × 16 960 96.0 6 DBCC-D 1 000 16 × 16 970 97.0 7 DBCC-E 1 000 16 × 16 973 97.3 8 DBCC 1 000 8 × 8 48 4.8 9 DBCC 1 000 16 × 16 979 97.9
下载: 导出CSV
表 5 算法加速策略对于本文识别算法的影响
Table 5 Effect of algorithm acceleration strategy on the recognition algorithm in this paper
桥梁裂缝图像的编号 未采用加速策略耗时(s) 采用加速策略耗时(s) 1 16.976065 3.345511 2 17.332235 3.324066 3 16.834522 3.295553 4 17.793048 3.500678 5 17.123332 3.349923
下载: 导出CSV
表 6 桥梁裂缝定位的位置坐标
Table 6 Position coordinates of bridge crack location
图像的编号 左上角点坐标(像素) 右下角点坐标(像素) 1 （239, 0） （381, 512） 2 （0, 98） （512, 370） 3 （0, 33） （512, 385） 4 （218, 0） （294, 512）
下载: 导出CSV
表 8 桥梁裂缝检测提取算法的量化分析对比
Table 8 Quantitative analysis and comparison of bridge crack detection and extraction algorithm
算法 迭代阈值分割算法 FFA算法 最小路径选择算法 本文算法 评价指标 Pre/Rec Pre/Rec Pre/Rec Pre/Rec 第1幅图像 15.7 %/16.3 % 89.3 %/91.5 % 90.3 %/91.5 % 97.5 %/98.7 第2幅图像 12.5 %/9.5 % 67.5 %/83.5 % 73.5 %/80.4 % 92.39 %/94.6 第3幅图像 5.6 %/5.57 % 90.5 %/90.3 % 63.3 %/64.39 % 94.5 %/96.3 第4幅图像 7.8 %/7.5 % 18.3 %/21.7 % 35.7 %/20.3 % 91.31 %/92.5 第5幅图像 10.1 %/13.4 % 73.2 %/63.7 % 91.2 %/93.5 % 91.6 %/90.4 平均运行时间(s) 0.936 3.896 11.314 3.357
下载: 导出CSV
-
[1] 马建, 孙守增, 杨琦.中国桥梁工程学术研究综述:2014.中国公路学报, 2014, 27(5):1-96 http://d.old.wanfangdata.com.cn/Periodical/zgglxb201405001Ma Jian, Sun Shou-Zeng, Yang Qi. Review on China's bridge engineering research:2014. China Journal of Highway and Transport, 2014, 27(5):1-96 http://d.old.wanfangdata.com.cn/Periodical/zgglxb201405001 [2] 魏武, 王俊杰, 蔡钊雄.基于小波和Radon变换的桥梁裂缝检测.计算机工程与设计, 2013, 34(9):3151-3157 http://d.old.wanfangdata.com.cn/Periodical/jsjgcysj201309031Wei Wu, Wang Jun-Jie, Cai Zhao-Xiong. Bridge crack detection based on wavelet and Radon transform. Computer Engineering and Design, 2013, 34(9):3151-3157 http://d.old.wanfangdata.com.cn/Periodical/jsjgcysj201309031 [3] Oh H, Garrick N W, Achenie L E K. Segmentation algorithm using iterative clipping for processing noisy pavement images. In:Proceedings of the 2nd International Conference on Imaging Technologies:Techniques and Applications in Civil Engineering. Davos, Switzerland:TRID, 1998. 138-147 https://cedb.asce.org/CEDBsearch/record.jsp?dockey=0109703 [4] Li Q Q, Liu X L. Novel approach to pavement image segmentation based on neighboring difference histogram method. In:Proceedings of the 2008 Congress on Image and Signal Processing. Sanya, Hainan, China:IEEE, 2008. 792-796 https://ieeexplore.ieee.org/document/4566413 [5] Landstrom A, Thurley M J. Morphology-based crack detection for steel slabs. IEEE Journal of Selected Topics in Signal Processing, 2012, 6(7):866-875 doi: 10.1109/JSTSP.2012.2212416 [6] Varadharajan S, Jose S, Sharma K, Wander L, Mertz C. Vision for road inspection. In:Proceedings of the 2014 IEEE Winter Conference on Applications of Computer Vision (WACV). Steamboat Springs, CO, USA:IEEE, 2014. 115-122 https://ieeexplore.ieee.org/document/6836111 [7] Jahanshahi M R, Masri S F, Padgett C W, Sukhatme G S. An innovative methodology for detection and quantification of cracks through incorporation of depth perception. Machine Vision and Applications, 2013, 24(2):227-241 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=c9ba111ad8b36c3b26d43287663c1fdb [8] Amhaz R, Chambon S, Idier J, Baltazart V. Automatic crack detection on two-dimensional pavement images:an algorithm based on minimal path selection. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(10):2718-2729 doi: 10.1109/TITS.2015.2477675 [9] Zou Q, Cao Y, Li Q Q, Mao Q Z, Wang S. CrackTree:automatic crack detection from pavement images. Pattern Recognition Letters, 2012, 33(3):227-238 http://d.old.wanfangdata.com.cn/NSTLQK/NSTL_QKJJ0226254615/ [10] Oliveira H, Correia P L. Automatic road crack detection and characterization. IEEE Transactions on Intelligent Transportation Systems, 2013, 14(1):155-168 http://d.old.wanfangdata.com.cn/Periodical/jsjfzsjytxxxb201409008 [11] Nguyen T S, Begot S, Duculty F, Avila M. Free-form anisotropy:a new method for crack detection on pavement surface images. In:Proceedings of the 18th IEEE International Conference on Image Processing (ICIP). Brussels, Belgium:IEEE, 2011. 1069-1072 https://ieeexplore.ieee.org/document/6115610 [12] Mustafa R, Mohamed E A. Concrete crack detection based multi-block CLBP features and SVM classifier. Journal of Theoretical and Applied Information Technology, 2015, 81(1):151-160 [13] Nguyen H N, Kam T Y, Cheng P Y. An automatic approach for accurate edge detection of concrete crack utilizing 2D geometric features of crack. Journal of Signal Processing Systems, 2014, 77(3):221-240 doi: 10.1007/s11265-013-0813-8 [14] Arena A, Piane C D, Sarout J. A new computational approach to cracks quantification from 2D image analysis:application to micro-cracks description in rocks. Computers & Geosciences, 2014, 66:106-120 https://www.sciencedirect.com/science/article/pii/S0098300414000107 [15] Quintana M, Torres J, Menéndez J M. A simplified computer vision system for road surface inspection and maintenance. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(3):608-619 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=e6dd7d675797cd3abc3bdd685ed72d27 [16] Shi Y, Cui L M, Qi Z Q, Meng F, Chen Z S. Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(12):3434-3445 doi: 10.1109/TITS.2016.2552248 [17] Zhang D J, Li Q Q, Chen Y, Cao M, He L, Zhang B L. An efficient and reliable coarse-to-fine approach for asphalt pavement crack detection. Image and Vision Computing, 2017, 57:130-146 doi: 10.1016/j.imavis.2016.11.018 [18] Hassan N, Mathavan S, Kamal K. Road crack detection using the particle filter. In:Proceedings of the 23rd International Conference on Automation and Computing. Huddersfield, UK:IEEE, 2017. 1-6 [19] Oliveira H, Correia P L. Road surface crack detection:Improved segmentation with pixel-based refinement. In:Proceedings of the 25th European Signal Processing Conference. Kos, Greece:IEEE, 2017. 2026-2030 https://ieeexplore.ieee.org/document/8081565/ [20] Simonyan K, Zisserman A. Two-stream convolutional networks for action recognition in videos. In:Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada:MIT Press, 2014. 568-576 http://www.oalib.com/paper/4082156 [21] Zeiler M D, Fergus R. Visualizing and understanding convolutional networks. In:Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland:Springer, 2014. 818-833 doi: 10.1007%2F978-3-319-10590-1_53 [22] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In:Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada:Curran Associates Inc., 2012. 1097-1105 https://dl.acm.org/citation.cfm?id=2999257 [23] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In:Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA:IEEE, 2015. 1-9 http://www.oalib.com/paper/4068635#.XY1zPvngmMs [24] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In:Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA:IEEE, 2016. 2818-2826 https://arxiv.org/abs/1512.00567v2 [25] Girshick R. Fast R-CNN. In:Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile:IEEE, 2015. 1440-1448 https://ieeexplore.ieee.org/document/7410526?arnumber=7410526 [26] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN:towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149 doi: 10.1109/TPAMI.2016.2577031 [27] Krizhevsky A. Convolutional deep belief networks on cifar-10[Online], available: http://www.cs.toronto.edu//kriz//conv-cifar10-aug2010.pdf, Aygyst 1, 2010 [28] Lin M, Chen Q, Yan S C. Network in network. arXiv PreprintarXiv: 1312.4400, 2014 http://www.oalib.com/paper/4042246#.XY1zuPngmMs [29] Hinton G E, Srivastava N, Krizhevsky A, Sutskever I, Salakhutdinov R R. Improving neural networks by preventing co-adaptation of feature detectors. arXiv Preprint, arXiv: 1207.0580, 2012 期刊类型引用(15)
1. 耿艳利，刘松岳，王希瑞，宣伯凯. 电动下肢假肢多路况控制策略研究. 中国康复医学杂志. 2024(02): 226-231 . 
百度学术2. 汪晓铭，黎林荣，陈长龙，孙洁，张哲文，孟巧玲，喻洪流. 人-机-环境共融的智能假肢膝关节研究进展. 中国生物医学工程学报. 2023(04): 486-501 . 百度学术3. 韩亚丽，韩子，金壮壮，徐闽海，吴应达. 一种主动型踝关节助力外骨骼设计及性能实验. 仪器仪表学报. 2023(11): 109-118 . 百度学术4. 刘克平，滕召纬，孙中波，李婉婷. 基于自适应模糊神经网络的下肢关节角度估计. 计算机仿真. 2022(09): 456-461 . 百度学术5. 盛敏，唐少波. 融合统计和几何特征的运动意图识别方法. 安庆师范大学学报(自然科学版). 2021(01): 34-39 . 百度学术6. 刘作军，许长寿，陈玲玲，张燕. 智能假肢膝关节的研发要点及其研究进展综述. 包装工程. 2021(10): 54-63 . 百度学术7. 崔建伟，李志钢，曹尔凡，杜韩，陆普东. 基于人体上肢运动信息的助残手抓取意图识别方法. 中国惯性技术学报. 2021(02): 250-255 . 百度学术8. 刘磊，杨鹏，刘作军，宋寅卯. 基于多核学习极限学习机的助行机器人运动相容性识别. 高技术通讯. 2021(09): 978-985 . 百度学术9. 苏本跃，倪钰，盛敏，赵丽丽. 基于改进卷积神经网络的动力下肢假肢运动意图识别. 控制与决策. 2021(12): 3031-3038 . 百度学术10. 黄品高，黄剑平，黄博俊，方鹏，刘志远，李光林. 实现下肢假肢智能仿生控制的神经功能重建及行走意图识别方法. 中国科学基金. 2021(S1): 227-235 . 百度学术11. 滕召纬，孙中波，刘克平. 基于自适应模糊神经网络的下肢关节运动意图估计. 长春工业大学学报. 2020(06): 558-562 . 百度学术12. 盛敏，刘双庆，王婕，苏本跃. 基于GMM-HMM模型的智能下肢假肢运动意图识别. 仪器仪表学报. 2019(05): 169-178 . 百度学术13. 刘双庆. 基于LSTM深度学习模型的智能下肢假肢运动意图识别. 合肥学院学报(综合版). 2019(05): 96-104 . 百度学术14. 杨帆，王远，郭鹏程. 针对下肢残障人士的无障碍卫浴产品设计分析. 工业设计. 2019(11): 48-49 . 百度学术15. 刘磊，陈增强，杨鹏，刘作军. 采用鱼群算法进化极限学习机的假肢步态识别. 振动.测试与诊断. 2019(06): 1225-1231+1360-1361 . 百度学术其他类型引用(22)
-
计量
- 文章访问数: 3584
- HTML全文浏览量: 1203
- PDF下载量: 591
- 被引次数: 37
