The Key Technology and Application of Multi-modal Fine Robot Inspection for Power Facilities
-
摘要: 电力设施巡检对于加快电网基础设施智能化改造和智能微电网建设, 提高电力系统互补互济和智能调节能力的需求具有重要作用, 近年来, 智能巡检机器人开始在电力巡检中广泛应用. 在提高电力设施巡检效率和准确性、提升安全性、降低成本和促进电力智能化发展等方面发挥关键作用. 本文从电力巡检机器人的智能感知和导航技术出发, 重点介绍目标检测、语义分割、自主导航等共性关键技术的国内外发展现状. 然后以可见光红外双光融合、可见光图像和点云数据融合、声纹和可见光融合为例, 阐述电力场景多模态数据融合方式. 并进一步介绍电力部件精准分割和异物检测、线路点云杆塔倾斜检测、输电线路覆冰多模态检测和电力架空线路缺陷分析及台账异常检测等电力设施多模态机器人相关案例. 最后探讨电力设施多模态精细化机器人巡检关键技术的发展趋势和所面临的挑战.Abstract: Power facilities inspection plays an important role in accelerating the intelligent transformation of power grid infrastructure and the construction of intelligent microgrid, and improving the complementary and intelligent adjustment ability of power system. In recent years, intelligent inspection robots have been widely used in power inspection. They play a key role in improving the efficiency and accuracy of power facilities inspection, improving safety, reducing costs and promoting the development of power intelligence. This paper starts from the intelligent perception and navigation technology of power inspection robots. This paper focuses on the development status of common key technologies such as target detection, semantic segmentation and autonomous navigation at home and abroad, and then takes visible light infrared dual-light fusion, visible light image and point cloud data fusion, voiceprint and visible light fusion as examples. This paper expounds the multi-modal data fusion method of power scene, and further introduces the related cases of multi-modal robots for power facilities, such as accurate segmentation and foreign body detection of power components, tilt detection of line point cloud towers, multi-modal detection of transmission line icing, defect analysis of power overhead lines and abnormal detection of ledgers, etc.. Finally, the development trend and challenges of key technologies for multi-modal fine robot inspection of power facilities are discussed.
-
Key words:
- Power facilities inspection /
- robots /
- intelligent perception /
- multi-modal /
- transmission lines
-
表 1 电力设施机器人巡检智能感知关键技术
Table 1 Key technologies for intelligent perception in robotic inspection of power facilities
表 2 2D语义分割技术分类
Table 2 Classification of 2D semantic segmentation techniques
表 3 可见光红外双光融合方法分类
Table 3 Classification of visible light infrared dual-light fusion methods
方法 优点 缺点 基于多尺度变换的方法 多层次子图像保留了更多图像细节信息 基于预先设定的基函数进行图像融合,
易忽略源图像部分重要特征基于稀疏表示的方法 超完备字典蕴涵丰富的基原子, 有利于图像更好的表达和提取 难以应对复杂图像融合 基于神经网络的方法 避免了传统算法手动设计复杂的分解级别和融合规则, 并有效保留源图像信息 对计算资源需求较大, 暂未大量应用 表 4 电力设施多模态精细化巡检应用
Table 4 Application of multi-modal fine inspection in power facilities
典型案例 融合模态 方法原理 电力部件精准分割 可见光+红外 结合可见光的高分辨率和红外图像的温度特性, 实现高精度部件分割 输电线路异物检测 可见光+点云 结合可见光的高分辨率和点云的形态特征, 充分识别线路异物 线路杆塔倾斜检测 可见光+点云 结合可见光的颜色信息和点云的位置信息, 准确分割杆塔, 实现倾斜度检测 线路覆冰多模态检测 可见光+点云 结合可见光的颜色信息和点云的位置信息定位电力线, 通过坐标计算覆冰厚度 台账异常检测 可见光+点云 通过可见光实现台账目标识别, 利用点云构建数字孪生网络以实现台账校准 -
[1] 吴庆, 赵涛, 佃松宜, 郭锐, 李胜川, 方红帏, 等. 基于FPSO的电力巡检机器人的广义二型模糊逻辑控制. 自动化学报, 2022, 48(6): 1482−1492Wu Qing, Zhao Tao, Dian Song-Yi, Guo Rui, Li Sheng-Chuan, Fang Hong-Wei, et al. General type-2 fuzzy logic control for a power-line inspection robot based on FPSO. Acta Automatica Sinica, 2022, 48(6): 1482−1492 [2] Dian S Y, Chen L, Hoang S, Pu M, Liu J Y. Dynamic balance control based on an adaptive gain-scheduled backstepping scheme for power-line inspection robots. IEEE/CAA Journal of Automatica Sinica, 2019, 6(1): 198−208 doi: 10.1109/JAS.2017.7510721 [3] 王耀南, 江一鸣, 姜娇, 张辉, 谭浩然, 彭伟星, 等. 机器人感知与控制关键技术及其智能制造应用. 自动化学报, 2023, 49(3): 494−513Wang Yao-Nan, Jiang Yi-Ming, Jiang Jiao, Zhang Hui, Tan Hao-Ran, Peng Wei-Xing, et al. Key technologies of robot perception and control and its intelligent manufacturing applications. Acta Automatica Sinica, 2023, 49(3): 494−513 [4] 张振国, 毛建旭, 谭浩然, 王耀南, 张雪波, 江一鸣. 重大装备制造多机器人任务分配与运动规划技术研究综述. 自动化学报, 2024, 50(1): 21−41Zhang Zhen-Guo, Mao Jian-Xu, Tan Hao-Ran, Wang Yao-Nan, Zhang Xue-Bo, Jiang Yi-Ming. A review of task allocation and motion planning for multi-robot in major equipment manufacturing. Acta Automatica Sinica, 2024, 50(1): 21−41 [5] Li J Q, Xu Y Q, Nie K H, Cao B F, Zuo S N, Zhu J. PEDNet: A lightweight detection network of power equipment in infrared image based on YOLOv4-tiny. IEEE Transactions on Instrumentation and Measurement, 2023, 72: Article No. 5004312 [6] Chen G, Shao F, Chai X L, Chen H W, Jiang Q P, Meng X C, et al. CGMDRNet: Cross-guided modality difference reduction network for RGB-T salient object detection. IEEE Transactions on Circuits and Systems for Video Technology, 2022, 32(9): 6308−6323 doi: 10.1109/TCSVT.2022.3166914 [7] Koji K, Takehisa Y, Tomohiro M, Katsuyoshi S. Development of substation patrol robot, overhead distribution line works manipulator and underground cable conduit monitor robot. Robot, 1988, 64: 65−73 [8] Masafumi N. Development of a patrolling and inspection robot at 500 kV Transformer substation. (Results of field test). R&D News Kansai, 1999, 386: 13−15 [9] Allan J F, Beaudry J. Robotic systems applied to power substations——A state-of-the-art survey. In: Proceedings of the 3rd International Conference on Applied Robotics for the Power Industry (CARPI). Foz do Iguacu, Brazil: IEEE, 2014. 1−6 [10] Beaudry J, Poirier S. Véhicule Téléopéré Pour Inspection Visuelle Et Thermographique Dans Les Postes De Transformation, Technical Report IREQ-2012-0121, IREQ (Hydro-Québec Research Institute), Canada, 2012. [11] 鲁守银, 钱庆林, 张斌, 王明瑞, 李向东, 王宏. 变电站设备巡检机器人的研制. 电力系统自动化, 2006, 30(13): 94−98 doi: 10.3321/j.issn:1000-1026.2006.13.020Lu Shou-Yin, Qian Qing-Lin, Zhang Bin, Wang Ming-Rui, Li Xiang-Dong, Wang Hong. Development of a mobile robot for substation equipment inspection. Automation of Electric Power Systems, 2006, 30(13): 94−98 doi: 10.3321/j.issn:1000-1026.2006.13.020 [12] Guo R, Han L, Cheng X Q. Omni-directional vision for robot navigation in substation environments. In: Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO). Guiling, China: IEEE, 2009. 1272−1275 [13] Guo R, Xiao P, Han L, Cheng X Q. GPS and DR integration for robot navigation in substation environments. In: Proceedings of the IEEE International Conference on Information and Automation. Harbin, China: IEEE, 2010. 2009−2012 [14] Zhou Z Y, Zhang C T, Xu C, Xiong F, Zhang Y, Umer T. Energy-efficient industrial internet of UAVs for power line inspection in smart grid. IEEE Transactions on Industrial Informatics, 2018, 14(6): 2705−2714 doi: 10.1109/TII.2018.2794320 [15] Vapnik V N. The Nature of Statistical Learning Theory. New York: Springer, 2000.Vapnik V N. The Nature of Statistical Learning Theory. New York: Springer, 2000. [16] Ho T K. Random decision forests. In: Proceedings of the 3rd International Conference on Document Analysis and Recognition. Montreal, Canada: IEEE, 1995. 278−282 [17] 何洪英, 姚建刚, 蒋正龙, 汪新秀, 李伟伟. 基于支持向量机的高压绝缘子污秽等级红外热像检测. 电力系统自动化, 2005, 29(24): 70−74, 82He Hong-Ying, Yao Jian-Gang, Jiang Zheng-Long, Wang Xin-Xiu, Li Wei-Wei. Infrared thermal image detecting of high voltage insulator contamination grades based on support vector machine. Automation of Electric Power Systems, 2005, 29(24): 70−74, 82 [18] Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez A N, et al. Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: ACM, 2017. 6000−6010 [19] Carion N, Massa F, Synnaeve G, Usunier N, Kirillov A, Zagoruyko S. End-to-end object detection with transformers. In: Proceedings of the 16th European Conference on Computer Vision. Glasgow, UK: Springer, 2020. 213−229 [20] Zhu X Z, Su W J, Lu L W, Li B, Wang X G, Dai J F. Deformable DETR: Deformable Transformers for end-to-end object detection. In: Proceedings of the 9th International Conference on Learning Representations. Vienna, Austria: OpenReview, 2021.Zhu X Z, Su W J, Lu L W, Li B, Wang X G, Dai J F. Deformable DETR: Deformable Transformers for end-to-end object detection. In: Proceedings of the 9th International Conference on Learning Representations. Vienna, Austria: OpenReview, 2021. [21] Zong Z F, Song G L, Liu Y. DETRs with collaborative hybrid assignments training. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). Paris, France: IEEE, 2023. 6725−6735 [22] Zhang S L, Wang X J, Wang J Q, Pang J M, Lyu C Q, Zhang W W, et al. Dense distinct query for end-to-end object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada: IEEE, 2023. 7329−7338 [23] Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, USA: IEEE, 2014. 580−587 [24] Girshick R. Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2015. 1440−1448 [25] Ren S Q, He K M, Girshick R, Sun J. Faster R-CNN: Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137−1149 doi: 10.1109/TPAMI.2016.2577031 [26] Tang X, Du D K, He Z Q, Liu J T. PyramidBox: A context-assisted single shot face detector. In: Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018. 812−828 [27] Cai Z W, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 6154−6162 [28] Redmon J, Divvala S, Girshick R, Farhadi A. You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016. 779−788 [29] Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, et al. SSD: Single shot multibox detector. In: Proceedings of the 14th European Conference on Computer Vision (ECCV). Amsterdam, The Netherlands: Springer, 2016. 21−37 [30] Lin T Y, Goyal P, Girshick R, He K M, Dollár P. Focalloss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. 2999−3007 [31] Yi J F, Mao J X, Zhang H, Zeng K, Tao Z M, Zhong H, et al. PSTL-Net: A patchwise self-texture-learning network for transmission line inspection. IEEE Transactions on Instrumentation and Measurement, 2024, 73: Article No. 5005714 [32] Liu X Y, Miao X R, Jiang H, Chen J, Wu M, Chen Z H. Component detection for power line inspection using a graph-based relation guiding network. IEEE Transactions on Industrial Informatics, 2023, 19(9): 9280−9290 doi: 10.1109/TII.2022.3227638 [33] Barrow H G, Tenenbaum J M. Recovering intrinsic scene characteristics from images. Computer Vision Systems. London: Academic Press, 1978. [34] Canny J. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, PAMI-8(6): 679−698 doi: 10.1109/TPAMI.1986.4767851 [35] Ke L, Tai Y W, Tang C K. Occlusion-aware instance segmentation via bilayer network architectures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(8): 10197−10211 doi: 10.1109/TPAMI.2023.3246174 [36] Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation. In: Proceedings of the 18th International Conference on Medical Image Computing and Computer-assisted Intervention (MICCAI). Munich, Germany: Springer, 2015. 234−241 [37] Badrinarayanan V, Kendall A, Cipolla R. SegNet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481−2495 doi: 10.1109/TPAMI.2016.2644615 [38] Zhao Z B, Liu B, Zhai Y J, Zhao W Q, Su P. Dual graph reasoning network for oil leakage segmentation in substation equipment. IEEE Transactions on Instrumentation and Measurement, 2024, 73: Article No. 3502415 [39] Zhao Z B, Feng S, Zhai Y J, Zhao W Q, Li G. Infrared thermal image instance segmentation method for power substation equipment based on visual feature reasoning. IEEE Transactions on Instrumentation and Measurement, 2023, 72: Article No. 5029613 [40] Chen L C, Papandreou G, Kokkinos I, Murphy K, Yuille A L. Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv: 1412.7062, 2015.Chen L C, Papandreou G, Kokkinos I, Murphy K, Yuille A L. Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv: 1412.7062, 2015. [41] Chen L C, Zhu Y K, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018. 833−851 [42] MacQueen J. Some methods for classification and analysis of multivariate observations. In: Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability. Berkeley, USA: University of California Press, 1967. 281−297 [43] Shi J B, Malik J. Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8): 888−905 doi: 10.1109/34.868688 [44] Goodfellow I J, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, et al. Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems (NeurIPS). Montreal, Canada: ACM, 2014. 2672−2680 [45] Jiang B, Zhang Z Y, Lin D D, Tang J , Luo B. Semi-supervised learning with graph learning-convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA: IEEE, 2019. 11305−11312Jiang B, Zhang Z Y, Lin D D, Tang J , Luo B. Semi-supervised learning with graph learning-convolutional networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA: IEEE, 2019. 11305−11312 [46] Mehta S, Rastegari M, Caspi A, Shapiro L, Hajishirzi H. ESPNet: Efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany: Springer, 2018. 561−580 [47] Maturana D, Scherer S. VoxNet: A 3D convolutional neural network for real-time object recognition. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany: IEEE, 2015. 922−928 [48] Charles R Q, Hao S, Mo K C, Guibas L J. PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 77−85Charles R Q, Hao S, Mo K C, Guibas L J. PointNet: Deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 77−85 [49] Charles R Q, Yi L, Su H, Guibas L J. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: ACM, 2017. 5105−5114Charles R Q, Yi L, Su H, Guibas L J. PointNet++: Deep hierarchical feature learning on point sets in a metric space. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: ACM, 2017. 5105−5114 [50] Zhao N, Chua T S, Lee G H. SESS: Self-ensembling semi-supervised 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 11076−11084 [51] Shi S, Wang Z, Wang X, Li H. Part-A2 net: 3D part-aware and aggregation neural network for object detection from point cloud. Pattern Recognition, 2022, 122: Article No. 108242 [52] Zhou Y, Tuzel O. VoxelNet: End-to-end learning for point cloud based 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City, USA: IEEE, 2018. 4490−4499 [53] Ouyang Z C, Dong X Y, Cui J H, Niu J W, Guizani M. PV-EncoNet: Fast object detection based on colored point cloud. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(8): 12439−12450 doi: 10.1109/TITS.2021.3114062 [54] Shi S S, Guo C X, Jiang L, Wang Z, Shi J P, Wang X G. PV-RCNN: Point-voxel feature set abstraction for 3D object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 10526−10535 [55] Dong Z, Li J, Li Y. 3DV-Net: Learning multi-level multi-scale 3D features from point cloud for object detection. Sensors, 2021, 21(11): Article No. 3875 doi: 10.3390/s21113875 [56] Shi S S, Jiang L, Deng J J, Wang Z, Guo C X, Shi J P, et al. PV-RCNN++: Point-voxel feature set abstraction with local vector representation for 3D object detection. arXiv: 2102.00463, 2022.Shi S S, Jiang L, Deng J J, Wang Z, Guo C X, Shi J P, et al. PV-RCNN++: Point-voxel feature set abstraction with local vector representation for 3D object detection. arXiv: 2102.00463, 2022. [57] Wang Y, Sun Y B, Liu Z W, Sarma S E, Bronstein M M, Solomon J M. Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics, 2019, 38(5): Article No. 146 [58] Hu Q Y, Yang B, Xie L H, Rosa S, Guo Y L, Wang Z H. RandLA-Net: Efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 11105−11114 [59] Shi W J, Rajkumar R. Point-GNN: Graph neural network for 3D object detection in a point cloud. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 1708−1716 [60] Yan X, Zheng C D, Li Z, Wang S, Cui S G. PointASNL: Robust point clouds processing using nonlocal neural networks with adaptive sampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA: IEEE, 2020. 5588−5597 [61] Li W, Luo Z P, Xiao Z L, Chen Y P, Wang C, Li J. A GCN-based method for extracting power lines and pylons from airborne LiDAR data. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: Article No. 5700614 [62] Jeong S, Kim D, Kim S, Ham J W, Lee J K, Oh K Y. Real-time environmental cognition and sag estimation of transmission lines using UAV equipped with 3-D Lidar system. IEEE Transactions on Power Delivery, 2021, 36(5): 2658−2667 doi: 10.1109/TPWRD.2020.3024965 [63] Munir N, Awrangjeb M, Stantic B. An iterative graph-based method for constructing gaps in high-voltage bundle conductors using airborne LiDAR point cloud data. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: Article No. 5700316 [64] Huang J J, Shen Y Q, Wang J G, Ferreira V. Automatic pylon extraction using color-aided classification from UAV LiDAR point cloud data. IEEE Transactions on Instrumentation and Measurement, 2023, 72: Article No. 2520611 [65] Ibrahim M, Akhtar N, Anwar S, Mian A. SAT3D: Slot attention Transformer for 3D point cloud semantic segmentation. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(5): 5456−5466 doi: 10.1109/TITS.2023.3243643 [66] 张贵峰, 张志强, 沈锋. 变电站巡检机器人现状与发展综述. 云南电力技术, 2022, 50(6): 2−8Zhang Gui-Feng, Zhang Zhi-Qiang, Shen Feng. Review of the current situation and development of substation inspection robots. Yunnan Electric Power, 2022, 50(6): 2−8 [67] 袁利, 姜甜甜, 魏春岭, 杨孟飞. 空间控制技术发展与展望. 自动化学报, 2023, 49(3): 476−493Yuan Li, Jiang Tian-Tian, Wei Chun-Ling, Yang Meng-Fei. Advances and perspectives of space control technology. Acta Automatica Sinica, 2023, 49(3): 476−493 [68] 黄观文, 王媛媛, 龙正鑫, 秦志伟, 张勤. GNSS卫星轨道机动探测技术进展. 导航定位学报, 2024, 12(2): 1−12Huang Guan-Wen, Wang Yuan-Yuan, Long Zheng-Xin, Qin Zhi-Wei, Zhang Qin. Technique advancements in GNSS satellite orbit maneuver detection. Journal of Navigation and Positioning, 2024, 12(2): 1−12 [69] Zhang X C, Demiris Y. Visible and infrared image fusion using deep learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(8): 10535−10554 doi: 10.1109/TPAMI.2023.3261282 [70] Li T, Yu H Y. Visual–inertial fusion-based human pose estimation: A review. IEEE Transactions on Instrumentation and Measurement, 2023, 72: Article No. 4007816 [71] Sun Z H, Ke Q H, Rahmani H, Bennamoun M, Wang G, Liu J. Human action recognition from various data modalities: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(3): 3200−3225 [72] Yuan D, Zhang H P, Shu X, Liu Q, Chang X J, He Z Y, et al. Thermal infrared target tracking: A comprehensive review. IEEE Transactions on Instrumentation and Measurement, 2024, 73: Article No. 5000419 [73] Tian C, Zhou Z K, Huang Y Q, Li G J, He Z Y. Cross-modality proposal-guided feature mining for unregistered RGB-thermal pedestrian detection. IEEE Transactions on Multimedia, 2024, 26: 6449−6461 doi: 10.1109/TMM.2024.3350926 [74] Li X Y, Chen S G, Tian C N, Zhou H, Zhang Z X. M2FNet: Mask-guided multi-level fusion for RGB-T pedestrian detection. IEEE Transactions on Multimedia, 2024, 26: 8678−8690 doi: 10.1109/TMM.2024.3381377 [75] Xu Y W, Fan K G, Hu Q, Zhang X T. Positioning of suspended permanent magnet maglev trains using satellite-ground multisensor fusion. IEEE Sensors Journal, 2024, 24(10): 16816−16825 doi: 10.1109/JSEN.2024.3384699 [76] Cossio-Montefinale L, Ruiz-Del-Solar J, Verschae R. Cherry CO dataset: A dataset for cherry detection, segmentation and maturity recognition. IEEE Robotics and Automation Letters, 2024, 9(6): 5552−5558 doi: 10.1109/LRA.2024.3393214 [77] Rayhana R, Ma Z Y, Liu Z, Xiao G Z, Ruan Y F, Sangha J S. A review on plant disease detection using hyperspectral imaging. IEEE Transactions on AgriFood Electronics, 2023, 1(2): 108−134 doi: 10.1109/TAFE.2023.3329849 [78] Song M, Um G M, Lee H K, Seo J, Kim W. Dynamic residual filtering with Laplacian pyramid for instance segmentation. IEEE Transactions on Multimedia, 2023, 25: 6892−6903 doi: 10.1109/TMM.2022.3215306 [79] Wu S J, Wang Y B. Seismic image dip estimation by multiscale principal component analysis. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: Article No. 5900410 [80] Grossmann A, Morlet J. Decomposition of hardy functions into square integrable wavelets of constant shape. SIAM Journal on Mathematical Analysis, 1984, 15(4): 723−736 doi: 10.1137/0515056 [81] Li X X, Cheng X Z, Xu Y W, Jiao Y, Huang W C, Cui Y Y, et al. Multi-frequency ultrasound imaging fusion method based on wavelet transform for guided screw insertion. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2024, 71(3): 395−407 doi: 10.1109/TUFFC.2023.3348100 [82] Yang B, Li S T. Multifocus image fusion and restoration with sparse representation. IEEE Transactions on Instrumentation and Measurement, 2010, 59(4): 884−892 doi: 10.1109/TIM.2009.2026612 [83] Liu Y, Liu S P, Wang Z F. A general framework for image fusion based on multi-scale transform and sparse representation. Information Fusion, 2015, 24: 147−164 doi: 10.1016/j.inffus.2014.09.004 [84] Yin H T. Sparse representation with learned multiscale dictionary for image fusion. Neurocomputing, 2015, 148: 600−610 doi: 10.1016/j.neucom.2014.07.003 [85] Yang B, Li S T. Pixel-level image fusion with simultaneous orthogonal matching pursuit. Information Fusion, 2012, 13(1): 10−19 doi: 10.1016/j.inffus.2010.04.001 [86] Engan K, Aase S O, Husoy J H. Method of optimal directions for frame design. In: Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. Phoenix, USA: IEEE, 1999. 2443−2446 [87] Aharon M, Elad M, Bruckstein A. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 2006, 54(11): 4311−4322 doi: 10.1109/TSP.2006.881199 [88] Zong Z Y, Fu T, Yin X Y. High-dimensional generalized orthogonal matching pursuit with singular value decomposition. IEEE Geoscience and Remote Sensing Letters, 2023, 20: Article No. 7502205 [89] Liu S J, Ma J J, Cui C K. FPGA implementation of threshold projection orthogonal matching pursuit algorithm for compressed sensing reconstruction. IEEE Transactions on Circuits and Systems I: Regular Papers, 2024, 71(3): 1184−1197 doi: 10.1109/TCSI.2023.3345537 [90] Bai L, Yao S L, Gao K, Huang Y J, Tang R J, Yan H, et al. Joint sparse representations and coupled dictionary learning in multisource heterogeneous image pseudo-color fusion. IEEE Sensors Journal, 2023, 23(24): 30620−30632 doi: 10.1109/JSEN.2023.3325364 [91] Li H, Wu X J, Kittler J. Infrared and visible image fusion using a deep learning framework. In: Proceedings of the 24th International Conference on Pattern Recognition (ICPR). Beijing, China: IEEE, 2018. 2705−2710 [92] Ren X Y, Meng F Y, Hu T, Liu Z J, Wang C. Infrared-visible image fusion based on convolutional neural networks (CNN). In: Proceedings of the 8th International Conference on Intelligent Science and Big Data Engineering. Lanzhou, China: Springer, 2018. 301−307 [93] Ma J Y, Yu W, Liang P W, Li C, Jiang J J. FusionGAN: A generative adversarial network for infrared and visible image fusion. Information Fusion, 2019, 48: 11−26 doi: 10.1016/j.inffus.2018.09.004 [94] Sun F M, Zhang K, Yuan X, Zhao C X. Feature enhancement and fusion for RGB-T salient object detection. In: Proceedings of the IEEE International Conference on Image Processing (ICIP). Kuala Lumpur, Malaysia: IEEE, 2023. 1300−1304 [95] Zhu X, Liu J, Xiong X Z, Luo Z Q. Maximize peak-to-sidelobe ratio for real-time RGB-T tracking. IEEE Transactions on Instrumentation and Measurement, 2024, 73: Article No. 4502104 [96] Liu J, Luo Z Q, Xiong X Z. Online learning samples and adaptive recovery for robust RGB-T tracking. IEEE Transactions on Circuits and Systems for Video Technology, 2024, 34(2): 724−737 doi: 10.1109/TCSVT.2023.3288853 [97] Lv Y, Liu Z, Li G Y. Context-aware interaction network for RGB-T semantic segmentation. IEEE Transactions on Multimedia, 2024, 26: 6348−6360 doi: 10.1109/TMM.2023.3349072 [98] Liang W L, Shan C F, Yang Y J, Han J G. Multi-branch differential bidirectional fusion network for RGB-T semantic segmentation. IEEE Transactions on Intelligent Vehicles, DOI: 10.1109/TIV.2024.3374793 [99] Chen G, Shao F, Chai X L, Chen H W, Jiang Q P, Meng X C, et al. Modality-induced transfer-fusion network for RGB-D and RGB-T salient object detection. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(4): 1787−1801 doi: 10.1109/TCSVT.2022.3215979 [100] 陈建, 廖燕俊, 王适, 郑明魁, 苏立超. 自适应分割的视频点云多模式帧间编码方法. 自动化学报, 2023, 49(8): 1707−1722Chen Jian, Liao Yan-Jun, Wang Shi, Zheng Ming-Kui, Su Li-Chao. An adaptive segmentation based multi-mode inter-frame coding method for video point cloud. Acta Automatica Sinica, 2023, 49(8): 1707−1722 [101] Chen L, Luo X. Tensor distribution regression based on the 3D conventional neural networks. IEEE/CAA Journal of Automatica Sinica, 2023, 10(7): 1628−1630 doi: 10.1109/JAS.2023.123591 [102] Ye S Q, Chen D D, Han S F, Liao J. Robust point cloud segmentation with noisy annotations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(6): 7696−7710 doi: 10.1109/TPAMI.2022.3225323 [103] 张凯, 杨朋澄, 彭开香, 陈志文. 基于深度置信网络的多模态过程故障评估方法及应用. 自动化学报, 2024, 50(1): 89−102Zhang Kai, Yang Peng-Cheng, Peng Kai-Xiang, Chen Zhi-Wen. A deep belief network-based fault evaluation method for multimode processes and its applications. Acta Automatica Sinica, 2024, 50(1): 89−102 [104] 刘云鹏, 来庭煜, 刘嘉硕, 魏晓光, 裴少通. 特高压直流换流阀饱和电抗器振动声纹特性与松动程度声纹检测方法. 电工技术学报, 2023, 38(5): 1375−1389Liu Yun-Peng, Lai Ting-Yu, Liu Jia-Shuo, Wei Xiao-Guang, Pei Shao-Tong. Vibration voiceprint characteristics and looseness detection method of UHVDC converter valve saturable reactor. Transactions of China Electrotechnical Society, 2023, 38(5): 1375−1389 [105] Khotimah K, Santoso A B, Ma'arif M, Azhiimah A N, Suprianto B, Sumbawati M S, et al. Validation of voice recognition in various Google voice languages using voice recognition module V3 based on microcontroller. In: Proceedings of the 3rd International Conference on Vocational Education and Electrical Engineering (ICVEE). Surabaya, Indonesia: IEEE, 2020. 1−6 [106] Brunelli R, Falavigna D. Person identification using multiple cues. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1995, 17(10): 955−966 doi: 10.1109/34.464560 [107] Noda K, Yamaguchi Y, Nakadai K, Okuno H G, Ogata T. Audio-visual speech recognition using deep learning. Applied Intelligence, 2015, 42(4): 722−737 doi: 10.1007/s10489-014-0629-7 [108] 张玉珍, 魏带娣, 王建宇, 戴跃伟. 基于多模态融合的足球视频语义分析. 计算机科学, 2010, 37(7): 273−276Zhang Yu-Zhen, Wei Dai-Di, Wang Jian-Yu, Dai Yue-Wei. Semantic analysis for soccer video based on fusion of multimodal features. Computer Science, 2010, 37(7): 273−276 [109] Xu Z J, Wang R F, Wang J, Yu D H. Parkinson's disease detection based on spectrogram-deep convolutional generative adversarial network sample augmentation. IEEE Access, 2020, 8: 206888−206900 doi: 10.1109/ACCESS.2020.3037775 [110] 施胜丹, 黄金军, 朱霄珣, 王瑜, 钱白云. 基于声纹SDP-CNN的变压器局部放电模式识别. 电力信息与通信技术, 2022, 20(10): 105−112Shi Sheng-Dan, Huang Jin-Jun, Zhu Xiao-Xun, Wang Yu, Qian Bai-Yun. Partial discharge pattern recognition on transformer based on voiceprint SDP-CNN. Electric Power Information and Communication Technology, 2022, 20(10): 105−112 [111] 陆云才, 廖才波, 李群, 王同磊, 邵剑, 张一. 基于声纹特征和集成学习的变压器缺陷诊断方法. 电力工程技术, 2023, 42(5): 46−55Lu Yun-Cai, Liao Cai-Bo, Li Qun, Wang Tong-Lei, Shao Jian, Zhang Yi. Transformer fault diagnosis method based on voiceprint feature and ensemble learning. Electric Power Engineering Technology, 2023, 42(5): 46−55 [112] 王欢, 王昕, 张峰, 齐笑, 柴方森, 李文鹏. 基于改进生成对抗网络的变压器声纹故障诊断. 智慧电力, 2024, 52(4): 24−31Wang Huan, Wang Xin, Zhang Feng, Qi Xiao, Chai Fang-Sen, Li Wen-Peng. Transformer voiceprint fault diagnosis based on improved generative adversarial network. Smart Power, 2024, 52(4): 24−31 [113] 林颖, 张峰达, 李壮壮, 郑文杰, 戈宁. 基于大模型的红外图像电力设备交互式分割. 网络新媒体技术, 2024, 13(2): 53−60, 67Lin Ying, Zhang Feng-Da, Li Zhuang-Zhuang, Zheng Wen-Jie, Ge Ning. Large model based interactive segmentation of infrared image for power equipment. Network New Media Technology, 2024, 13(2): 53−60, 67 [114] 杨权, 樊绍胜. 基于图像预处理和语义分割的电力巡检机器人视觉导航方法. 电力科学与技术学报, 2023, 38(6): 248−258Yang Quan, Fan Shao-Sheng. Visual navigation method for electric power inspection robot based on image preprocessing and semantic segmentation. Journal of Electric Power Science and Technology, 2023, 38(6): 248−258 [115] 马青山, 朱建宝, 俞鑫春, 张斌. 基于改进DSD-LinkNet的电力安全带分割算法. 电气自动化, 2023, 45(3): 106−108Ma Qing-Shan, Zhu Jian-Bao, Yu Xin-Chun, Zhang Bin. Power safety belt segmentation algorithm based on improved DSD-LinkNet. Electrical Automation, 2023, 45(3): 106−108 [116] 许刚, 李果. 轻量化航拍图像电力线语义分割. 中国图象图形学报, 2021, 26(11): 2605−2618 doi: 10.11834/jig.200690Xu Gang, Li Guo. Research on lightweight neural network of aerial powerline image segmentation. Journal of Image and Graphics, 2021, 26(11): 2605−2618 doi: 10.11834/jig.200690 [117] 左安全, 秦伦明, 王悉, 边后琴, 陈思林. 基于改进DeepLabv3+模型的电力线语义分割方法. 无线电工程, 2023, 53(1): 96−104Zuo An-Quan, Qin Lun-Ming, Wang Xi, Bian Hou-Qin, Chen Si-Lin. Powerline semantic segmentation method based on improved DeepLabv3+ model. Radio Engineering, 2023, 53(1): 96−104 [118] Cui C, Gao T Q, Wei S Y, Du Y N, Guo R Y, Dong S L, et al. PP-LCNet: A lightweight CPU convolutional neural network. arXiv: 2109.15099, 2021.Cui C, Gao T Q, Wei S Y, Du Y N, Guo R Y, Dong S L, et al. PP-LCNet: A lightweight CPU convolutional neural network. arXiv: 2109.15099, 2021. [119] Chollet F. Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, USA: IEEE, 2017. 1800−1807 [120] Park J, Woo S, Lee J Y, Kweon I S. BAM: Bottleneck attention module.arXiv: 1807.06514, 2018. [121] Mao M F, Chen Y, Chen W X, Du W, Zhang M, Mao T Q. Power transmission line image segmentation method based on binocular vision and feature pyramid network. In: Proceedings of the International Conference on Sensing, Measurement & Data Analytics in the Era of Artificial Intelligence (ICSMD). Nanjing, China: IEEE, 2021. 1−4 [122] Wang L J, Chen Z L, Hua D, Zheng Z X. Semantic segmentation of transmission lines and their accessories based on UAV-taken images. IEEE Access, 2019, 7: 80829−80839 doi: 10.1109/ACCESS.2019.2923024 [123] Yang L, Kong S Y, Cui S L, Huang H Y, Liu Y H. An efficient end-to-end CNN network for high-voltage transmission line segmentation. In: Proceedings of the 8th International Conference on Cloud Computing and Intelligent Systems (CCIS). Chengdu, China: IEEE, 2022. 565−570 [124] 唐小煜, 黄进波, 冯洁文, 陈锡和. 基于U-net和YOLOv4的绝缘子图像分割与缺陷检测. 华南师范大学学报(自然科学版), 2020, 52(6): 15−21Tang Xiao-Yu, Huang Jin-Bo, Feng Jie-Wen, Chen Xi-He. Image segmentation and defect detection of insulators based on U-net and YOLOv4. Journal of South China Normal University (Natural Science Edition), 2020, 52(6): 15−21 [125] Yu Y J, Cao H, Wang Z Z, Li Y Q, Li K, Xie S Q. Texture-and-shape based active contour model for insulator segmentation. IEEE Access, 2019, 7: 78706−78714 doi: 10.1109/ACCESS.2019.2922257 [126] Tang X, Mao J X, Yi J F, Tao Z M, He Z Y. An insulator feature-enhanced segmentation method based on the residual-type attention mechanism. In: Proceedings of the 5th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). Chongqing, China: IEEE, 2022. 1507−1511 [127] Han Y F, Han J, Ni Z J, Wang W S, Jiang H Y. Instance segmentation of transmission line images based on an improved D-SOLO network. In: Proceedings of the 3rd International Conference on Power Data Science (ICPDS). Harbin, China: IEEE, 2021. 40−46 [128] Choi H, Yun J P, Kim B J, Jang H, Kim S W. Attention-based multimodal image feature fusion module for transmission line detection. IEEE Transactions on Industrial Informatics, 2022, 18(11): 7686−7695 doi: 10.1109/TII.2022.3147833 [129] Xu C, Li Q W, Jiang X B, Yu D B, Zhou Y Q. Dual-space graph-based interaction network for RGB-thermal semantic segmentation in electric power scene. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(4): 1577−1592 doi: 10.1109/TCSVT.2022.3216313 [130] Ma J L, Qian K, Zhang X B, Ma X D. Weakly supervised instance segmentation of electrical equipment based on RGB-T automatic annotation. IEEE Transactions on Instrumentation and Measurement, 2020, 69(12): 9720−9731 doi: 10.1109/TIM.2020.3001796 [131] 孙阳, 李佳. 基于通道剪枝的YOLOv7-tiny输电线路异物检测算法. 计算机工程与应用, 2024, 60(14): 319−328Sun Yang, Li Jia. YOLOv7-tiny transmission line foreign object detection algorithm based on channel pruning. Computer Engineering and Applications, 2024, 60(14): 319−328 [132] 沈晓峰, 谢伟, 孙路, 李轶, 贺润平. 基于相邻帧差法的输电线路异物目标检测研究. 自动化仪表, 2023, 44(10): 20−24Shen Xiao-Feng, Xie Wei, Sun Lu, Li Yi, He Run-Ping. Research on foreign object target detection on transmission lines based on adjacent frame difference method. Process Automation Instrumentation, 2023, 44(10): 20−24 [133] Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2): 91−110 doi: 10.1023/B:VISI.0000029664.99615.94 [134] 余沿臻, 邱志斌, 周银彪, 朱轩, 王青. 基于卷积神经网络与ECOC-SVM的输电线路异物检测. 智慧电力, 2022, 50(3): 87−92, 107Yu Yan-Zhen, Qiu Zhi-Bin, Zhou Yin-Biao, Zhu Xuan, Wang Qing. Foreign body detection for transmission lines based on convolutional neural network and ECOC-SVM. Smart Power, 2022, 50(3): 87−92, 107 [135] Jiang H, Huang W L, Chen J, Liu X Y, Miao X R, Zhuang S B. Detection of bird nests on power line patrol using single shot detector. In: Proceedings of the Chinese Automation Congress (CAC). Hangzhou, China: IEEE, 2019. 3409−3414 [136] 杨征. 基于ZigBee和GSM技术输电线路杆塔倾斜监测系统的研究 [硕士学位论文], 华北电力大学, 中国, 2012.Yang Zheng. Tilt Monitoring System of Transmission Line Towers Based on the ZigBee and GSM Technology [Master thesis], North China Electric Power University, China, 2012. [137] Li Q Q, Chen Z P, Hu Q W. A model-driven approach for 3D modeling of pylon from airborne LiDAR data. Remote Sensing, 2015, 7(9): 11501−11524 doi: 10.3390/rs70911501 [138] 沈小军, 秦川, 杜勇, 于忻乐. 复杂地形电力线机载激光雷达点云自动提取方法. 同济大学学报(自然科学版), 2018, 46(7): 982−987Shen Xiao-Jun, Qin Chuan, Du Yong, Yu Xin-Le. An automatic power line extraction method from airborne light detection and ranging point cloud in complex terrain. Journal of Tongji University (Natural Science), 2018, 46(7): 982−987 [139] 柳长安, 孙书明, 赵丽娟. 基于激光点云实现杆塔提取的轻量级网络. 激光技术, 2021, 45(3): 367−372Liu Chang-An, Sun Shu-Ming, Zhao Li-Juan. A lightweight network for power tower extraction from laser point cloud. Laser Technology, 2021, 45(3): 367−372 [140] 芦竹茂, 龚浩, 金秋衡, 胡庆武, 李加元. 无人机激光雷达点云电力杆塔倾斜状态测量. 应用科学学报, 2022, 40(3): 389−399Lu Zhu-Mao, Gong Hao, Jin Qiu-Heng, Hu Qing-Wu, Li Jia-Yuan. Tilt rate measurement of power tower based on UAV LiDAR point cloud. Journal of Applied Sciences, 2022, 40(3): 389−399 [141] 徐梁刚, 时磊, 陈凤翔, 王时春, 龙新, 王迪. 基于激光点云的输电线路杆塔倾斜检测算法. 激光技术, 2022, 46(3): 390−396Xu Liang-Gang, Shi Lei, Chen Feng-Xiang, Wang Shi-Chun, Long Xin, Wang Di. Transmission line tower tilt detection algorithm based on laser point cloud. Laser Technology, 2022, 46(3): 390−396 [142] 应斌, 唐斌, 潘俊杰, 郭震. 基于YOLOv3的电力杆塔检测算法研究. 浙江电力, 2021, 40(5): 53−59Ying Bin, Tang Bin, Pan Jun-Jie, Guo Zhen. Research on a detection algorithm of power tower based on YOLOv3. Zhejiang Electric Power, 2021, 40(5): 53−59 [143] 郝美. 输电线路覆冰在线监测的关键技术. 电子技术, 2020, 49(3): 90−91Hao Mei. Key of icing online monitoring for transmission line. Electronic Technology, 2020, 49(3): 90−91 [144] Huang T T, Liu Z, Chen X W, Bai X. EPNet: Enhancing point features with image semantics for 3D object detection. In: Proceedings of the 16th European Conference on Computer Vision (ECCV). Glasgow, UK: Springer, 2020. 35−52 [145] 安秋娥. 设备台帐与备件管理. 化工设备与防腐蚀, 1999(4): 27−28, 62An Qiu-E. Equipment ledger and spare parts management. Chemical Equipment and Anticorrosion, 1999(4): 27−28, 62
计量
- 文章访问数: 220
- HTML全文浏览量: 150
- 被引次数: 0