2.624

2020影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于序列注意力和局部相位引导的骨超声图像分割网络

陈芳 张道强 廖洪恩 赵喆

陈芳, 张道强, 廖洪恩, 赵喆. 基于序列注意力和局部相位引导的骨超声图像分割网络. 自动化学报, 2022, 48(x): 1−9 doi: 10.16383/j.aas.c210298
引用本文: 陈芳, 张道强, 廖洪恩, 赵喆. 基于序列注意力和局部相位引导的骨超声图像分割网络. 自动化学报, 2022, 48(x): 1−9 doi: 10.16383/j.aas.c210298
Chen Fang, Zhang Dao-Qiang, Liao Hon-Gen, Zhao Zhe. Bone ultrasound segmentation network based on sequential attention and local phase guidance. Acta Automatica Sinica, 2022, 48(x): 1−9 doi: 10.16383/j.aas.c210298
Citation: Chen Fang, Zhang Dao-Qiang, Liao Hon-Gen, Zhao Zhe. Bone ultrasound segmentation network based on sequential attention and local phase guidance. Acta Automatica Sinica, 2022, 48(x): 1−9 doi: 10.16383/j.aas.c210298

基于序列注意力和局部相位引导的骨超声图像分割网络

doi: 10.16383/j.aas.c210298
基金项目: 国家自然科学基金(U20A20389, 61901214)中国博士后科学基金(2021T140322, 2020M671484)资助
详细信息
    作者简介:

    陈芳:南京航空航天大学计算机学院副教授, 主要研究方向为医学图像处理, 计算机辅助手术导航. E-mail: chenfang@nuaa.edu.cn

    张道强:南京航空航天大学计算机学院教授, 主要研究方向为医学图像处理, 机器学习. E-mail: dqzhang@nuaa.edu.cn

    廖洪恩:清华大学医学院教授, 主要研究方向为三维成像, 微创诊疗. E-mail: liao@tsinghua.edu.cn

    赵喆:北京清华长庚医院骨科副主任医师, 清华大学临床医学院副教授. 主要研究方向为创伤骨科、计算机导航手术、骨科植入物设计. E-mail: zhaozhao_02@163.com

Bone Ultrasound Segmentation Network based on Sequential Attention and Local Phase Guidance

Funds: Supported by National Natural Science Foundation of P. R. China (U20A20389, 61901214) China Postdoctoral Science Foundation (2021T140322, 2020M671484)
More Information
    Author Bio:

    CHEN Fang Professor in the College of Computer Science and Technology of Nanjing University of Aeronautics and Astronautics. Her research interests include medical image processing and image-guided surgery

    ZHANG Dao-Qiang Professor in the College of Computer Science and Technology of Nanjing University of Aeronautics and Astronautics. His research interests include medical image processing and machine learning

    LIAO Hon-Gen Professor in the School of Medicine, Tsinghua University, Beijing. His research interests include 3D image, minimally invasive diagnosis and therapy

    ZHAO Zhe Attending surgeon of the Department of Orthopeadics, Beijing Tsinghua Changgung Hospital. Associate professor of School of Clinical Medicine, Tsinghua University. His research interest covers orthopaedic trauma, computer assisted surgery and orthopeadic implant development

  • 摘要: 在超声辅助的骨科手术导航中, 需要从采集的超声图像序列中精确分割出骨结构, 并展示给医生, 来辅助医生进行术中决策. 但是, 图像噪声、成像伪影以及模糊的骨边界导致从超声图像序列中精确分割提取骨结构十分困难. 为解决该问题, 本文提出了一种新的基于序列注意力与局部相位引导的骨超声图像分割网络. 该网络一方面自适应地利用了超声序列帧之间的关系即序列注意力来辅助骨结构的语义分割. 另一方面, 该网络通过引入局部相位引导模块, 突出骨边缘信息, 进一步提高分割精度. 利用包含19050张图像的骨超声数据集, 进行了交叉实验、消融实验并与最新的超声骨分割方法进行了比较. 实验结果表明本文方法对骨结构分割精度高, 优于现有的超声骨分割方法.
  • 图  1  基于序列注意力与局部相位引导的骨超声图像分割网络系统框图; 图中ConvA表示卷积核为1*1, 步长为1的卷积操作, ConvB表示卷积核为3*3, 步长为1的卷积操作.

    Fig.  1  Bone ultrasound segmentation network based on sequential attention and local phase guidance; ConvA denotes the convolution operation with kernel size of 1×1 and a stride of 1; ConvB denotes the convolution operation with kernel size of 3×3 and a stride of 1

    图  2  骨超声图像序列采集示意图

    Fig.  2  Diagram of bone ultrasound sequence acquisition

    图  3  十折交叉实验中训练和测试超声图像帧数的分布

    Fig.  3  The distributions of training and testing frames in10-fold validation experiments.

    图  4  10次交叉实验的交叉比IOU值的箱线图

    Fig.  4  Boxplots for IOU values in10-fold validation experiments.

    图  5  利用所提出的分割网络对骨结构进行分割的实际结果; 第一行: 待分割的骨超声图像; 第二行: 专家手动标注的骨结构; 第三行: 利用所提出的方法自动分割的骨结构;

    Fig.  5  Bone segmentation results by using the proposed segmentation network; The first line: ultrasound bone images to be segmented; The second line: bone structures manually delineated by experts; Line 3: segmented bone structures using the proposed method

    图  6  消融实验结果, 本文所提出的分割网络与两个变体模型的骨分割实验结果比较

    Fig.  6  Ablation results. Comparison between our proposed network with two variants.

    图  7  局部相位模块消融实验结果示例. 第一张: 超声图像帧; 第二张: 手动标注的骨结构; 第三张: 带有局部相位模块的模型分割结果; 第四张: 去除局部相位模块的模型分割结果

    Fig.  7  Ablation results of local phase guidance. The first graph: ultrasound image frame; the second graph: manually delineated bone structures; the third graph: results of the model with local phase guidance; the fourth graph: results of the model without local phase guidance

    表  1  本研究采集的超声图像序列数据集的信息; 对于每个志愿者数据, 提供像素分辨率以及所标注的超声图像帧数信息

    Table  1  Summary of the collected dataset. For each volunteer, we report the pixel resolution and the number of the annotated ultrasound image.

    志愿
    者ID
    图像帧
    数量
    图像分辨率
    (mm/pixel)
    志愿者ID图像帧
    数量
    图像分辨率
    (mm/pixel)
    119000.19 ~ 0.21619800.17 ~ 0.21
    220100.17 ~ 0.21715600.18 ~ 0.23
    317400.19 ~ 0.21819800.19 ~ 0.21
    418700.21 ~ 0.23919000.21 ~ 0.23
    520100.19 ~ 0.231021000.17 ~ 0.23
    下载: 导出CSV

    表  2  所提出的分割网络对10名志愿者采集的超声序列图像的分割结果

    Table  2  Results of our proposed model obtained on the ultrasound images from ten volunteers

    Exp_K交叉比IOU平均欧式距离AED
    Exp_10.91 ± 0.080.41±0.06
    Exp_20.90 ± 0.080.39±0.07
    Exp_30.91 ± 0.070.39±0.06
    Exp_40.92 ± 0.070.37±0.05
    Exp_50.90 ± 0.070.41±0.08
    Exp_60.89 ± 0.080.43±0.08
    Exp_70.91 ± 0.060.42±0.07
    Exp_80.90 ± 0.070.41±0.09
    Exp_90.92 ± 0.060.40±0.07
    Exp_100.91 ± 0.080.39±0.09
    下载: 导出CSV

    表  3  针对主干网络的比较实验结果

    Table  3  Comparison experiments by using different backbones.

    主干网络ResNet18ResNet34ResNet50ResNet101VGGNet
    超声图像骨分割平均交叉比IOU值0.89 ± 0.100.90 ± 0.080.91 ± 0.070.89 ± 0.090.89 ± 0.09
    下载: 导出CSV

    表  4  本文所提出的超声图像分割网络与其他最新的分割方法的实验结果比较

    Table  4  Comparison results with state-of-arts

    实验结果/
    方法
    局部相位引导
    CNN [16]
    BoneNet模型[18]滤波层引导
    CNN [17]
    时空CNN[28]注意引导网络
    AGNet [29]
    三重注意力网络
    TriANet[30]
    本文方法
    Exp_10.880.880.870.850.850.860.91
    Exp_20.870.880.860.840.850.850.90
    Exp_30.870.890.870.860.860.880.91
    Exp_40.890.890.870.850.860.870.92
    Exp_50.870.890.850.860.870.870.90
    Exp_60.860.880.850.840.850.850.89
    Exp_70.860.900.860.840.860.870.91
    Exp_80.870.880.840.850.860.860.90
    Exp_90.890.910.890.860.870.880.92
    Exp_100.870.890.870.870.870.890.91
    平均值0.87 ± 0.130.88 ± 0.080.86 ± 0.140.87 ± 0.120.86 ± 0.090.88 ± 0.100.91 ± 0.07
    下载: 导出CSV
  • [1] Zhe Z, Jianjin Z, Fei S, et al. Intraoperative ultrasound-guided reduction of femoral shaft fractures using intramedullary nailing: a technical note. Archives of Orthopaedic and Trauma Surgery, 2019, 139(5): 589-596. doi: 10.1007/s00402-018-3085-8
    [2] Zhou H, Zhang G, Li M, et al. Ultrasonography-guided closed reduction in the treatment of displaced transphyseal fracture of the distal humerus. Journal of Orthopaedic Surgery and Research, 2020, 15(1): 1-7. doi: 10.1186/s13018-019-1510-5
    [3] Wein W, Karamalis A, Baumgartner A, et al. Automatic bone detection and soft tissue aware ultrasound–CT registration for computer-aided orthopedic surgery. International Journal of Computer Assisted Radiology and Surgery, 2015, 10(6): 971-979. doi: 10.1007/s11548-015-1208-z
    [4] Hacihaliloglu I. Ultrasound imaging and segmentation of bone surfaces: A review. Technology, 2017, 5(02): 74-80. doi: 10.1142/S2339547817300049
    [5] Pandey P U, Quader N, Guy P, et al. Ultrasound bone segmentation: A scoping review of techniques and validation practices. Ultrasound in Medicine & Biology, 2020, 46(4): 921-935.
    [6] Masson-Sibut A, Nakib A, Petit E, et al. Computer-assisted intramedullary nailing using real-time bone detection in 2D ultrasound images. In: Proceedings of International Workshop on Machine Learning in Medical Imaging. Berlin, Heidelberg: Springer, 2011. 18−25.
    [7] Wang P, Patel V M, Hacihaliloglu I. Simultaneous segmentation and classification of bone surfaces from ultrasound using a multi-feature guided CNN. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham, Switzerland: Springer, 2018. 134−142.
    [8] Huang Z, Wang L W, Leung F H F, et al. Bone feature segmentation in ultrasound spine image with robustness to speckle and regular occlusion noise. In: Proceedings of 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC). Toronto, ON, Canada: IEEE, 2020. 1566−1571.
    [9] Hacihaliloglu I, Abugharbieh R, Hodgson A, et al. Bone segmentation and fracture detection in ultrasound using 3D local phase features. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin, Heidelberg: Springer, 2008. 287−295.
    [10] 范家伟, 张如如, 陆萌, 等. 深度学习方法在糖尿病视网膜病变诊断中的应用. 自动化学报, 2021, 47(5): 20.

    Fan Jia-Wei, Zhang Ru-Ru, Lu Meng, et al. Applications of deep learning techniques for diabetic retinal diagnosis. Acta Automatica Sinica, 2021, 47(5): 20.
    [11] 蒋芸, 谭宁. 基于条件深度卷积生成对抗网络的视网膜血管分割. 自动化学报, 2021, 47(1): 136-147.

    Jiang Yun, Tan Ning. Retinal vessel segmentation based on conditional deep convolutional generative adversarial networks. Acta Automatica Sinica, 2021, 47(1): 136-147.
    [12] 夏平, 施宇, 雷帮军, 等. 复小波域混合概率图模型的超声医学图像分割. 自动化学报, 2021, 47(1): 185-196.

    Xia Ping, Shi Yu, Lei Bang-Jun, et al. Ultrasound medical image segmentation based on hybrid probabilistic graphical model in complex-wavelet domain. Acta Automatica Sinica, 2021, 47(1): 185-196.
    [13] Ouahabi A, Taleb-Ahmed A. Deep learning for real-time semantic segmentation: Application in ultrasound imaging. Pattern Recognition Letters, 2021, 144: 27-34. doi: 10.1016/j.patrec.2021.01.010
    [14] Baka N, Leenstra S, van Walsum T. Ultrasound aided vertebral level localization for lumbar surgery. IEEE Transactions on Medical Imaging, 2017, 36(10): 2138-2147. doi: 10.1109/TMI.2017.2738612
    [15] Ciganovic M, Özdemir F, Farshad M, et al. Deep learning techniques for bone surface delineation in ultrasound. In: Proceedings of Medical Imaging 2019: Ultrasonic Imaging and Tomography. International Society for Optics and Photonics, 2019. 10955: 109550Y.
    [16] Wang P, Vives M, Patel V M, et al. Robust real-time bone surfaces segmentation from ultrasound using a local phase tensor-guided CNN. International Journal of Computer Assisted Radiology and Surgery, 2020, 15: 1127-1135. doi: 10.1007/s11548-020-02184-1
    [17] Alsinan A Z, Patel V M, Hacihaliloglu I. Automatic segmentation of bone surfaces from ultrasound using a filter-layer-guided CNN. International Journal of Computer Assisted Radiology and Surgery, 2019, 14(5): 775-783. doi: 10.1007/s11548-019-01934-0
    [18] Luan K, Li Z, Li J. An efficient end-to-end CNN for segmentation of bone surfaces from ultrasound. Computerized Medical Imaging and Graphics, 2020, 84: 101766. doi: 10.1016/j.compmedimag.2020.101766
    [19] Hu P, Caba F, Wang O, et al. Temporally distributed networks for fast video semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, WA, USA: IEEE, 2020. 8818−8827.
    [20] Yao R, Xu X, Zhou Y, et al. Joint attention mechanism for unsupervised video object Segmentation. In Proceedings of Chinese Conference on Pattern Recognition and Computer Vision (PRCV). Cham, Switzerland: Springer, 2021. 154−165.
    [21] Alcazar J L, Bravo M A, Jeanneret G, et al. MAIN: Multi-Attention Instance Network for Video Segmentation[J]. Computer Vision and Image Understanding, 2021, 210: 103240. doi: 10.1016/j.cviu.2021.103240
    [22] He K, Zhang X, Ren S, et al. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. Las Vegas, NV, USA: IEEE, 2016. 770−778.
    [23] He K, Zhang X, Ren S, et al. Identity mappings in deep residual networks. In Proceedings of European conference on computer vision. Cham, Switzerland: Springer, 2016. 630−645.
    [24] Cox J, Rubin S, Adams J, et al. Hyperparameter selection for ResNet classification of malignancy from thyroid ultrasound images. In Proceedings of Medical Imaging 2020: Computer-Aided Diagnosis. International Society for Optics and Photonics, 2020. 11314: 1131447.
    [25] Zhang Q, Cui Z, Niu X, et al. Image segmentation with pyramid dilated convolution based on ResNet and U-Net. In Proceedings of International Conference on Neural Information Processing. Cham, Switzerland: Springer, 2017. 364−372.
    [26] Hacihaliloglu I, Rasoulian A, Rohling R N, et al. Local phase tensor features for 3-D ultrasound to statistical shape+ pose spine model registration. IEEE Transactions on Medical Imaging, 2014, 33(11): 2167-2179. doi: 10.1109/TMI.2014.2332571
    [27] Hacihaliloglu I. Localization of bone surfaces from ultrasound data using local phase information and signal transmission maps. In Proceedings of International Workshop on Computational Methods and Clinical Applications in Musculoskeletal Imaging. Cham, Switzerland: Springer, 2017. 1−11.
    [28] Xu K, Wen L, Li G, et al. Spatiotemporal cnn for video object segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. CA, USA: IEEE, 2019. 1379−1388.
    [29] Li J, Zhao Y, Fu J, et al. Attention-guided network for semantic video segmentation. IEEE Access, 2019, 7: 140680-140689. doi: 10.1109/ACCESS.2019.2943365
    [30] Tian Y, Zhang Y, Zhou D, et al. Triple attention network for video segmentation. Neurocomputing, 2020, 417: 202-211. doi: 10.1016/j.neucom.2020.07.078
  • 加载中
计量
  • 文章访问数:  92
  • HTML全文浏览量:  60
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-04-08
  • 录用日期:  2021-11-26
  • 网络出版日期:  2022-02-05

目录

    /

    返回文章
    返回