一种边界增强的医学图像小样本分割网络

贾熹滨; 郭雄; 王珞; 杨大为; 杨正汉

doi:10.16383/j.aas.c220994

一种边界增强的医学图像小样本分割网络

doi: 10.16383/j.aas.c220994

贾熹滨^{1, 本,},
郭雄^1,,
王珞^1,,
杨大为^2,,
杨正汉^2,

1.
北京工业大学信息学部北京 100124
2.
首都医科大学附属北京友谊医院北京 100050

基金项目: 国家重点研发项目中国和韩国政府间联合研究项目(2019YFE0107800), 国家自然科学基金(62171298, 82071876) 资助

详细信息

作者简介:
贾熹滨：北京工业大学信息学部教授. 主要研究方向为视觉信息认知与计算, 智能医学图像分析和诊断, 情感计算. 本文通信作者. E-mail: jiaxibin@bjut.edu.cn

郭雄：北京工业大学信息学部硕士研究生. 主要研究方向为计算机视觉, 医学图像分割, 小样本学习. E-mail: guox@emails.bjut.edu.cn

王珞：北京工业大学信息学部讲师. 主要研究方向为图像检索, 深度学习, 医学影像处理以及多模态数据融合. E-mail: wangluo@bjut.edu.cn

杨大为：首都医科大学附属北京友谊医院副教授. 2020年获得首都医科大学博士学位. 主要研究方向为肝脏疾病影像研究. E-mail:yangdawei@ccmu.edu.cn

杨正汉：首都医科大学附属北京友谊医院教授. 1999年获得北京医科大学博士学位. 主要研究方向为腹部疾病影像诊断, 肝细胞癌及癌前病变的早期影像诊断, MRI新技术的开发与应用. E-mail: yangzhenghan@vip.163.com

计量
- 文章访问数: 249
- HTML全文浏览量: 297
- 被引次数: 0
出版历程
- 收稿日期: 2022-12-23
- 录用日期: 2023-07-22
- 网络出版日期: 2023-10-15

A Few-shot Medical Image Segmentation Network With Iterative Boundary Refinement

JIA Xi-Bin^{1, 本
,},
GUO Xiong^1
,,
WANG Luo^1
,,
YANG Da-Wei^2
,,
YANG Zheng-Han^2
,

1.
Faculty of Information Technology, Beijing University of Technology, Beijing 100124
2.
Beijing Friendship Hospital, Capital Medical University, Beijing 100050

Funds: Supported by National Key Research and Development Program of China and South Korea Intergovernmental Joint Research Project (2019YFE0107800), National Natural Science Foundation of China (62171298, 82071876)

More Information

Author Bio:
JIA Xi-Bin　Professor at Faculty of Information Technology, Beijing University of Technology. Her research interest covers visual information cognition and computing, intelligent medical image analysis and diagnosis and emotional calculation. Corresponding author of this paper

GUO Xiong　Master student at the Faculty of Information Technology, Beijing University of Technology. His research interest covers computer vision, medical image segmentation and few-shot learning

LUO Wang　Lecturer at the Faculty of Information Technology, Beijing University of Technology. His research interests include image retrieval, deep learning, medical image processing and multi-modal data fusion

YANG Da-Wei　Vice professor of Beijing Friendship Hospital, Capital Medical University. He received his Ph.D degree from Capital Medical University in 2020. His main research interest is imaging diagnosis and research on liver disease

YANG Zheng-Han　Professor of Beijing Friendship Hospital, Capital Medical University. He received his Ph.D degree from Beijing Medical University in 1999. His research interests include imaging diagnosis of abdominal diseases, early imaging diagnosis of hepatocellular carcinoma and precancerous lesions, and development and application of new MRI technology

摘要

摘要: 精准的医学图像自动分割是临床影像学诊断和影像三维重建的重要基础.但医学图像数据的目标对象间对比度差异小、受器官运动影响大, 加之标注样本规模小, 因此在小样本下建立高性能的医学分割模型仍是目前的难点问题. 针对主流原型学习小样本分割网络对医学图像边界分割性能差的问题, 提出一种迭代边界优化的小样本分割网络(Iterative boundary refinement based few-shot-segmentation network, IBR-FSS-Net). 以双分支原型学习的小样本分割框架为基础引入类别注意力机制和密集比较模块, 对粗分割掩码进行迭代优化, 引导分割模型在多次迭代学习过程中关注边界, 从而提升边界分割精度. 为进一步克服医学图像训练样本少且多样性不足的问题, 使用超像素方法生成伪标签, 扩充训练数据以提升模型泛化性. 在主流的ABD-MR和ABD-CT医学图像分割公共数据集上进行实验, 与现有多种先进的医学图像小样本分割方法进行了广泛的对比分析和消融实验. 结果表明, 该方法有效提升了未见医学类别的分割性能.
- 医学图像分割 /
- 小样本学习 /
- 注意力机制 /
- 边界优化
Abstract: Accurate automatic segmentation of medical images is an important basis for clinical imaging diagnosis and 3D image reconstruction. However, medical image data has small contrast differences between target objects, is greatly affected by organ movement, and the scale of labeled samples is small. Therefore, it is still a difficult problem to establish a high-performance medical segmentation model under few samples. In view of the poor performance of the mainstream prototype learning few-shot segmentation network for medical image boundary segmentation, an iterative boundary refinement few-shot segmentation network (IBR-FSS-Net) is proposed.Based on the few-shot segmentation framework of dual-branch prototype learning, the category attention mechanism and dense comparison module are introduced to iteratively refine the coarse segmentation mask, and guide the segmentation model to focus on the boundary during multiple iterative learning processes, thereby improving the boundary segmentation accuracy. In order to further overcome the problem of few training samples and insufficient diversity of medical images, this paper uses the superpixel method to generate pseudo-labels and expand the training data to improve the generalization of the model. Experiments on the mainstream ABD-MR and ABD-CT medical image segmentation public datasets is done, and conducts extensive comparative analysis and ablation experiments with various existing advanced medical image few-shot segmentation methods. The results show that this method effectively improves the segmentation performance of unseen medical categories.
- Medical image segmentation /
- few-shot learning /
- attention mechanism /
- boundary refinement
注释:

1) 本文责任编委 XXX Recommended by Associate Editor BIAN Wei

HTML全文

图 1 基于注意力机制的边界优化模块

Fig. 1 Boundary refinement module based on attention mechanism

下载: 全尺寸图片幻灯片

图 2 基于注意力机制的边界优化模块

Fig. 2 Boundary refinement module based on attention mechanism

下载: 全尺寸图片幻灯片

图 3 原型网络结构图

Fig. 3 Prototype network structure diagram

下载: 全尺寸图片幻灯片

图 4 密集比较模块

Fig. 4 Dense comparison module

下载: 全尺寸图片幻灯片

图 5 基于超像素的样本扩充方法

Fig. 5 Self-supervised learning process based on superpixel algorithm

下载: 全尺寸图片幻灯片

图 6 数据集划分示意图

Fig. 6 ABD-MR dataset partition diagram

下载: 全尺寸图片幻灯片

图 7 医学图像小样本分割网络的Baseline模型

Fig. 7 Baseline model of few-shot medical image segmentation network

下载: 全尺寸图片幻灯片

图 8 核磁共振成像图像中的四种器官样例的预测掩码

Fig. 8 Prediction masks for four organ samples in magnetic resonance images

下载: 全尺寸图片幻灯片

图 9 电子计算机断层扫描图像中的四种器官样例的预测掩码

Fig. 9 Prediction masks for four organ samples in computed tomography images

下载: 全尺寸图片幻灯片

表 1 在 ABD-CT 与 ABD-MR 数据集上不同模型的量化分割结果(%)

Table 1 Quantitative segmentation results of different models on ABD-CT and ABD-MR datasets (%)

方法	ABD-CT数据集					ABD-MR数据集
方法	脾脏	左肾	右肾	肝脏	平均值	脾脏	左肾	右肾	肝脏	平均值
SE-Net	0.23	32.83	14.34	0.27	11.91	51.80	62.11	61.32	27.43	50.66
PANet	25.59	32.34	17.37	38.42	29.42	50.90	53.45	38.64	42.26	46.33
ALPNet	60.25	63.34	54.82	73.65	63.02	67.02	73.63	78.39	73.05	73.02
GCN-DE	56.53	68.13	75.50	46.77	61.73	60.63	76.07	83.03	49.47	67.30
RP-Net	69.85	70.48	70.00	79.62	72.48	76.35	81.40	85.78	73.51	79.26
ADNet	—	—	—	—	—	75.92	75.28	83.28	80.81	78.82
PoissonSeg	52.33	50.11	47.02	58.74	52.05	52.85	50.58	53.57	61.03	54.51
AAS-DCL	66.36	64.71	69.95	71.61	68.16	74.86	76.90	83.75	69.94	76.36
IBR-FSS-Net	71.73	73.78	72.02	78.13	73.92	75.12	82.19	85.64	75.89	79.71

下载: 导出CSV

表 2 不同组件的量化分割结果(%)

Table 2 Quantified segmentation results for different components (%)

组件	脾脏	左肾	右肾	肝脏	平均值
Baseline	62.33	63.65	66.87	64.18	64.26
Baseline+Concat	60.62	65.55	68.53	66.56	65.32
Baseline+BRM	66.63	72.10	74.83	69.17	70.68
Baseline+3Concat	61.20	68.72	70.38	66.95	66.81
Baseline+3BRM	75.12	82.19	85.64	75.89	79.71

下载: 导出CSV

表 3 边界优化模块的数量的影响(%)

Table 3 The effect of the number of boundary refinement modules (%)

组件	脾脏	左肾	右肾	肝脏	平均值
Baseline	62.33	63.65	66.87	64.18	64.26
Baseline+BRM	66.63	72.10	74.83	69.17	70.68
Baseline+2BRM	69.88	79.98	82.12	73.56	76.39
Baseline+3BRM	75.12	82.19	85.64	75.89	79.71
Baseline+4BRM	68.57	77.23	78.82	69.60	73.56
Baseline+5BRM	64.13	70.55	72.69	66.42	68.45

下载: 导出CSV

表 4 不同特征提取网络的量化分割结果(%)

Table 4 Quantified segmentation results of different feature extraction networks (%)

骨干网络	脾脏	左肾	右肾	肝脏	平均值
VGG-16	52.09	63.83	64.48	57.88	59.57
U-Net	69.66	78.94	80.46	72.15	75.30
Res U-Net	71.82	78.24	81.10	73.41	76.14
Attention U-Net	73.96	79.14	83.51	73.60	77.55
ResNet-50	71.23	78.19	82.57	73.68	76.42
ResNet-101	75.12	82.19	85.64	75.89	79.71

下载: 导出CSV

表 5 度量网络组合方式的影响(%)

Table 5 The effect of the combination of metric network (%)

原型网络拼接方式	脾脏	左肾	右肾	肝脏	平均值
Prototypical-Net	70.92	80.61	83.70	74.48	77.43
DCM	71.53	81.36	83.44	74.81	77.79
DCM + Prototypical-Net	72.97	81.49	83.68	74.83	78.24
Prototypical-Net +DCM	75.12	82.19	85.64	75.89	79.71

下载: 导出CSV

表 6 与使用半监督、对比学习、弱监督的医学分割模型在 ABD-CT 与 ABD-MR 数据集上结果对比(%)

Table 6 Comparison of results with medical segmentation models ssing semi-supervised, comparative learning, and weakly supervised on ABD-CT and ABD-MR datasets (%)

方法(比例)	ABD-CT数据集					ABD-MR数据集
方法(比例)	脾脏	左肾	右肾	肝脏	平均值	脾脏	左肾	右肾	肝脏	平均值
MagicNet(30%)	91.42	86.19	84.64	93.89	89.04	—	—	—	—	—
CVCL(部分)	95.40	94.60	94.60	96.70	95.33	—	—	—	—	—
C-CAM(0%)	—	—	—	—	—	74.16	81.00	84.75	72.68	78.15
IBR-FSS-Net(5%)	71.73	73.78	72.02	78.13	73.92	75.12	82.19	85.64	75.89	79.71

下载: 导出CSV

参考文献(38)

[1]	Zhang B, Zhang L, Zhang L, Karray F. Retinal vessel extraction by matched filter with first-order derivative of Gaussian. Computers in Biology and Medicine, 2010, 40(4): 438-445 doi: 10.1016/j.compbiomed.2010.02.008
[2]	Vinyals O, Blundell C, Lillicrap T, Wierstra D. Matching networks for one shot learning. In: Proceedings of the PMLR on Advances in Neural Information Processing Systems. Barcelona, Spain: PMLR, 2016. 29
[3]	Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the PMLR on International Conference on Machine Learning. Sydney, Australia: PMLR, 2017. 1126−1135
[4]	Sung F, Yang Y, Zhang L, Tao X, Philip H S T, Timothy M. Learning to compare: Relation network for few-shot learning. In: Proceedings of the IEEE on Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 1199−1208
[5]	Li G, Jampani V, Sevilla-Lara L, Sun D, Kim Jonghyun, Joongkyu Kim. Adaptive prototype learning and allocation for few-shot segmentation. In: Proceedings of the IEEE on Conference on Computer Vision and Pattern Recognition. Virtual Event: IEEE, 2021. 8334−8343
[6]	Yu Q, Dang K, Tajbakhsh N, et al. A location-sensitive local prototype network for few-shot medical image segmentation. In: Proceedings of the IEEE on Image Processing. Anchorage, Alaska, USA: IEEE, 2021. 262−266
[7]	孙君梅, 葛青青, 李秀梅, 赵宝奇. 具有边界增强功能的医学图像分割网络. 电子与信息学报, 2022, 44(5): 1643-1652 doi: 10.11999/JEIT210784 Sun Jun-Mei, Ge Qing-Qing, Li Xiu-Mei, Zhao Bao-Qi. A medical image segmentation network with boundary enhancement. Journal of Electronics, 2022, 44(5): 1643-1652. doi: 10.11999/JEIT210784
[8]	Yuan Y, Chen X, Wang J. Object-contextual representations for semantic segmentation. In: Proceedings of the European Conference on Computer Vision. Virtual Event: Springer, 2020. 23-28, 173−190
[9]	Kim T, Lee H, Kim D. Uacanet: Uncertainty augmented context attention for polyp segmentation. In: Proceedings of the IEEE Conference on ACM International Conference on Multimedia. Chengdu, China: ACM, 2021. 2167−2175
[10]	Zhang C, Lin G, Liu F, Rui Y, Chunhua S. Canet: Class-agnostic segmentation networks with iterative refinement and attentive few-shot learning. In: Proceedings of the IEEE Conference on Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019. 5217−5226
[11]	Ouyang C, Biffi C, Chen C, Kart T, Qiu H, Rueckert D. Self-supervision with superpixels: Training few-shot medical image segmentation without annotation. In: Proceedings of the European Conference on Computer Vision. Virtual Event: Springer, 2020. 762−780
[12]	Tang H, Liu X, Sun S, Yan X, Xie X. Recurrent mask refinement for few-shot medical image segmentation. In: Proceedings of IEEE on International Conference on Computer Vision: Seoul, South Korea: IEEE, 2021. 3918−3928
[13]	Fan D P, Ji G P, Zhou T, Chen G, Fu H, Shen J, et al. Pranet: Parallel reverse attention network for polyp segmentation. In: Proceedings of the Medical Image Computing and Computer Assisted Intervention. Lima, Peru: Springer, 2020. 263−273
[14]	Shaban A, Bansal S, Liu Z, Essa I, Byron B. One-shot learning for semantic segmentation. In: Proceedings of the IEEE on Computer Vision and Pattern Recognition. Hawaii, USA: IEEE, 2017.
[15]	Wang K, Liew J H, Zou Y, Zhou D, Feng J. Panet: Few-shot image semantic segmentation with prototype alignment.In: Proceedings of the IEEE on International Conference on Computer Vision. Seoul, South Korea: IEEE, 2019. 9197−9206
[16]	Tian Z, Zhao H, Shu M, Yang Z, Li R, Jia J. Prior guided feature enrichment network for few-shot segmentation. Pattern Analysis and Machine Intelligence, 2020. 44(2): 1050-1065
[17]	Zhang G, Kang G, Yang Y, Wei Y. Few-shot segmentation via cycle-consistent transformer. In: Proceedings of the Advances in Neural Information Processing Systems. Montreal Canada: NeurIPS, 2021. 21984−21996
[18]	Roy A G, Siddiqui S, Polsterl S, Navab N, Wachinger C. “Squeeze & excit” guided few-shot segmentation of volumetric images. Medical Image Analysis, 2020, 59: Article No. 101587
[19]	Sun L, Li C, Ding X, Huang Y, Chen Z, Wang G, et al. Few-shot medical image segmentation using a global correlation network with discriminative embedding. Computers in Biology an Medicine, 2022, 140: Article No. 105067
[20]	Tang H, Liu X, Sun S, Yan X, Xie X. Recurrent mask refinement for few-shot medical image segmentation. In: Proceedings of the IEEE on International Conference on Computer Vision. Montreal, Canada: IEEE, 2021. 3918−3928
[21]	Hansen S, Gautam S, Jenssen R, Kampffmeyer M. Anomaly detection-inspired few-shot medical image segmentation through self-supervision with supervoxels. Medical Image Analysis, 2022, 78: Article No. 102385
[22]	Shen X, Zhang G, Lai H, et al. PoissonSeg: Semi-supervised few-shot medical image segmentation via poisson learning. In: Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine. Houston, USA: IEEE, 2021. 1513−1518
[23]	Wu H, Xiao F, Liang C. Dual Contrastive learning with anatomical auxiliary supervision for few-shot medical image segmentation. In: Proceedings of the European Conference on Computer Vision. Tel Aviv, Israel: Springer, 2022. 417−434
[24]	Chen D, Bai Y, Shen W, Li Q, Yu L, Wang Y. MagicNet: Semi-supervised multi-Organ segmentation via magic-cube partition and recovery. In: Proceedings of the IEEE on Computer Vision and Pattern Recognition. Oxford, UK: IEEE, 2023. 23869−23878
[25]	Liu P, Zheng G. Context-aware voxel-wise contrastive learning for label efficient multi-organ segmentation. In: Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. Singapore: Springer, 2022. 653−662
[26]	Chen Z, Tian Z, Zhu J, Li C, Du S. C-cam: Causal cam for weakly supervised semantic segmentation on medical image. In: Proceedings of the IEEE on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE, 2022. 11676−11685
[27]	Chen S, Tan X, Wang B, Hu X. Reverse attention for salient object detection. In: Proceedings of the European Conference on Computer Vision. Munich, Germany: Springer, 2018. 234−250
[28]	陈琼, 杨永, 黄田琳, 冯媛. 新型全向立体视觉系统设计. 数据与计算发展前沿, 2022, 3(6): 17-34 Chen Qiong, Yang Yong, Huang Tian-Lin, Feng Yuan. A survey on few-shot image semantic segmentation. Frontiers of Data and Domputing, 2022, 3(6): 17-34.
[29]	Snell J, Swersky K, Zemel R. Prototypical networks for few-shot learning. In: Proceedings of the Conference on Neural Information Processing Systems. Long Beach, USA: NeurIPS, 2017. 30
[30]	Landman B, Xu Z, Igelsias J, Styner M, Langerak T, Klein A. Miccai multi-atlas labeling beyond the cranial vault-workshop and challenge.In: Proceedings of the Medical Image Computing and Computer Assisted Intervention. Munich, Germany: Springer, 2015. 12
[31]	Kavur A E, Gezer N S, Baris M, Aslan S, Conze P H, Groza V. CHAOS challenge-combined (CT-MR) healthy abdominal organ segmentation. Medical Image Analysis, 2021, 69: Atricle No. 101950
[32]	Irving B. MaskSLIC: regional superpixel generation with application to local pathology characterisation in medical images. In: Proceedings of the IEEE on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016. 09518
[33]	Lin T Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, et al. Microsoft coco: Common objects in context.In: Proceedings of the European Conference on Computer Vision. Zurich, Switzerland: Springer, 2014. 740−755
[34]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proceedings of the IEEE on Computer Vision and Pattern Recognition. Columbus, USA: IEEE, 2014. 1409. 1556
[35]	Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In: Proceedings of the Medical Image Computing and Computer-Assisted Intervention. Munich, Germany: Springer, 2015. 234−241
[36]	Xiao X, Lian S, Luo Z, Li S. Weighted res-unet for high-quality retina vessel segmentation. In: Proceedings of the International Conference on Information Technology in Medicine and Education. Hangzhou, China: IEEE, 2018. 327−331
[37]	Oktay O, Schlemper J, Folgoc L L, Lee M, Heinrich M, Misawa K. Attention U-net: Learning where to look for the pancreas. In: Proceedings of the IEEE on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018. 1804. Atricle No. 03999
[38]	Kervadec H, Bouchtiba J, Desrosiers C, Granger E, Dolz J, Ayed I.B. Boundary loss for highly unbalanced segmentation.In: Proceedings of the International Conference on Medical Imaging with Deep Learning. Shanghai, China: PMLR, 2019: 285−296