Real-time 2D/3D Registration of Vertebra via Machine Learning and Geometric Transformation
-
摘要: 在图像引导的脊柱手术中，实时高效的2D/3D配准是一项重要且具有挑战性的任务.通常的2D/3D配准一般是将三维图像投影到二维平面，然后进行2D/2D的配准.由于投影空间涉及到3个平移以及3个旋转参数，其投影空间的复杂度为O（n6），使得配准很难兼具高准确性和高实时性.本文提出了一个结合机器学习与几何变换的2D/3D配准方法，首先，使用统计形状模型对目标脊椎进行建模，并构建了一种新的投影方式，使得6个投影参数中的4个可以使用几何的方法计算出来；接下来利用回归学习的方法学习目标脊椎的形状与投影参数之间的关系；最终，结合学到的关系和几何变换完成配准.本方法的两个姿态参数的平均预测误差为0.84°和0.81°，平均目标配准误差（Mean target registration error，mTRE）为0.87mm，平均配准时间为0.9s.实验结果表明本方法具有很好的实时性和准确性.Abstract: In spine operations, an effective 2D/3D registration is of great importance yet a challenging task. Traditional 2D/3D registration approach projects 3D data to a 2D plan then conducts a 2D/2D registration. It is difficult to obtain both high accuracy and good real-time performance, because the projection space has 3 translation and 3 rotation degrees of freedom and its complexity O(n6). In this paper we propose a method which combines machine learning strategy with geometric transformation. We build a shape model using statistical shape model and construct a new projection method, which makes it possible to calculate that 4 of the 6 projection parameters with the geometric method. Then we use regression learning method to learn a pose model between shape of vertebra and projection parameters. Finally we fulfill the registration by using the learned pose model and geometric transformation. Using the proposed method, the mean predicted errors of two pose parameters are 0.84° and 0.81°, respectively. The mean target registration error (mTRE) is 0.87mm. And the mean time of registration is 0.9s. These results show that our method owns both high accuracy and good real-time performance.
-
Key words:
- 2D/3D registration /
- machine learning /
- geometry translation /
- statistical shape model /
- real-time
1) 本文责任编委 黄庆明 -
图 3 提取标志点(我们对每幅图像手动提取93个标志点, 所有标志点总体分为6个部分: (a)为所有的标志点, (b)、(c)、(d)、(e)、(f)、(g)为6个部分每个部分的标志点, 其中(b)为椎体轮廓, (c)为中央灰度凹陷, (d)和(e)接近于生理结构的椎弓根, (f)和(g)为椎体左右下切角)
Fig. 3 Extract landmarks (We manually extract 93 landmarks for each image, all landmarks are divided into 6 parts, in which (a) contains all landmarks, while (b), (c), (d), (e), (f), (g) contain one of 6 parts, among them, (b) is vertebral body contour, (c) is the central gray depression, (d) and (e) are close to the pedicle of the physiological structure, and (f) and (g) are the left and right bottom of the vertebral body.)
图 4 投影变换((a)图是传统的投影方式, ($x, y, z$)为世界坐标系, ($t_x, t_y, t_z$)是三个平移参数($r_x, r_y, r_z$)为三个旋转参数. (b)图是本文构建的投影方式, ($x, y, z$)为世界坐标系, ($x'O'y'$)为投影平面坐标系, 此坐标系沿世界坐标系$z$轴向投影与($xOy$)重合. ($r, \theta, \phi$)为球形坐标系的参数. ($u, v, w$)为投影对象自身姿态坐标系, 此姿态坐标系与球形坐标系联动, 姿态坐标系的$u$轴平行于球坐标系的纬线切线指向如图 4(b)中所示方向, $v$轴平行于球坐标系的经线指向如图 4(b)中所示所示方向, $w$轴沿径向方法指向背离圆心的方向.)
Fig. 4 Projection transformation ((a) is traditional projection transformation, and ($x, y, z$)is world coordinate system, and ($t_x, t_y, t_z$) are three translation parameters, while ($r_x, r_y, r_z$) are three rotation parameters. b) is the proposed projection method. ($x, y, z$) is the world coordinate system, and $x'O'y'$) is the coordinate system of projective plane. This coordinate system coincides with the axial projection of $z$ in the world coordinate system ($xOy$). ($r, \theta, \phi$) are parameters for the spherical coordinate system. ($u, v, w$) is pose projection coordinates of object, and it coact with spherical coordinates as the (b) shows.)
图 6 线性和高阶拟合((a)和(b))与不同采样间隔下预测误差的变化((c)和(d))
Fig. 6 The difference of prediction error between linear model and high order model (a), (b) and by difference sampling intervals (c), (d)
A2 $\Delta r$近似等效于相似变换
A2 The effect of $\Delta r$ approximate to similar transformation
表 1 配准结果
Table 1 Results of registration
对象 姿态误差ru(°) 姿态误差rv(°) mTRE (mm) 时间(s) PA1 0.92±0.69 0.88±0.71 0.88±0.73 0.96 PA2 0.62±0.51 0.70±0.62 1.13±0.75 0.88 PA3 0.52±0.44 0.70±0.58 1.01±0.62 0.88 PA4 1.43±1.05 1.13±0.92 0.77±0.58 0.89 PA5 0.78±0.61 0.62±0.48 0.73±0.45 0.88 PA6 0.76±0.63 0.81±0.64 0.68±0.46 0.88 平均 0.84 0.81 0.87 0.90 
下载: 导出CSV
表 2 各种方法对比
Table 2 Comparison with other methods
作者 方法框架 相似度度量 mTRE (mm) 时间(s) Russakof 基于搜索 互信息 1.3 - Russakof 基于搜索 交叉相关 1.5 - Russakof 基于搜索 梯度相关 1.3 - Russakof 基于搜索 灰度模式 1.6 - Russakof 基于搜索 梯度差 1.3 - Russakof 基于搜索 Diff.图像熵 1.9 - Otake 基于搜索 NGI - 6.3 ~ 54 Philipp 基于学习 纹理 1.05 0.02 本文 基于学习 形状 0.87 0.90
下载: 导出CSV
-
[1] Wu J, Kim M, Peters J, Chung H, Samant S S. Evaluation of similarity measures for use in the intensity-based rigid 2D-3D registration for patient positioning in radiotherapy. Medical Physics, 2009, 36(12):5391-5403 doi: 10.1118/1.3250843 [2] Russakoff D B, Rohlfing T, Adler J R, Maurer C R. Intensity-based 2D-3D spine image registration incorporating a single fiducial marker. Academic Radiology, 2005, 12(1):37-50 doi: 10.1016/j.acra.2004.09.013 [3] Otake Y, Wang A S, Webster S J, Uneri A, Kleinszig G, Vogt S, Khanna A J, Gokaslan Z L, Siewerdsen J H. Robust 3D-2D image registration:application to spine interventions and vertebral labeling in the presence of anatomical deformation. Physics in Medicine and Biology, 2013, 58(23):8535-8553 doi: 10.1088/0031-9155/58/23/8535 [4] Russakoff D B, Rohlfing T, Ho A, Kim D H, Shahidi R, Adler J R, Maurer C R. Evaluation of intensity-based 2D-3D spine image registration using clinical gold-standard data. In:Proceedings of the 2nd International Workshop on Biomedical Image Registration (WBIR 2003). Philadelphia, PA, USA:Springer-Verlag, 2003. 151-160 [5] Zollei L, Grimson E, Norbash A, Wells W. 2D-3D rigid registration of X-ray fluoroscopy and CT images using mutual information and sparsely sampled histogram estimators. In:Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001). Kauai, HI, USA:IEEE, 2001. Ⅱ-696-Ⅱ-703 [6] Kim J, Yin F F, Zhao Y, Kim H J. Effects of X-ray and CT image enhancements on the robustness and accuracy of a rigid 3D/2D image registration. Medical Physics, 2005, 32(4):866-873 doi: 10.1118/1.1869592 [7] Chen H M, Goela A, Garvin G J, Li S. A parameterization of deformation fields for diffeomorphic image registration and its application to myocardial delineation. In:Proceedings of the 13th international conference on Medical Image Computing and Computer-Assisted Intervention——(MICCAI 2010). Beijing, China:Springer-Verlag, 2010. 340-348 [8] Pohl K M, Wells W M, Guimond A, Kasai K, Shenton M E, Kikinis R, et al. Incorporating non-rigid registration into expectation maximization algorithm to segment MR images. In:Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention——(MICCAI 2002). Tokyo, Japan:Springer-Verlag, 2002. 564-571 [9] Chou C R, Frederick B, Mageras G, Chang S, Pizer S. 2D/3D Image Registration using Regression Learning. Computer Vision and Image Understanding, 2013, 117(9):1095-1106 doi: 10.1016/j.cviu.2013.02.009 [10] Cyr C M, Kamal A F, Sebastian T B, Kimia B B. 2D-3D registration based on shape matching. In:Proceedings of the 2000 IEEE Workshop on Mathematical Methods in Biomedical Image Analysis. Washington, D.C., USA:IEEE Computer Society, 2000. 198 [11] Philipp S, Markus N, Karl F, Felix S, Deutschmann H. A novel class of machine-learning-driven real-time 2D/3D tracking methods:texture model registration (TMR). Proceedings of the SPIE, 2011, DOI: 10.1117/12.878147 [12] Gouveia A R, Metz C, Freire L, Klein S. 3D-2D image registration by nonlinear regression. In:Proceedings of the 9th IEEE International Symposium on Biomedical Imaging. Barcelona, Spain:IEEE, 2012. 1343-1346 [13] Cootes T F, Taylor C J, Cooper D H, Graham J. Active shape models-their training and application. Computer Vision and Image Understanding, 1995, 61(1):38-59 doi: 10.1006/cviu.1995.1004 [14] Cootes T F, Edwards G J, Taylor C J. Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(6):681-685 doi: 10.1109/34.927467 [15] van Ginneken B, Frangi A F, Staal J J, Ter Haar Romeny B M, Viergever M A. Active shape model segmentation with optimal features. IEEE Transactions on Medical Imaging, 2002, 21(8):924-933 doi: 10.1109/TMI.2002.803121 [16] Cootes T F, Hill A, Taylor C J, Haslam J. The use of active shape models for locating structures in medical images. In:Proceedings of the 13th International Conference on Information Processing in Medical Imaging. London, UK:Springer-Verlag, 1993. 33-47 [17] Gower J G. Procrustes Analysis. International Encyclopedia of the Social and Behavioral Sciences, 2001, 39(4):12141-12143 [18] Lee D T, Schachter B J. Two algorithms for constructing a Delaunay triangulation. International Journal of Parallel Programming, 1980, 9(3):219-242 -
计量
- 文章访问数: 3003
- HTML全文浏览量: 501
- PDF下载量: 1128
- 被引次数: 0
