在脊柱外科手術中，如脊椎骨鋼釘固定手術，可利用C型臂X光機提供實時的X光透視圖並結合術前之模型以獲得更縝密的脊柱立體模型，其中，將術前的電腦斷層圖與術中的X光透視圖準確地對位是提高手術成功率並降低風險之關鍵。然而，查閱目前已出版之文獻，常見之作法多需以較長時間進行對位，而此舉將使C型臂之優點蕩然無存，相較之下，利用神經網路架構之深度學習具有加速模型對位之開發潛能。
本研究提出了一種將電腦斷層影像與X光透視圖準確對位的方法，此方法係採用卷積神經網絡體系結構，先由手術中的X光透視圖提取出影像中隱含的特徵值及變換矩陣，並根據提取出的資料重建出一個解剖結構，且此結構與從電腦斷層影像得到之結構相同。本系統主要先經由兩個卷積編碼器，再經過中間的融合層，以融合兩個維度間的特徵值，最後經由一個解碼器將結果輸出。根據實驗結果顯示，此方法具有高準確度及更快的處理速度，故本研究具有高度可行性。

In spine surgical procedures, such as spinal instrumentation, a C-arm which provides a real-time fluoroscopic x-ray imaging can be enhanced by involving preoperative models to obtain a more comprehensive 3D descriptions of vertebrae. Accurate registration of preoperative computed tomography (CT) image to intraoperative fluoroscopic x-ray image is crucial for surgical guidance. However, most published research addressed the registration issue with a considerable time cost which cannot afford the advantage of real-time mapping C-arm to 3D CT model. Deep learning approaches have the potential to accelerate the registration procedure with the neural network architecture during operations.
In this paper, we propose a CT-to-fluoroscopy rigid registration approach using convolutional neural network to extract internal features and obtain the transformation matrix in fluoroscopy and to recover anatomical structures from the same in CT. The system consists of two convolutional encoders and one decoder after fusing features in between two dimensions. Experimental results have shown the feasibility and good accuracy at a much faster speed.

[1] Freedman, Daniel, and Tao Zhang. “Interactive graph cut based segmentation with shape priors.” 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). Vol. 1. IEEE, 2005.
[2] Zhang, J et al. “Fast segmentation of bone in CT images using 3D adaptive thresholding.” Computers in biology and medicine vol. 40,2 (2010): 231-6.
[3] Lim, Poay Hoon, Ulas Bagci, and Li Bai. “Introducing Willmore flow into level set segmentation of spinal vertebrae.” IEEE Transactions on Biomedical Engineering 60.1 (2012): 115-122.
[4] Yao, Jianhua, et al. “Detection of vertebral body fractures based on cortical shell unwrapping.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Berlin, Heidelberg, 2012.
[5] Glocker, Ben, et al. “Automatic localization and identification of vertebrae in arbitrary field-of-view CT scans.” International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Berlin, Heidelberg, 2012.
[6] Chen, Hao, et al. “Automatic localization and identification of vertebrae in spine CT via a joint learning model with deep neural networks.” International conference on medical image computing and computer-assisted intervention. Springer, Cham, 2015.
[7] Ronneberger, Olaf, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for biomedical image segmentation.” International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 2015.
[8] Lu, Jen-Tang, et al. “Deepspine: Automated lumbar vertebral segmentation, disc-level designation, and spinal stenosis grading using deep learning.” arXiv preprint arXiv:1807.10215 (2018).
[9] P. Markelj, D. Tomaževič, B. Likar, and F. Pernuš, “A review of 3D/2D registration methods for image-guided interventions,” Medical Image Anal., vol. 16, pp. 642–661, Apr. 2012.
[10] Zheng, Guoyan et al. “Reconstruction of patient-specific 3D bone surface from 2D calibrated fluoroscopic images and point distribution model.” Medical image computing and computer-assisted intervention: MICCAI ... International Conference on Medical Image Computing and Computer-Assisted Intervention vol. 9,Pt 1 (2006): 25-32.
[11] Gill, Sean, et al. “Group-wise registration of ultrasound to CT images of human vertebrae.” Medical Imaging 2009: Visualization, Image-Guided Procedures, and Modeling. Vol. 7261. International Society for Optics and Photonics, 2009.
[12] Gueziri, Houssem-Eddine, and D. Louis Collins. “Fast registration of CT with intra-operative ultrasound images for spine surgery.” International Workshop and Challenge on Computational Methods and Clinical Applications for Spine Imaging. Springer, Cham, 2018.
[13] Miao, Shun, et al. “Real-time 2D/3D registration via CNN regression.” 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI). IEEE, 2016.
[14] Toth, Daniel, et al. “3D/2D model-to-image registration by imitation learning for cardiac procedures.” International journal of computer assisted radiology and surgery 13.8 (2018): 1141-1149.
[15] Doerr, S. A., et al. “Data-driven detection and registration of spine surgery instrumentation in intraoperative images.” Medical Imaging 2020: Image-Guided Procedures, Robotic Interventions, and Modeling. Vol. 11315. International Society for Optics and Photonics, 2020.
[16] Han, Xianfeng, Hamid Laga, and Mohammed Bennamoun. “Image-based 3d object reconstruction: State-of-the-art and trends in the deep learning era.” IEEE transactions on pattern analysis and machine intelligence (2019).
[17] Wang, Ge. “A perspective on deep imaging.” IEEE access 4 (2016): 8914-8924.
[18] Henzler, Phlipp, et al. “Single‐image Tomography: 3D Volumes from 2D Cranial X‐Rays.” Computer Graphics Forum. Vol. 37. No. 2. 2018.
[19] Mandikal, Priyanka, and Venkatesh Babu Radhakrishnan. “Dense 3d point cloud reconstruction using a deep pyramid network.” 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2019.
[20] Çiçek, Özgün, et al. “3D U-Net: learning dense volumetric segmentation from sparse annotation.” International conference on medical image computing and computer-assisted intervention. Springer, Cham, 2016.
[21] Lorensen, William E., and Harvey E. Cline. “Marching cubes: A high resolution 3D surface construction algorithm.” ACM siggraph computer graphics 21.4 (1987): 163-169.
[22] Yuksel, Cem. “Sample elimination for generating Poisson disk sample sets.” Computer Graphics Forum. Vol. 34. No. 2. 2015.
[23] SpineWeb: http://spineweb.digitalimaginggroup.ca/
[24] Lopes, Adriano, and Ken Brodlie. “Improving the robustness and accuracy of the marching cubes algorithm for isosurfacing.” IEEE Transactions on Visualization and Computer Graphics 9.1 (2003): 16-29.
[25] Krizhevsky, Alex, and Geoff Hinton. “Convolutional deep belief networks on cifar-10.” Unpublished manuscript 40.7 (2010): 1-9.
[26] Paszke, Adam, et al. “Pytorch: An imperative style, high-performance deep learning library.” Advances in neural information processing systems. 2019.
[27] Sorkine-Hornung, Olga, and Michael Rabinovich. “Least-squares rigid motion using svd.” Computing 1.1 (2017): 1-5.
[28] J-Y, Hsue. “Registration for 3D CT Images and 2D C-arm Image for Spine Surgery Navigation System”
[29] Taha, Abdel Aziz, and Allan Hanbury. “An efficient algorithm for calculating the exact Hausdorff distance.” IEEE transactions on pattern analysis and machine intelligence 37.11 (2015): 2153-2163.
[30] Franaszek, Marek, and Geraldine S. Cheok. “Selection of fiducial locations and performance metrics for point-based rigid-body registration.” Precision engineering 47 (2017): 362-374.
[31] Chai, Tianfeng, and Roland R. Draxler. “Root mean square error (RMSE) or mean absolute error (MAE)?–Arguments against avoiding RMSE in the literature.” Geoscientific model development 7.3 (2014): 1247-1250.