近年來，微創手術慢慢成為醫生進行手術時主流的選擇之一，其中微創手術對於病患的優點有: 1.減少手術在過程的出血 2.減少病患疼痛感 3.傷口快速恢復…等優點，微創手術的方法為醫生需要利用螢幕上的內視鏡影像或者超音波影像來輔助手術，但醫生僅僅透過螢幕進行手術時，會損失與病灶的距離感，所以手術前的規畫及訓練的系統是被醫生所需要的。
超音波影像經常被利用來診斷肌肉外傷、軟組織的腫瘤及肌腱病變的嚴重程度，而近年來，板機指為常見的臨床手術疾病議題，所以本論文也以板機指手術來當我們的示範場域，板機指需要透過超音波影像來導引整個微創手術的過程，然而執行微創手術需要有完整的術前規劃、術前練習來讓手術進行得更順利，因此手術訓練系統的建立可以成為手術前輔助醫生的利器。
為了解決上述問題，本論文提出了低成本但高還原度的手術訓練系統來幫助醫生且可以不用在非常特定的環境下進行手術前的訓練，此系統裡主要利用一顆 RGB-D相機，並透過電腦視覺、影像處理、機器學習的技術來完成此系統。 本文中提出的系統技術包括幾個架構:手術假體模型與手術術前影像的空間座標 系之校正、手術器具的偵測、追蹤、姿態辨識，鏡頭世界跟真實世界的校正等。

1.利用深度學習的網路來完成手部外觀的切割並找尋手部特徵點，利用練習的假手的特徵點與手術前檢查時所拍攝的真實手部醫學影像的特徵點來進行影像對位，再藉由影像形變的方式讓假手產生出真人的超音波影像
2.利用深度學習來完成器械的偵測手術模擬器械上的標記點，並透過標記點上的圖形位置來完成標記點的標定達到追蹤的效果，並快速計算出模擬超音波探頭的各種姿態來進行醫療影像的顯示、手術器械的導航。
3.利用校正版與 Pin hole 相機模型公式，透過擷取特徵點與 RANSAC 方法，達到相機世界座標與真實世界座標連結，完成了手術模擬器械、練習的假手與真實的手部醫學影像的彼此連結。
透過以上架構我們可以提供醫生方便且高還原度的術前手術系統，幫助醫生對於手術時進行的步驟進行練習，也讓病患得到良好的超音波影像微創手術的結果。

In recent years, many surgeons have come to prefer minimally invasive surgery to be traditional (open) surgery, which requires larger incisions and, usually a longer hospital stay.
In general, minimally invasive surgery is associated with less pain, a shorter hospital stay,
and few complications.
Minimally invasive surgery is also known as endoscopic surgery or ultrasound-guided surgery.These minimally invasive procedures utilize an endoscope or ultrasound to reach internal organs through tiny incisions.
This technique allows the surgeon to see inside the
patient’s body and operate through a much smaller incision, but the surgeon loses depth perception when viewing the organs on a computer monitor, so surgical planning is needed,in order to predefine the surgical steps and pre-visualizing a surgical intervention.
Ultrasound images are commonly used for the clinical diagnosis of tendinopathy.
Sonography is important in musculoskeletal imaging. As a prototypical scenario, we chose trigger finger procedure in this work. A patient with trigger finger needs to receive treatment through ultrasound-guided minimally invasive surgery. Since surgeons need adequate experience in performing minimally invasive surgery, a surgical training system becomes more and more critical.
In order to address these problems, we aim to provide the most comprehensive and hyper-realistic surgical training system. The system consists of RGB-D sensors, techniques based on computer vision algorithms, image processing, and machine learning.
This system, consist of multiple components including Registration of the preoperative data with hand phantoms, surgical instrument detection, tracking and pose
estimation, and calibration camera information between surgical space and real world.
1. A method is designed to discriminate hand components and to locate features in RGB-D image and to register image based on these features. We use the feature points and find the correspondence of features in the hand phantom and preoperative medical image. With that, we can register the images based on the correspondence of the feature and make hand phantom with realistic medical image information. These augmented reality images help the surgeon to visualize the intra-palm structures and therefore, to practice the operation precisely and to improve clinical outcomes.
2. In this work we propose landmark localization approach based on a Deep Learning architecture that utilizes one stage CNN detection model. We design a tracking by detection algorithm to ensure that the target can be retrieved quickly even if the target is lost or target tracking fails. The entire set of algorithm tracks the landmark accurately for a long time, estimates each instrument pose and generates medical image from pre-operative data on
hand phantom.
3. Design specific calibration board and use a so-called pinhole camera model. In this model, a scene view is formed by projecting 3D points into the image plane using a perspective transformation, solved these problems by robust estimation method for camera parameters based on RANdom Sample Consensus(RANSAC) algorithm. After this
procedure, we can align real-world and image plane coordinate.
After these approaches, we can provide the most comprehensive and hyper-realistic surgical training system for the surgeon. Surgical training system improves patient care,helps to reduce surgical risks, increases the surgeon’s confidence, and thus enhances overall patient safety.

[1] B. P. Crawshaw, H.-L. Chien, K. M. Augestad, and C. P. Delaney, "Effect of Laparoscopic Surgery on Health Care Utilization and Costs in Patients Who Undergo ColectomyCosts of Laparoscopic ColectomyCosts of Laparoscopic Colectomy," JAMA Surgery, vol. 150, no. 5, pp. 410-415, 2015, doi: 10.1001/jamasurg.2014.3171.
[2] J. J. LORTHIOIR, "Surgical Treatment of Trigger-Finger By a Subcutaneous Method," JBJS, vol. 40, no. 4, pp. 793-795, 1958. [Online]. Available: https://journals.lww.com/jbjsjournal/Fulltext/1958/40040/Surgical_Treatment_of_Trigger_Finger_By_a.3.aspx.
[3] M. Ryzewicz and J. M. Wolf, "Trigger digits: principles, management, and complications," (in eng), J Hand Surg Am, vol. 31, no. 1, pp. 135-46, Jan 2006, doi: 10.1016/j.jhsa.2005.10.013.
[4] T. Bamroongshawgasame, "A comparison of open and percutaneous pulley release in trigger digits," (in eng), J Med Assoc Thai, vol. 93, no. 2, pp. 199-204, Feb 2010.
[5] I. M. Jou and T. C. Chern, "Sonographically assisted percutaneous release of the a1 pulley: a new surgical technique for treating trigger digit," The Journal of Hand Surgery: British & European Volume, vol. 31, no. 2, pp. 191-199, 2006/04/01/ 2006, doi: https://doi.org/10.1016/j.jhsb.2005.11.003.
[6] G. E. Hinton and R. R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," (in eng), Science, vol. 313, no. 5786, pp. 504-7, Jul 28 2006, doi: 10.1126/science.1127647.
[7] G. Hinton et al., "Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups," IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 82-97, 2012, doi: 10.1109/MSP.2012.2205597.
[8] J. Deng, W. Dong, R. Socher, L. Li, L. Kai, and F.-F. Li, "ImageNet: A large-scale hierarchical image database," in 2009 IEEE Conference on Computer Vision and Pattern Recognition, 20-25 June 2009 2009, pp. 248-255, doi: 10.1109/CVPR.2009.5206848.
[9] S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2015arXiv150203167I
[10] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," presented at the Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, Lake Tahoe, Nevada, 2012.
[11] C. Szegedy et al., "Going Deeper with Convolutions," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2014arXiv1409.4842S
[12] K. Simonyan and A. Zisserman, "Very Deep Convolutional Networks for Large-Scale Image Recognition," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2014arXiv1409.1556S
[13] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2015arXiv151203385H
[14] G. Huang, Z. Liu, L. van der Maaten, and K. Q. Weinberger, "Densely Connected Convolutional Networks," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2016arXiv160806993H
[15] Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," (in eng), Nature, vol. 521, no. 7553, pp. 436-44, May 28 2015, doi: 10.1038/nature14539.
[16] J. Redmon and A. Farhadi, "YOLO9000: Better, Faster, Stronger," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2016arXiv161208242R
[17] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff, and H. Adam, "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2018arXiv180202611C
[18] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge University Press, 2003, p. 700.
[19] M. A. Fischler and R. C. Bolles, "Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, no. 6, pp. 381-395, 1981, doi: 10.1145/358669.358692.
[20] T.-L. Shen, "Trigger Finger Surgical Training System with 3D Image Reconstruction from Orthogonal Ultrasound Images," 104.
[21] L. Keselman, J. Iselin Woodfill, A. Grunnet-Jepsen, and A. Bhowmik, "Intel RealSense Stereoscopic Depth Cameras," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2017arXiv170505548K
[22] S. Ren, K. He, R. Girshick, and J. Sun, "Faster R-CNN: towards real-time object detection with region proposal networks," presented at the Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 1, Montreal, Canada, 2015.
[23] W. Liu et al., "SSD: Single Shot MultiBox Detector," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2015arXiv151202325L
[24] J. Redmon and A. Farhadi, YOLO9000: Better, Faster, Stronger. 2016.
[25] J. Redmon and A. Farhadi, "YOLOv3: An Incremental Improvement," arXiv e-prints. [Online]. Available: https://ui.adsabs.harvard.edu/abs/2018arXiv180402767R
[26] K. He, X. Zhang, S. Ren, and J. Sun, "Deep Residual Learning for Image Recognition," in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 27-30 June 2016 2016, pp. 770-778, doi: 10.1109/CVPR.2016.90.
[27] D. G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int. J. Comput. Vision, vol. 60, no. 2, pp. 91-110, 2004, doi: 10.1023/b:Visi.0000029664.99615.94.
[28] H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool, "Speeded-Up Robust Features (SURF)," Comput. Vis. Image Underst., vol. 110, no. 3, pp. 346-359, 2008, doi: 10.1016/j.cviu.2007.09.014.