腕部疾病的臨床診斷中，磁振與超音波影像被廣泛的應用。近年來，許多研究指出正中神經的位移以及形變對於診斷腕隧道症候群患者的嚴重程度有相當大的幫助。然而，醫療影像的判讀能力與觀察者的經驗有相當大的關連。對於經驗不足的觀察者來說，判讀影像的過程也可能存在造成診斷失誤的危險因子。在本篇論文當中，我們基於卷積神經網路的技術，發展了兩套方法來輔助觀察者進行影像判讀。
在磁振影像的判讀中，我們利用卷積神經網路分割出腕隧道、屈肌腱以及正中神經的區域。為了提升分割的準確性，我們首先將T1和T2權重的影像進行對位，以增加輸入影像中的資訊量。我們也在實驗中證明了多模態圖像的對位有助於改善分割結果，三個目標組織的分割結果在平均Dice係數上分別提升了0.2％，2.1％和1.8％。隨後，在腕隧道以及屈肌腱的分割方面，我們修改了DeepLabv3+的架構，結合空洞卷積與DenseNet。在截面積較小且邊界較模糊的正中神經分割中，我們加入了影像序列的分割架構MaskTrack來進行二次分割。在實驗中顯示，利用本方法可以在平均Dice係數上達到80％以上。
在超音波影像的判讀中，我們提出了一個稱做DeepNerve的卷積神經網路架構來完成正中神經的追蹤。我們以修改過後的U-Net做為主要架構，並在編碼器之後加入卷積長短期記憶網路來整合時序上的訊息。此外我們也加入了MaskTrack結構來穩定影像序列的追蹤和分割。在實驗中，不同的卷積神經網路架構之間的比較證實了DeepNerve可以更有效的追蹤正中神經的區域，並在平均Dice係數上達到89％以上。另外，我們也對不同的初始遮罩進行評估，證明DeepNerve能夠以更方便的方式進行初始化。
本篇論文中所提出的兩個關於組織分割與追蹤的方法，都能夠獲得良好分割的效果，期望未來能夠實際應用於醫療儀器當中，即時為受試者提供腕隧道症候群嚴重程度的診斷。

Magnetic resonance (MR) and ultrasound images had been widely used in the clinical diagnosis of wrist diseases. In recent years, many studies have demonstrated that the displacement and deformation of the median nerve can effectively use to measure the severity of carpal tunnel syndrome (CTS). However, the interpretations of medical images are quite dependent to the experience of the observer. The inexperienced observers always have risks when they interpret these images for diagnosing of CTS symptoms. In this thesis, we develop two convolutional neural network (CNN) models to assist clinical physicians to assess these medical images.
In the MR images, we use a CNN model to segment three primary structures in the carpal tunnel that are the carpal tunnel, flexor tendon and median nerve. In order to improve the accuracy of segmentation, we first align the T1-weighted (T1) images with the corresponding T2-weighted (T2) images to effectively increase tissue segmentation information, and then demonstrate the registration procedure can improve the average of Dice similarity coefficient (ADSC) of carpal tunnel, flexor tendon and median nerve on 0.2%, 2.1%, and 1.8%, respectively. In this study, we first modify the DeepLabv3+ by replacing backbone with the DenseNet and adding the dilated convolution, to segment the carpal tunnel and the flexor tendon. To precisely segment median nerve which is small and blur, the MaskTrack architecture is used to integrate the continuity of it. The experimental results demonstrated that the proposed method can exceed to 80% on ADSC.
In the ultrasound image sequence, we propose a new CNN architecture, called the DeepNerve, to track the median nerve. We use a U-Net as the main backbone of MaskTrack and add a convolutional long short-term memory (ConvLSTM) network in the final stage of the encoder. In the experiments, we compare several different CNN architectures and DeepNerve. We find that DeepNerve can track the median nerve of ultrasonic image sequence more effectively, and the corresponding ADSC can superior to 0.89 on ADSC measurement. Furthermore, we also evaluate different initial masks in first frame of test image sequence in order to show that DeepNerve is robust as it uses arbitrary initialization.
The two proposed methods in this thesis can achieve good segmentation results, and we expect that it can be practically applied in medical instruments to immediately assess of the severity of CTS of patients.

[1] A. Isam, G. Christina, J. Ragnar, E. Ornstein, J. Ranstam and I. Rosén, "Prevalence of carpal tunnel syndrome in a general population," Jama, pp. 153-158, 1999.
[2] Y. Yoshii, H. R. Villarraga, J. Henderson, C. Zhao, K.-N. An and P. C. Amadio, "Ultrasound Assessment of the Displacement and Deformation of the Median Nerve in the Human Carpal Tunnel with Active Finger Motion," J Bone Joint Surg Am, p. 2922–2930, 2009.
[3] J. Long, E. Shelhamer and T. Darrell, "Fully Convolutional Networks for Semantic Segmentation," in Conference on Computer Vision and Pattern Recognition, 2015.
[4] O. Ronneberger, P. Fischer and T. Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation," in Computer Vision and Pattern Recognition, 2015.
[5] K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition," in Computer Vision and Pattern Recognition, 2015.
[6] G. Huang, Z. Liu, L. v. d. Maaten and K. Q. Weinberger, "Densely Connected Convolutional Networks," in Computer Vision and Pattern Recognition, 2016.
[7] D. E. Rumelhart, G. E. Hinton and R. J. Williams, "Learning representations by back-propagating errors," Nature, pp. 533-536, 1986.
[8] S. Jégou, M. Drozdzal, D. Vazquez, A. Romero and Y. Bengio, "The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation," in Computer Vision and Pattern Recognition, 2017.
[9] L.-C. Chen, Y. Zhu, G. Papandreou, F. Schroff and H. Adam, "Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation," in Computer Vision and Pattern Recognition, 2018.
[10] A. Khoreva, F. Perazzi, R. Benenson, B. Schiele and A. Sorkine-Hornung, "Learning Video Object Segmentation from Static Images," in Computer Vision and Pattern Recognition, 2016.
[11] J. L. Elman, "Finding Structure in Time," Cognitive Science, pp. 179-211, 1990.
[12] M. I.Jordan, "Advances in Psychology," Advances in Psychology, pp. 471-495, 1997.
[13] S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, pp. 1735-1780, 1997.
[14] K. Cho, B. v. Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk and Y. Bengio, "Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation," in Computation and Language, 2014.
[15] X. Shi, Z. Chen, H. Wang, D.-Y. Yeung, W.-k. Wong and W.-c. Woo, "Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting," in Computer Vision and Pattern Recognition, 2015.
[16] B. Romera-Paredes and P. H. S. Torr, "Recurrent Instance Segmentation," in Computer Vision and Pattern Recognition, 2015.
[17] L. M. Wong, L. Shi, F. Xiao and J. F. Griffith, "Fully automated segmentation of wrist bones on T2-weighted fat-suppressed MR images in early rheumatoid arthritis," Quant Imaging Med Surg, pp. 579-589, 2019.
[18] J. Dolz, K. Gopinath, J. Yuan, H. Lombaert, C. Desrosiers and I. B. Ayed, "HyperDense-Net: A hyper-densely connected CNN for multi-modal image segmentation," in Computer Vision and Pattern Recognition, 2018.
[19] H.-C. Chen, Y.-N. Sun, Y.-Y. Wang, C.-H. Lin, C.-K. Wang and I.-M. Jou, "Segmentation of flexor tendons within carpal tunnel from magnetic resonance image," 2010 International Computer Symposium (ICS2010), pp. 932-935, 2010.
[20] M. Alkhatib, A. Hafiane, O. Tahri, P. Vieyres and A. Delbos, "Adaptive median binary patterns for fully automatic nerves tracking in ultrasound images," Computer Methods and Programs in Biomedicine, pp. 129-140, 2018.
[21] H. Fan and H. Ling, "SANet: Structure-Aware Network for Visual Tracking," in Computer Vision and Pattern Recognition, 2017.
[22] A. Hafiane, P. Vieyres and A. Delbos, "Deep learning with spatiotemporal consistency for nerve segmentation in ultrasound images," in Computer Vision and Pattern Recognition , 2017.
[23] "Philips Ingenia 3.0T," [Online]. Available: https://www.philips.com.tw/healthcare/product/HC781342/ingenia-30t-mr-system.
[24] N. Qian, "On the momentum term in gradient descent learning algorithms," Neural Netw., pp. 145-151, 12 Jan 1999.
[25] L.-C. Chen, G. Papandreou, F. Schroff and H. Adam, "Rethinking Atrous Convolution for Semantic Image Segmentation," in Computer Vision and Pattern Recognition, 2017.
[26] F. Chollet, "Xception: Deep Learning with Depthwise Separable Convolutions," in Computer Vision and Pattern Recognition, 2017.
[27] J. Dai, H. Qi, Y. Xiong, Y. Li, G. Zhang, H. Hu and Y. Wei, "Deformable Convolutional Networks," in Computer Vision and Pattern Recognition, 2017.
[28] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy and A. L. Yuille, "DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs," in Computer Vision and Pattern Recognition, 2017.
[29] H. Chen, Q. Dou, L. Yu and P.-A. Heng, "VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation," in Computer Vision and Pattern Recognition, 2016.
[30] Q. Dou, H. Chen, Y. Jin, L. Yu, J. Qin and P.-A. Heng, "3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes," in Computer Vision and Pattern Recognition, 2016.
[31] D. P. Kingma and J. Ba, "Adam: A Method for Stochastic Optimization," in International Conference for Learning Representations, San Diego, 2015.
[32] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg and L. Fei-Fei, "ImageNet Large Scale Visual Recognition Challenge," in Computer Vision and Pattern Recognition, 2014.
[33] "Siemens ACUSON S2000 Ultrasound System," [Online]. Available: https://www.siemens-healthineers.com/ultrasound/general-imaging/acuson-s2000-ultrasound-machine.
[34] "Siemens ACUSON S2000 18L6 HD Transducer," [Online]. Available: https://www.umiultrasound.com/transducers/siemens-acuson-s2000-18l6-hd-transducer.
[35] S. Ioffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," Proceedings of Machine Learning Research, pp. 448-456, 2015.
[36] P. J. Werbos, "Backpropagation through time: what it does and how to do it," Proceedings of the IEEE, pp. 1550 - 1560, 1990.
[37] R. J. Williams and D. Zipser, "A Learning Algorithm for Continually Running Fully Recurrent Neural Networks," Neural Computation, pp. 270-280, 1989.
[38] T. Tieleman and G. Hinton, "Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude," COURSERA: Neural Networks for Machine Learning, 2012.