肝纖維化是因為肝臟組織受到反覆的破壞及修復而導致細胞外基質蛋白過度堆積，而長期的纖維化將進一步造成肝硬化甚至是肝癌。肝臟的纖維化是可逆反應，然而一旦變為肝硬化將導致肝臟永遠無法回復至健康狀態，因此早期診斷肝臟纖維化是非常重要的議題。以肝臟穿刺搭配病理解剖學分析是臨床上評估肝臟纖維化程度的黃金準則，但此方法具有侵入性不可長期、反覆的使用以追蹤病情，如果取樣的範圍是病人沒有纖維堆積或者硬化的部分則可能導致誤診。因此，本研究將利用非侵入性的醫療超音波系統來擷取Sprague Drawley大鼠的肝臟切片影像，使用卷積神經網路來提取超音波影像之特徵並自動化區分肝臟的纖維化程度。為了取得大鼠由健康至硬化的肝臟超音波影像，我們注射四氯化碳混和橄欖油之藥劑進行肝纖維化的誘發，並使用中心頻率7.5MHz的單振源換能器以Raster-scan的掃描方式取得超音波影像。神經網路部份我們選擇在自然影像中具有優秀分類能力的AlexNet, VGGNet, 以及GoogleNet來找出適合用於醫學影像分類之數學模型。因醫學影像獲取不易，而訓練資料的不足將導致模型過度擬合訓練資料並造成模型準確度降低，轉換學習在此就扮演了非常重要的腳色，將上述模型在自然影像中訓練完之權重參數轉換至訓練醫學影像的模型中，這將幫助我們使用少量的醫學影像即可完成模型訓練。此外利用影像位移、旋轉、縮放以及增加對比度或雜訊等方式也可以增加訓練影像的數量。為了瞭解CNN模型是依照什麼樣的特徵來辨別不同纖維化的肝臟影像，我們使用一種可視化技術稱為Class Activation Map (CAM) 來標記出CNN模型認為重要的特徵位置。醫學影像難以取得標準答案因此訓練資料的標籤須利用其他的方式取得，先前研究顯示超音波積體逆散射(Integrated Backscatter, IB) 參數僅在肝臟纖維化第三期(S3)才有顯著差異，而Nakagami-m參數上升的趨勢與肝纖維化的嚴重程度一致，其區間為0.55±0.07至0.83±0.02，因此藉由這個一致性的特質，我們可以使用m參數來將肝臟纖維化分為健康(S0)、中度(S1-S2)以及嚴重(S3-S4)的三個區間，而我們提出的CNN模型可以用該標籤達到55.2-58.8% 的分類準確度。

Liver fibrosis is caused by repeating destruction and repair of liver tissue leading to excessively accumulated extracellular matrix proteins. Moreover, long-term fibrosis will further cause liver cirrhosis and even liver cancer. Liver fibrosis is a reversible reaction, but once liver becomes cirrhotic, it will never return to the health state. Therefore, early diagnosis of liver fibrosis is a critical issue. Combination of liver biopsy and pathological anatomy is the gold standard in practice to assess liver fibrosis. However, since this method is invasive, it is difficult to track the condition repeatedly in the long term. In addition, it may cause misdiagnosis if the sample of biopsy is collected from the cirrhosis liver but the sampling area is without fiber accumulation. Therefore, the non-invasive medical ultrasonic system is used in this thesis to capture images of slices in Sprague Drawley rats’ liver, and the convolutional neural network trained by these ultrasonic images selects the feature maps and automatically classifies the degree of liver fibrosis. In order to collect the images of liver from healthy to severe states, we injected carbon tetrachloride (CCl_4) mixed with olive oil in rats to induce liver fibrosis and used a single element transducer with 7.5MHz central frequency to obtain ultrasonic images by Raster-scanning. For convolutional neural network, we applied the AlexNet, VGGNet and the GoogleNet, with excellent classification capability in natural images, to find the mathematical model suitable for medical images classification. We know that medical images are difficult to be acquired, but insufficient training data will lead CNN model to over-fit the training data and cause the low accuracy when classifying testing data. Therefore, transfer learning plays an important role in this dilemma. Apply the weighting parameters from the CNN model trained with natural images, our CNN model could be trained even with fewer medical images. In addition, data augmentation is able to increase the number of training data by shifting, rotating and zooming the images. In order to understand what features that CNN model considered to identify different fibrosis stages images, we applied a visualization technique called Class Activation Map (CAM) to mark out the position that CNN considered important. However, it is difficult to realize the status of tissues in medical images, so the labels of training data have to be obtained in other ways. Previous research demonstrated that ultrasonic Integrated Backscattering (IB) parameter had a significant difference only in the third stage of liver fibrosis (S3), whereas the trend of Nakagami-m parameter changing is consistent with the degree of liver fibrosis, and the interval of m-parameter from healthy to severe states (S0-S4) is 0.66 ± 0.07 to 0.83 ± 0.09. Because of the consistency, we could divide liver fibrosis into three stages of healthy (S0), moderate (S1-S2) and severe stages (S3-S4) through Nakagami-m parameter. Based on the Nakagami-m labels, Alexnet, VGGNet and GoogleNet achieved 58.2%, 58.8% and 55.2% accuracy respectively.

[1] K. P. Batts and J. Ludwig, "Chronic hepatitis. An update on terminology and reporting," The American journal of surgical pathology, vol. 19, no. 12, pp. 1409-1417, 1995.
[2] P. Bedossa, "Intraobserver and interobserver variations in liver biopsy interpretation in patients with chronic hepatitis C," Hepatology, vol. 20, no. 1, pp. 15-20, 1994.
[3] P. Hammel et al., "Regression of liver fibrosis after biliary drainage in patients with chronic pancreatitis and stenosis of the common bile duct," New England Journal of Medicine, vol. 344, no. 6, pp. 418-423, 2001.
[4] K. K. Shung, "General engineering principles in diagnostic ultrasound," IEEE Engineering in Medicine and Biology Magazine, vol. 6, no. 4, pp. 7-13, 1987.
[5] K. K. Shung and G. A. Thieme, Ultrasonic scattering in biological tissues. CRC press, 1992.
[6] D. H. Evans, W. N. McDicken, R. Skidmore, and J. Woodcock, Doppler ultrasound: Physics, instrumentation, and clinical applications. J. Wiley & Sons, 1989.
[7] E. Papini et al., "Risk of malignancy in nonpalpable thyroid nodules: predictive value of ultrasound and color-Doppler features," The Journal of Clinical Endocrinology & Metabolism, vol. 87, no. 5, pp. 1941-1946, 2002.
[8] J. Bamber et al., "EFSUMB guidelines and recommendations on the clinical use of ultrasound elastography. Part 1: Basic principles and technology," Ultraschall in der Medizin-European Journal of Ultrasound, vol. 34, no. 02, pp. 169-184, 2013.
[9] D. Cosgrove et al., "EFSUMB guidelines and recommendations on the clinical use of ultrasound elastography. Part 2: Clinical applications," Ultraschall in der Medizin-European Journal of Ultrasound, vol. 34, no. 03, pp. 238-253, 2013.
[10] P. M. Shankar, "Ultrasonic tissue characterization using a generalized Nakagami model," IEEE transactions on ultrasonics, ferroelectrics, and frequency control, vol. 48, no. 6, pp. 1716-1720, 2001.
[11] P. M. Shankar, "A general statistical model for ultrasonic backscattering from tissues," IEEE transactions on ultrasonics, ferroelectrics, and frequency control, vol. 47, no. 3, pp. 727-736, 2000.
[12] M.-C. Ho et al., "Using ultrasound Nakagami imaging to assess liver fibrosis in rats," Ultrasonics, vol. 52, no. 2, pp. 215-222, 2012.
[13] B.-Y. Huang, "In vivo Quantitative Assessment of Hepatic Fibrosis by Ultrasonic Backscattering and Statistical Model," National Cheng Kung University, 2017.
[14] Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015.
[15] N. M. Nasrabadi, "Pattern recognition and machine learning," Journal of electronic imaging, vol. 16, no. 4, p. 049901, 2007.
[16] M. Lin, Q. Chen, and S. Yan, "Network in network," arXiv preprint arXiv:1312.4400, 2013.
[17] J. Long, E. Shelhamer, and T. Darrell, "Fully convolutional networks for semantic segmentation," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 3431-3440.
[18] K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
[19] K. K. Shung, Diagnostic ultrasound: Imaging and blood flow measurements. CRC press, 2015.
[20] G. E. Dahl, T. N. Sainath, and G. E. Hinton, "Improving deep neural networks for LVCSR using rectified linear units and dropout," in Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on, 2013, pp. 8609-8613: IEEE.
[21] V. Nair and G. E. Hinton, "Rectified linear units improve restricted boltzmann machines," in Proceedings of the 27th international conference on machine learning (ICML-10), 2010, pp. 807-814.
[22] S. Hochreiter, "The vanishing gradient problem during learning recurrent neural nets and problem solutions," International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 6, no. 02, pp. 107-116, 1998.
[23] G. E. Hinton and R. R. Salakhutdinov, "Replicated softmax: an undirected topic model," in Advances in neural information processing systems, 2009, pp. 1607-1614.
[24] E. Jang, S. Gu, and B. Poole, "Categorical reparameterization with gumbel-softmax," arXiv preprint arXiv:1611.01144, 2016.
[25] J. Shore and R. Johnson, "Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy," IEEE Transactions on information theory, vol. 26, no. 1, pp. 26-37, 1980.
[26] P.-T. De Boer, D. P. Kroese, S. Mannor, and R. Y. Rubinstein, "A tutorial on the cross-entropy method," Annals of operations research, vol. 134, no. 1, pp. 19-67, 2005.
[27] Y. Bengio, P. Simard, and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult," IEEE transactions on neural networks, vol. 5, no. 2, pp. 157-166, 1994.
[28] C. Burges et al., "Learning to rank using gradient descent," in Proceedings of the 22nd international conference on Machine learning, 2005, pp. 89-96: ACM.
[29] L. Bottou, "Large-scale machine learning with stochastic gradient descent," in Proceedings of COMPSTAT'2010: Springer, 2010, pp. 177-186.
[30] T. G. Marques et al., "Review of experimental models for inducing hepatic cirrhosis by bile duct ligation and carbon tetrachloride injection," Acta cirurgica brasileira, vol. 27, no. 8, pp. 589-594, 2012.
[31] A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in neural information processing systems, 2012, pp. 1097-1105.
[32] C. Szegedy et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9.
[33] C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 2818-2826.
[34] M. Nakagami, "The m-distribution—A general formula of intensity distribution of rapid fading," in Statistical methods in radio wave propagation: Elsevier, 1960, pp. 3-36.
[35] O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in International Conference on Medical image computing and computer-assisted intervention, 2015, pp. 234-241: Springer.
[36] L. Perez and J. Wang, "The effectiveness of data augmentation in image classification using deep learning," arXiv preprint arXiv:1712.04621, 2017.
[37] M. Oquab, L. Bottou, I. Laptev, and J. Sivic, "Learning and transferring mid-level image representations using convolutional neural networks," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2014, pp. 1717-1724.
[38] K. He, X. Zhang, S. Ren, and J. Sun, "Delving deep into rectifiers: Surpassing human-level performance on imagenet classification," in Proceedings of the IEEE international conference on computer vision, 2015, pp. 1026-1034.
[39] S. Hoo-Chang et al., "Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning," IEEE transactions on medical imaging, vol. 35, no. 5, p. 1285, 2016.
[40] D. Meng, L. Zhang, G. Cao, W. Cao, G. Zhang, and B. Hu, "Liver fibrosis classification based on transfer learning and FCNet for ultrasound images," IEEE Access, 2017.
[41] T. Heimann, P. Mountney, M. John, and R. Ionasec, "Real-time ultrasound transducer localization in fluoroscopy images by transfer learning from synthetic training data," Medical image analysis, vol. 18, no. 8, pp. 1320-1328, 2014.
[42] S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Transactions on knowledge and data engineering, vol. 22, no. 10, pp. 1345-1359, 2010.
[43] B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba, "Learning deep features for discriminative localization," in Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on, 2016, pp. 2921-2929: IEEE.
[44] A. H. Fischer, K. A. Jacobson, J. Rose, and R. Zeller, "Hematoxylin and eosin staining of tissue and cell sections," Cold Spring Harbor Protocols, vol. 2008, no. 5, p. pdb. prot4986, 2008.
[45] J. Goldner, "A modification of the Masson trichrome technique for routine laboratory purposes," The American journal of pathology, vol. 14, no. 2, p. 237, 1938.
[46] A. Sagir, A. Erhardt, M. Schmitt, and D. Häussinger, "Transient elastography is unreliable for detection of cirrhosis in patients with acute liver damage," Hepatology, vol. 47, no. 2, pp. 592-595, 2008.
[47] S. A. Khan and S.-P. Yong, "An Evaluation of Convolutional Neural Nets for Medical Image Anatomy Classification," in Advances in Machine Learning and Signal Processing: Springer, 2016, pp. 293-303.