大腸癌是世界常見的癌症之一，同時也是最常見的癌症死因之一。大腸癌根據其病情嚴重不同，其治療的方式包括：手術摘除腫瘤區域、放射性療法和化學療法等方法 殺死癌細胞以及自體免疫療法。然而這些方法都有其相應的副作用，為了減少病人的痛苦以及提高治癒的成功率，選擇對病人最合適的治療方式就是至關重要的課題 即－精準醫療。所謂精準醫療，除了常規的檢查外，另外會再加上生醫檢測，如：基因測序。利用個人的基因表現等資訊，為每個癌症病人量身打造最合適的治療方式 。 對於同為大腸癌的患者來說，美國國家癌症資訊網近年於指引中推薦，確診為大腸癌的患者應當進一步進行微衛星不穩定測試。其理由為，臨床發現，具有微衛星不 穩定狀態的大腸癌患者對免疫療法有著相較於微衛星穩定的患者更佳的反應和預後。 除此之外，也有研究揭露拷貝數變異對於腫瘤的發展和病人的預後有其關聯性。這些臨床研究跡象表明， 透過了解大腸癌患者的基因變異亦有助於精準醫療的研究開發。 但並非所有病人均做過相關的基因檢測或定序。因此本研究希望透過每位癌症患 者常規會接受的病理切片檢驗結果作為資料輸入，利用深度學習模型強大的特徵提取能力，自動關聯影像特徵和各種基因變異，並預測特定基因變異機率。由於數位病理切片的影像解析度相當高且可能只有少部分切片區域有著較為顯著的變異訊號可供判讀，因此我們採用階層式的注意力機制，模型也是圍繞注意力機制來做開發，以此來學習並逐步捕捉重要的影像特徵，最終得到整張切片的影像特徵並以此作出準確預測。 在本研究中，我們使用美國癌症基因基因體圖譜計畫中的大腸直腸癌患者的病理切片影像來進行實驗評估，並與前人已提出的研究方法進行比較 。 其中包括三種基因型的變異：拷貝數變異、微衛星不穩定和全基因複製。實驗數據表明我們提出來的分類模型能夠提高預測的精準度，同時亦顯示我們提出來的模型有著更好發覺變異訊號於切片影像上的能力。此外，我們也發現到，於注意力權重較大的影像上，我們也驗證了與臨床研究發現的同樣特徵，如：淋巴球浸潤。最後，我們還針對額外 14 種的基因拷貝數變異進行實驗，並以此結果當作後續研究的基準。希望我們的研究能夠在未來啟發後續的更多研究，並成為臨床和研究上的一種低成本、具可擴展性的工具。

Colorectal cancer is one of the most common cancers and life-threatening cancers in the world. Modern treatments for colorectal cancer usually involve surgery, radiation therapy, chemotherapy, and immunotherapy. However, those treatments have their side effects. To reduce unnecessary suffering on the patients, selecting the appropriate treatment has become much more important which is also called precision medicine. Expect routine inspections, precision medicine includes biomedical testing such as genetic testing. Using personal genomic expression to tailor the most suitable treatment for each cancer patient. For patients with colorectal cancer, according to the guidelines published by National Comprehensive Cancer Network, colorectal cancer patients are suggested to further take the microsatellite instability (MSI) tests. Since clinical findings show that colorectal cancer patients with MSI respond well to immunotherapy. Moreover, there is research revealed that copy number alterations (CNAs) play an important role in cancer development and patient prognosis. These clinical researches or findings indicate that cancer patient’s genetic alterations status is critical to precision treatment. However, genetic testing is not a regular examination of all patients. Therefore, in this study, we aim to use ubiquitously available clinical data – histopathology images as inputs and predict the probability of alteration by using the deep learning model. We build our model based on the attention mechanism to automatically capture meaningful features of the entire input slides for making predictions accurately. In this study, we conduct our experiments on whole slide images from The Cancer Genome Atlas (TCGA) colon adenocarcinoma (COAD) and rectum adenocarcinoma (READ) datasets and evaluate the performance compared with previous studies. The experiments include CNAs, MSI, and whole-genome doubling (WGD) three different genetic alterations binary classification. The results demonstrate that our proposed attention-based model outperformed previous studies in multiple independent tasks. We also find that some clinical findings, such as lymphocyte infiltration, exist in the sub-regions that have the highest attention weight. Finally, we also conduct 14 different copy number alteration experiments as the baseline for the following research. We hope that our research can inspire more follow-up studies in the future and become a low-cost and scalable tool for clinical and research purposes.

Ba, J. L., Kiros, J. R., & Hinton, G. E. "Layer Normalization", 2016.
Bejnordi, B. E., Zuidhof, G., Balkenhol, M., Hermsen, M., Bult, P., van Ginneken, B., … van der Laak, J. "Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images". ArXiv, 4(4), 2017.
Brosens, R. P. M., Belt, E. J. T. H., Haan, J. C., Buffart, T. E., Carvalho, B., Grabsch, H., … Meijer, G. A. "Deletion of chromosome 4q predicts outcome in stage II colon cancer patients". Analytical Cellular Pathology (Amsterdam), 33(2), 95–104, 2010.
Brosens, R. P. M., Belt, E. J. T. H., Haan, J. C., Buffart, T. E., Carvalho, B., Grabsch, H., … Meijer, G. A. "Deletion of chromosome 4q predicts outcome in stage II colon cancer patients". Cellular Oncology (Dordrecht), 34(3), 215–223, 2011.
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., … Fuchs, T. J. "Clinical-grade computational pathology using weakly supervised deep learning on whole slide images". Nature Medicine, 25(8), 1301–1309, 2019.
Chen, M., Zhang, B., Topatana, W., Cao, J., Zhu, H., Juengpanich, S., … Cai, X. "Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning". Npj Precision Oncology, 4(1), 1–7, 2020.
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. "A simple framework for contrastive learning of visual representations". ArXiv, (Figure 1), 2020.
Chikontwe, P., Kim, M., Nam, S. J., Go, H., & Park, S. H. "Multiple Instance Learning with Center Embeddings for Histopathology Classification". In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2020.
Ciga, O., Martel, A. L., & Xu, T. "Self supervised contrastive learning for digital histopathology", 2020.
Combalia, M., & Vilaplana, V. "Monte-carlo sampling applied to multiple instance learning for histological image classification". Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 11045 LNCS, 274–281, 2018.
Coudray, N., Ocampo, P. S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., … Tsirigos, A. "Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning". Nature Medicine, 24(10), 1559–1567, 2018.
Coudray, N., Ocampo, P. S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., … Tsirigos, A. "Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning". Nature Medicine, 24(10), 1559–1567, 2018.
Dehaene, O., Camara, A., Moindrot, O., de Lavergne, A., & Courtiol, P. "Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histology", 2020.
Dietterich, T. G., Lathrop, R. H., & Lozano-Pérez, T. "Solving the multiple instance problem with axis-parallel rectangles". Artificial Intelligence, 89(1–2), 31–71, 1997.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., … Houlsby, N. "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale". 1–21, 2020.
Dudley, J. C., Lin, M. T., Le, D. T., & Eshleman, J. R. "Microsatellite instability as a biomarker for PD-1 blockade". Clinical Cancer Research, 22(4), 813–820, 2016. 45
Echle, A., Rindtorff, N. T., Brinker, T. J., Luedde, T., Pearson, A. T., & Kather, J. N. "Deep learning in cancer pathology: a new generation of clinical biomarkers". British Journal of Cancer, 124(4), 686–696, 2021.
Everingham, M., van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. "The pascal visual object classes (VOC) challenge". International Journal of Computer Vision, 88(2), 303–338, 2010.
Fu, Y., Jung, A. W., Torne, R. V., Gonzalez, S., Vöhringer, H., Shmatko, A., … Gerstung, M. "Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis". Nature Cancer, 1(8), 800–810, 2020.
Greenson, J. K., Huang, S.-C., Herron, C., Moreno, V., Bonner, J. D., Tomsho, L. P., … Gruber, S. B. "Pathologic predictors of microsatellite instability in colorectal cancer". The American Journal of Surgical Pathology, 33(1), 126–133, 2009.
Hashimoto, N., Fukushima, D., Koga, R., Takagi, Y., Ko, K., Kohno, K., … Takeuchi, I. "Multi-scale Domain-Adversarial Multiple-instance CNN for Cancer Subtype Classification with Unannotated Histopathological Images". Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 3851–3860, 2020.
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. "Momentum Contrast for Unsupervised Visual Representation Learning". Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 9726–9735, 2020.
He, K., Zhang, X., Ren, S., & Sun, J. "Deep residual learning for image recognition". Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem, 770–778, 2016.
Hou, L., Samaras, D., Kurc, T. M., Gao, Y., Davis, J. E., & Saltz, J. H. "Patch-Based Convolutional Neural Network for Whole Slide Tissue Image Classification". Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-Decem, 2424–2433, 2016.
Iizuka, O., Kanavati, F., Kato, K., Rambeau, M., Arihiro, K., & Tsuneki, M. "Deep Learning Models for Histopathological Classification of Gastric and Colonic Epithelial Tumours". Scientific Reports, 10(1), 1–11, 2020.
Ilse, M., Tomczak, J. M., & Welling, M. "Attention-based deep multiple instance learning". 35th International Conference on Machine Learning, ICML 2018, 5(Mil), 3376–3391, 2018.
Ioffe, S., & Szegedy, C. "Batch normalization: Accelerating deep network training by reducing internal covariate shift". 32nd International Conference on Machine Learning, ICML 2015, Vol. 1, pp. 448–456, 2015.
Jang, H. J., Lee, A., Kang, J., Song, I. H., & Lee, S. H. "Prediction of clinically actionable genetic alterations from colorectal cancer histopathology images using deep learning". World Journal of Gastroenterology, 26(40), 6207–6223, 2020.
Kather, J. N., Heij, L. R., Grabsch, H. I., Loeffler, C., Echle, A., Muti, H. S., … Luedde, T. "Pan-cancer image-based detection of clinically actionable genetic alterations". Nature Cancer, 1(8), 789–799, 2020.
Kather, J. N., Pearson, A. T., Halama, N., Jäger, D., Krause, J., Loosen, S. H., … Luedde, T. "Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer". Nature Medicine, 25(7), 1054–1056, 2019.
Lafarge, M. W., Pluim, J. P. W., Eppenhof, K. A. J., Moeskops, P., & Veta, M. "Domain-adversarial neural networks to address the appearance variability of histopathology images". Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 10553 LNCS, 83–91, 2017.
Le, D. T., Durham, J. N., Smith, K. N., Wang, H., Bartlett, B. R., Aulakh, L. K., … Diaz, L. A. "Mismatch repair deficiency predicts response of solid tumors to PD-1 blockade". Science, 357(6349), 409–413, 2017.
Litjens, G., Sánchez, C. I., Timofeeva, N., Hermsen, M., Nagtegaal, I., Kovacs, I., … Van Der Laak, J. "Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis". Scientific Reports, 6(January), 1–11, 2016.
Liu, Y., Sethi, N. S., Hinoue, T., Schneider, B. G., Cherniack, A. D., Sanchez-Vega, F., … Laird, P. W. "Comparative Molecular Analysis of Gastrointestinal Adenocarcinomas". Cancer Cell, 33(4), 721-735.e8, 2018.
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. "Data-efficient and weakly supervised computational pathology on whole-slide images". Nature Biomedical Engineering, 1–35, 2021.
Naik, N., Madani, A., Esteva, A., Keskar, N. S., Press, M. F., Ruderman, D., … Socher, R. "Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E stains". Nature Communications, 11(1), 2020.
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., … Fei-Fei, L. "ImageNet Large Scale Visual Recognition Challenge". International Journal of Computer Vision, 115(3), 211–252, 2015.
Shia, J., Schultz, N., Kuk, D., Vakiani, E., Middha, S., Segal, N. H., … Klimstra, D. S. "Morphological characterization of colorectal cancers in The Cancer Genome Atlas reveals distinct morphology-molecular associations: clinical and biological implications". Modern Pathology : An Official Journal of the United States and Canadian Academy of Pathology, Inc, 30(4), 599–609, 2017.
Srinidhi, C. L., Ciga, O., & Martel, A. L. "Deep neural network models for computational histopathology: A survey". Medical Image Analysis, 67, 2021.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … Polosukhin, I. "Attention is all you need". Advances in Neural Information Processing Systems, 2017-Decem(Nips), 5999–6009, 2017.
Wang, H., Liang, L., Fang, J.-Y., & Xu, J. "Somatic gene copy number alterations in colorectal cancer: new quest for cancer drivers and biomarkers". Oncogene, 35(16), 2011–2019, 2016.
Wei, J. W., Tafe, L. J., Linnik, Y. A., Vaickus, L. J., Tomita, N., & Hassanpour, S. "Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks". Scientific Reports, 9(1), 1–8, 2019.
Yao, J., Zhu, X., Jonnagaddala, J., Hawkins, N., & Huang, J. "Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks". Medical Image Analysis, 65, 2020.