小分子核糖核酸(MicroRNA)是生物體內一種微小的非編碼核糖核酸，透過與信使核糖核酸的結合來抑制轉譯的進行。此機制已經被證實與一些癌症和重大疾病相關，因此小分子核糖核酸與信使核糖核酸的結合機制在生醫領域中是很重要的研究課題。隨著近年來深度學習的蓬勃發展，越來越多研究者將深度學習應用於小分子核糖核酸與信使核醣核酸的結合預測。相較於傳統機器學習採用的決策樹(Decision Tree)或是支持向量機(Support Vector Machine)等技術，深度學習的好處是不需經由人工設計特徵，可以直接以原始的核糖核酸序列資料作為輸入，讓深度學習模型自行從序列中萃取出有用的特徵，取代人工設計的過程。人工設計特徵的優勢是，在設計時會考慮小分子核糖核酸與信使核糖核酸兩端結合的資訊，如序列互補性與結合位點的自由能等。但由於深度學習是直接以原始序列作為輸入，因此如何讓模型學習小分子核糖核酸與信使核糖核酸的結合資訊，是本研究主要探討的課題。
本研究提出了兩種小分子核糖核酸與信使核醣核酸的結合預測模型，其中主要差異是模型如何結合兩序列的資訊，其一是在特徵層級連接兩序列，稱之為嵌入連接網路(Embedding Concatenation Network, ECN)；其二是使用交互注意力機制融和兩序列，稱之為交互注意力機制網路(Cross-Attention Network, CAN)。相較於過往使用深度學習的研究，本研究所提出的兩個模型皆取得更好的F度量分數以及準確率，其中又以嵌入連接網路的表現更好。為了更深入探討模型的可解釋性，本研究將這兩個模型進行視覺化，發現交互注意力機制網路的視覺化結果與種子區域的序列互補性高度相關，未來可以運用至結合位點偵測。

MicroRNAs are small non-encoding RNAs that suppress gene expressions by binding to their target mRNAs. This suppression mechanism is confirmed to be related to some cancers and severe diseases. Therefore, studying the binding mechanism of microRNAs and mRNAs is an important topic in systems biology. With the development of deep learning in recent years, more and more studies have adopted deep learning methods in microRNA target prediction. In comparison with conventional machine learning methods like Decision Tree or Support Vector Machine (SVM), the advantage of deep learning is that it requires no hand-crafted features. Namely, deep learning models can extract meaningful features from raw data (i.e., biological sequences in the context). The advantage of the hand-crafted features is that most of them have already considered the information of microRNAs and mRNAs at the same time (i.e., complementarity and the free energy of binding sites). But since deep learning takes the biological sequences as input, how to make the model learn the combination information of microRNAs and mRNAs is the key to adopting deep learning methods in microRNA target prediction.
This work proposes two neural networks, the main difference of them is how the model combines the information of microRNAs and mRNAs. One is Embedding Concatenation Network (ECN), which combines the information of two sequences at embedding level. The other is Cross-Attention Network (CAN), which combines the information of two sequences through cross-attention mechanism. In comparison with recent deep learning studies, the proposed networks achieve the best F1 score and accuracy, and ECN achieve the state-of-the-art performance. Moreover, the two models are visualized to explore their interpretability. The visualization of CAN is highly related to the complementary base pairs of the seed region, which can be applied to binding site detection in the future.

[1] Lewis BP, Burge CB, Bartel DP: Conserved seed pairing, often flanked by adenosines, indicates that thousands of human genes are microRNA targets. cell 2005, 120(1):15-20.
[2] Ardekani AM, Naeini MM: The role of microRNAs in human diseases. Avicenna journal of medical biotechnology 2010, 2(4):161.
[3] Li Y, Kowdley KV: MicroRNAs in common human diseases. Genomics, proteomics & bioinformatics 2012, 10(5):246-253.
[4] Brennecke J, Stark A, Russell RB, Cohen SM: Principles of microRNA–target recognition. PLoS biology 2005, 3(3):e85.
[5] Seok H, Ham J, Jang E-S, Chi SW: MicroRNA target recognition: insights from transcriptome-wide non-canonical interactions. Molecules and cells 2016, 39(5):375.
[6] Agarwal V, Bell GW, Nam J-W, Bartel DP: Predicting effective microRNA target sites in mammalian mRNAs. elife 2015, 4:e05005.
[7] Menor M, Ching T, Zhu X, Garmire D, Garmire LX: mirMark: a site-level and UTR-level classifier for miRNA target prediction. Genome biology 2014, 15(10):500.
[8] Bandyopadhyay S, Mitra R: TargetMiner: microRNA target prediction with systematic identification of tissue-specific negative examples. Bioinformatics 2009, 25(20):2625-2631.
[9] Schirle NT, Sheu-Gruttadauria J, MacRae IJ: Structural basis for microRNA targeting. Science 2014, 346(6209):608-613.
[10] Moore MJ, Scheel TK, Luna JM, Park CY, Fak JJ, Nishiuchi E, Rice CM, Darnell RB: miRNA–target chimeras reveal miRNA 3′-end pairing as a major determinant of Argonaute target specificity. Nature communications 2015, 6:8864.
[11] Broughton JP, Lovci MT, Huang JL, Yeo GW, Pasquinelli AE: Pairing beyond the seed supports microRNA targeting specificity. Molecular cell 2016, 64(2):320-333.
[12] Min S, Lee B, Yoon S: Deep learning in bioinformatics. Briefings in bioinformatics 2017, 18(5):851-869.
[13] Quinlan JR: Induction of decision trees. Machine learning 1986, 1(1):81-106.
[14] Cortes C, Vapnik V: Support-vector networks. Machine learning 1995, 20(3):273-297.
[15] Lee B, Baek J, Park S, Yoon S: deepTarget: end-to-end learning framework for microRNA target prediction using deep recurrent neural networks. In: Proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics: 2016. ACM: 434-442.
[16] Pla A, Zhong X, Rayner S: miRAW: A deep learning-based approach to predict microRNA targets by analyzing whole microRNA transcripts. PLoS computational biology 2018, 14(7):e1006185.
[17] ZHENG, Xueming, et al., Prediction of miRNA targets by learning from interaction sequences. Plos one, 2020, 15.5: e0232578.
[18] GU, Tongjun, et al., miTAR: a hybrid deep learning-based approach for predicting miRNA targets. BMC bioinformatics, 2021, 22.1: 1-16.
[19] MIN, Seonwoo; LEE, Byunghan; YOON, Sungroh. TargetNet: functional microRNA target prediction with deep neural networks. Bioinformatics, 2022, 38.3: 671-677.
[20] YANG, Tzu-Hsien, et al., Identifying piRNA targets on mRNAs in C. elegans using a deep multi-head attention network. BMC bioinformatics, 2021, 22.1: 1-23.
[21] KARAGKOUNI, Dimitra, et al., DIANA-TarBase v8: a decade-long collection of experimentally supported miRNA–gene interactions. Nucleic acids research, 2018, 46.D1: D239-D245.
[22] IORIO, Marilena V.; CROCE, Carlo M. Causes and consequences of microRNA dysregulation. Cancer journal(Sudbury, Mass.), 2012, 18.3: 215.
[23] John B, Enright AJ, Aravin A, Tuschl T, Sander C, Marks DS: Human microRNA targets. PLoS biology 2004, 2(11):e363.
[24] Kertesz M, Iovino N, Unnerstall U, Gaul U, Segal E: The role of site accessibility in microRNA target recognition. Nature genetics 2007, 39(10):1278.
[25] Lorenz R, Bernhart SH, Zu Siederdissen CH, Tafer H, Flamm C, Stadler PF, Hofacker IL: ViennaRNA Package 2.0. Algorithms for Molecular Biology 2011, 6(1):26.
[26] Grosswendt S, Filipchyk A, Manzano M, Klironomos F, Schilling M, Herzog M, Gottwein E, Rajewsky N: Unambiguous identification of miRNA: target site interactions by different types of ligation reactions. Molecular cell 2014, 54(6):1042-1054.
[27] Helwak A, Kudla G, Dudnakova T, Tollervey D: Mapping the human miRNA interactome by CLASH reveals frequent noncanonical binding. Cell 2013, 153(3):654-665.
[28] Paraskevopoulou MD, Georgakilas G, Kostoulas N, Vlachos IS, Vergoulis T, Reczko M, Filippidis C, Dalamagas T, Hatzigeorgiou AG: DIANA-microT web server v5. 0: service integration into miRNA functional analysis workflows. Nucleic acids research 2013, 41(W1):W169-W173.
[29] HE, Kaiming, et al., Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016. p. 770-778.
[30] Lin M, Chen Q, Yan S: Network in network. arXiv preprint arXiv:13124400 2013.
[31] VASWANI, Ashish, et al., Attention is all you need. Advances in neural information processing systems, 2017, 30.
[32] HUANG, Hsi-Yuan, et al., miRTarBase update 2022: an informative resource for experimentally validated miRNA–target interactions. Nucleic acids research, 2022, 50.D1: D222-D230.
[33] Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P: Natural language processing(almost)from scratch. Journal of Machine Learning Research 2011, 12(Aug):2493-2537.
[34] Mikolov T, Chen K, Corrado G, Dean J: Efficient estimation of word representations in vector space. arXiv preprint arXiv:13013781 2013.
[35] Johnson R, Zhang T: Effective use of word order for text categorization with convolutional neural networks. arXiv preprint arXiv:14121058 2014.
[36] HU, Jie; SHEN, Li; SUN, Gang.: Squeeze-and-excitation networks. Proceedings of the IEEE conference on computer vision and pattern recognition. 2018. p. 7132-7141.
[37] DEVLIN, Jacob, et al.,: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
[38] COCK, Peter JA, et al.,: Biopython: freely available Python tools for computational molecular biology and bioinformatics. Bioinformatics, 2009, 25.11: 1422-1423.