洪水災害往往造成大量的生命和財產損失，可說是最具破壞性的自然災害之一，因此準確的集水區降雨逕流模擬一直都是水文分析的長期挑戰，降雨逕流是一段非線性的時序列過程，在數值驅動方法中，已經有許多研究證明將深度學習模型諸如長短期記憶(LSTM)、卷積神經網路(CNN)與Transformer等應用於河道水位預測可以取得良好的表現。本研究採用結合CNN與Transformer技術的深度學習架構Conformer對降雨逕流模型建模，該模型同時以自注意力機制與卷積運算補捉多個測站的水位、降雨及氣象特徵，並且將這些特徵聚合後輸出水位序列。本研究將此模型應用於蘭陽溪流域並預測蘭陽大橋水位站未來7天的水位以驗證其在降雨逕流建模的可行性和效能，在消融實驗的結果中顯示了卷積運算能夠幫助模型捕捉水位與其他參數間的局部關係，且於自注意力運算後執行卷積運算的效果更佳。在與其它模型模擬結果的比較中，Conformer模型在NSE與R2指標的表現皆顯著優於CNN、LSTM與傳統的Transformer模型，證明了其在水文領域中取代過去常用的深度學習方法的潛力，期望藉由本研究能夠擴大深度學習在水文科學中的應用，為水文分析和水資源管理等領域提供更多樣化的解決方案。

Flood disasters often result in significant losses of life and property, making them among the most devastating natural hazards. As such, reliable and accurate water level forecasting is of vital importance. Rainfall-runoff modeling is a complex and nonlinear time series process. In data-driven methods, numerous studies have demonstrated their promising performance in water level prediction by using deep learning approaches such as Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) and Transformer. This study introduces the Conformer, a novel deep learning architecture that combines the strengths of CNN and Transformer technologies for rainfall-runoff modelling. The framework employs self-attention mechanisms coupled with convolutional computations to extract water level, precipitation, and meteorological features from multiple stations, then aggregates these features to output the subsequent water level series. In this study, the proposed model is applied to the Lanyang Stream basin to assess its effectiveness and feasibility in rainfall-runoff modelling by predicting the 7-day-ahead water levels. Results from the ablation experiments indicate that convolutional computations substantially enhance the model's capacity to capture local relationships between the water level and other parameters. Furthermore, performing convolution computations after self-attention operations yield even better results. When compared with simulations from other models, the proposed model significantly outperforms CNN, LSTM and traditional Transformer models in terms of the Coefficient of determination (R2) and the Nash–Sutcliffe Efficiency (NSE) indicators, demonstrating the Conformer model’s potential to replace the commonly used deep learning methods in the field of hydrology.

[1] Abbott, M. B., et al. (1986). "An introduction to the European Hydrological System—Systeme Hydrologique Europeen,“SHE”, 1: History and philosophy of a physically-based, distributed modelling system." Journal of Hydrology 87(1-2): 45-59.
[2] Anctil, F., et al. (2004). "Impact of the length of observed records on the performance of ANN and of conceptual parsimonious rainfall-runoff forecasting models." Environmental Modelling & Software 19(4): 357-368.
[3] Bahdanau, D., et al. (2014). "Neural machine translation by jointly learning to align and translate." arXiv preprint arXiv:1409.0473.
[4] Bathurst, J. and P. O'connell (1992). "Future of distributed modelling: the Système Hydrologique Européen." Hydrological processes 6(3): 265-277.
[5] Beven, K. J., et al. (1984). "Testing a physically-based flood forecasting model (TOPMODEL) for three UK catchments." Journal of Hydrology 69(1-4): 119-143.
[6] Castangia, M., et al. (2023). "Transformer neural networks for interpretable flood forecasting." Environmental Modelling & Software 160: 105581.
[7] Chang, L. C., et al. (2004). "A two‐step‐ahead recurrent neural network for stream‐flow forecasting." Hydrological processes 18(1): 81-92.
[8] Cho, K., et al. (2014). "Learning phrase representations using RNN encoder-decoder for statistical machine translation." arXiv preprint arXiv:1406.1078.
[9] Devia, G. K., et al. (2015). "A review on hydrological models." Aquatic procedia 4: 1001-1007.
[10] Duchi, J., et al. (2011). "Adaptive subgradient methods for online learning and stochastic optimization." Journal of machine learning research 12(7).
[11] Dumoulin, V. and F. Visin (2016). "A guide to convolution arithmetic for deep learning." arXiv preprint arXiv:1603.07285.
[12] Fidal, J. and T. Kjeldsen (2020). "Accounting for soil moisture in rainfall-runoff modelling of urban areas." Journal of Hydrology 589: 125122.
[13] Gao, S., et al. (2020). "Short-term runoff prediction with GRU and LSTM networks without requiring time step optimization during sample generation." Journal of Hydrology 589: 125188.
[14] Gulati, A., et al. (2020). "Conformer: Convolution-augmented transformer for speech recognition." arXiv preprint arXiv:2005.08100.
[15] Hochreiter, S. and J. Schmidhuber (1997). "Long short-term memory." Neural computation 9(8): 1735-1780.
[16] Howard, A. G., et al. (2017). "Mobilenets: Efficient convolutional neural networks for mobile vision applications." arXiv preprint arXiv:1704.04861.
[17] Ismail Fawaz, H., et al. (2019). "Deep learning for time series classification: a review." Data mining and knowledge discovery 33(4): 917-963.
[18] Ji, S., et al. (2012). "3D convolutional neural networks for human action recognition." IEEE transactions on pattern analysis and machine intelligence 35(1): 221-231.
[19] Kingma, D. P. and J. Ba (2014). "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980.
[20] Kohonen, T. (1988). "An introduction to neural computing." Neural networks 1(1): 3-16.
[21] Kratzert, F., et al. (2018). "Rainfall–runoff modelling using Long Short-Term Memory (LSTM) networks." Hydrology and Earth System Sciences 22(11): 6005-6022.
[22] LeCun, Y., et al. (1998). "Gradient-based learning applied to document recognition." Proceedings of the IEEE 86(11): 2278-2324.
[23] Legates, D. R. and G. J. McCabe Jr (1999). "Evaluating the use of “goodness‐of‐fit” measures in hydrologic and hydroclimatic model validation." Water resources research 35(1): 233-241.
[24] Liu, W., et al. (2020). "Accelerating federated learning via momentum gradient descent." IEEE Transactions on Parallel and Distributed Systems 31(8): 1754-1766.
[25] Liu, Y., et al. (2022). "Directed graph deep neural network for multi-step daily streamflow forecasting." Journal of Hydrology 607: 127515.
[26] Loshchilov, I. and F. Hutter (2017). "Decoupled weight decay regularization." arXiv preprint arXiv:1711.05101.
[27] McCulloch, W. S. and W. Pitts (1943). "A logical calculus of the ideas immanent in nervous activity." The bulletin of mathematical biophysics 5: 115-133.
[28] Montanari, A., et al. (1997). "Fractionally differenced ARIMA models applied to hydrologic time series: Identification, estimation, and simulation." Water resources research 33(5): 1035-1044.
[29] Moradkhani, H. and S. Sorooshian (2008). "General review of rainfall-runoff modeling: model calibration, data assimilation, and uncertainty analysis." Hydrological modelling and the water cycle: Coupling the atmospheric and hydrological models: 1-24.
[30] Rajurkar, M., et al. (2004). "Modeling of the daily rainfall-runoff relationship with artificial neural network." Journal of Hydrology 285(1-4): 96-113.
[31] Ramachandran, P., et al. (2017). "Searching for activation functions." arXiv preprint arXiv:1710.05941.
[32] Rogers, C., et al. (1985). "Sensitivity analysis, calibration and predictive uncertainty of the Institute of Hydrology Distributed Model." Journal of Hydrology 81(1-2): 179-191.
[33] Rosenblatt, F. (1958). "The perceptron: a probabilistic model for information storage and organization in the brain." Psychological review 65(6): 386.
[34] Rumelhart, D. E., et al. (1985). Learning internal representations by error propagation, California Univ San Diego La Jolla Inst for Cognitive Science.
[35] Salas, J. D., et al. (1985). "Approaches to multivariate modeling of water resources time series 1." JAWRA Journal of the American Water Resources Association 21(4): 683-708.
[36] Todini, E. (1996). "The ARNO rainfall—runoff model." Journal of Hydrology 175(1-4): 339-382.
[37] Van, S. P., et al. (2020). "Deep learning convolutional neural network in rainfall–runoff modelling." Journal of Hydroinformatics 22(3): 541-561.
[38] Vaswani, A., et al. (2017). "Attention is all you need." Advances in neural information processing systems 30.
[39] Wang, Q., et al. (2019). "Learning deep transformer models for machine translation." arXiv preprint arXiv:1906.01787.
[40] Xiang, Z., et al. (2020). "A rainfall‐runoff model with LSTM‐based sequence‐to‐sequence learning." Water resources research 56(1): e2019WR025326.
[41] Yang, T., et al. (2017). "Developing reservoir monthly inflow forecasts using artificial intelligence and climate phenomenon information." Water resources research 53(4): 2786-2812.
[42] Yin, H., et al. (2022). "RR-Former: Rainfall-runoff modeling based on Transformer." Journal of Hydrology 609: 127781.
[43] Yin, H., et al. (2021). "Rainfall-runoff modeling using LSTM-based multi-state-vector sequence-to-sequence model." Journal of Hydrology 598: 126378.
[44] Yoosefdoost, I., et al. (2022). Hydrological Models. Climate Change in Sustainable Water Resources Management, Springer: 283-329.
[45] Yousfi, S., et al. (2017). "Contribution of recurrent connectionist language models in improving LSTM-based Arabic text recognition in videos." Pattern Recognition 64: 245-254.
[46] Zhang, X., et al. (2015). "Are hybrid models integrated with data preprocessing techniques suitable for monthly streamflow forecasting? Some experiment evidences." Journal of Hydrology 530: 137-152.
[47] Zhou, H., et al. (2021). Informer: Beyond efficient transformer for long sequence time-series forecasting. Proceedings of the AAAI conference on artificial intelligence.