庫存現金管理為銀行成本控制、風險管理、客戶服務管理的一大重點。現行多數銀行庫存現金管理規劃大多以人工憑藉經驗值訂定庫存現金限額去控管。隨著商業模式的多元化以及金融科技的進步，各式交易付款、收付形式愈來愈多樣，人工經驗值判斷已無法承擔需大量資料運算的現金庫存預測需求。而導入人工智慧(Artificial Intelligence, AI)及商業智慧(Business Intelligence, BI)技術，能協助銀行分析大量金融資料、預測未來金流變動，並從預測結果來輔佐現金運補策略的制定。
本研究以國內一家中型商業銀行做為研究對象，應用處方式分析學於金融科技現金運補規劃，並設計兩階段模型輔助銀行經理人作出適當的現金運補決策。本研究先實行預測性分析，並使用長短期記憶網路(Long Short-Term Memory, LSTM)技術，建置多變量時間序列深度學習預測模型，預測各分行未來金流變動量。得到預測結果後，再實行處方式分析，利用三種深度強化學習框架，分別為深度Q網路(Deep Q-learning Network, DQN)、策略梯度(Policy Gradient)及Actor-Critic，找出最佳現金運補策略。從實驗結果發現，本研究之兩階段模型LSTM-RL相較於其他現金管理模型，包含Miller-Orr 模型、LSTM-LP 模型，在成本節省比率及調度次數上皆有更佳的表現。其中，深度Q網路、策略梯度訓練速度快且有最穩定的表現；Actor Critic 雖通常能節省最多成本比率，但於少數情況下則需面對緊急叫鈔的高機率風險性。此外，本研究之模型可依經理人對風險偏好的不同程度，進而找出最適應該風險情境下的最佳現金運補決策。

Cash management has become a major issue in banking owing to the diversity of transactions and the substantially rising of customer demand. The bank's target is to ensure a sufficient level of liquidity to meet customer needs and invest surplus cash to make profits. This research presents an approach to cash management for a financial institution using prescriptive analytics.
We followed a two-phase model, proposing a decision support system aimed at minimizing cash-related cost in each branch. First, we built Long Short-Term Memory model to forecast daily cash flow. Secondly, we used deep reinforcement learning (RL) framework, including Deep Q-learning Network (DQN), Policy Gradient (PG) and Actor-Critic (AC) to find the optimal cash replenishment policy with the prior result. In experimental phase, a real-world transaction dataset from a domestic commercial bank was used. We formulated an objective function with risk preference for evaluating the performance of policy. From results, the policies obtained from our proposed model has a better performance than other methods. In particular, AC is the RL method with the highest savings performance, while PG and DQN have higher stability and training speed. For managerial implication, our research can apply to different scenarios by tuning risk and holding cost index. It means the policies based on our proposed framework can be optimized according to the particular risk preferences of cash managers.

Abbasimehr, H., Shabani, M., & Yousefi, M. (2020). An optimized model using LSTM network for demand forecasting. Computers & Industrial Engineering, 143, 106435. Retrieved from http://www.sciencedirect.com/science/article/pii/S0360835220301698 doi:https://doi.org/10.1016/j.cie.2020.106435
Acuña, G., Ramirez, C., & Curilem, M. (2012). Comparing NARX and NARMAX models using ANN and SVM for cash demand forecasting for ATM. Paper presented at the The 2012 International Joint Conference on Neural Networks (IJCNN).
Akerkar, R. (2013). Advanced data analytics for business. Big data computing, 377(9).
Armenise, R., Birtolo, C., Sangianantoni, E., & Troiano, L. (2010). A generative solution for ATM cash management. Paper presented at the 2010 International Conference of Soft Computing and Pattern Recognition.
Audzeyeva, A., & Hudson, R. (2016). How to get the most from a business intelligence application during the post implementation phase? Deep structure transformation at a UK retail bank. European Journal of Information Systems, 25(1), 29-46.
Batrinca, B., & Treleaven, P. C. (2015). Social media analytics: a survey of techniques, tools and platforms. AI & SOCIETY, 30(1), 89-116. Retrieved from https://doi.org/10.1007/s00146-014-0549-4. doi:10.1007/s00146-014-0549-4
Berk, L., Bertsimas, D., Weinstein, A. M., & Yan, J. (2019). Prescriptive analytics for human resource planning in the professional services industry. European Journal of Operational Research, 272(2), 636-641.
Bertsimas, D., & Kallus, N. (2020). From predictive to prescriptive analytics. Management Science, 66(3), 1025-1044.
Bertsimas, D., & Parys, B.V. (2018). Bootstrap Robust Prescriptive Analytics. arXiv: Optimization and Control.
Birge, J. R., & Louveaux, F. (2011). Introduction to stochastic programming. Springer Science & Business Media.
Cabrero, A., Camba‐Mendez, G., Hirsch, A., & Nieto, F. (2009). Modelling the daily banknotes in circulation in the context of the liquidity management of the European Central Bank. Journal of Forecasting, 28(3), 194-217.
Cardona, L., & Moreno, L. A. (2012). Cash management cost reduction using data mining to forecast cash demand and LP to optimize resources. Memetic Computing, 4(2), 127-134.
Cheng, M.-Y., & Roy, A. F. V. (2011). Evolutionary fuzzy decision model for cash flow prediction using time-dependent support vector machines. International Journal of Project Management, 29(1), 56-65. Retrieved from http://www.sciencedirect.com/science/article/pii/S0263786310000062 doi:https://doi.org/10.1016/j.ijproman.2010.01.004
Chen-xu, N., & Jie-sheng, W. (2015, July). Auto regressive moving average (ARMA) prediction method of bank cash flow time series. Paper presented at the 2015 34th Chinese Control Conference (CCC).
Conboy, K., Mikalef, P., Dennehy, D., & Krogstie, J. (2020). Using business analytics to enhance dynamic capabilities in operations research: A case analysis and research agenda. European Journal of Operational Research, 281(3), 656-672. Retrieved from http://www.sciencedirect.com/science/article/pii/S0377221719305442. doi:https://doi.org/10.1016/j.ejor.2019.06.051
da Costa Moraes, M. B., Nagano, M. S., & Sobreiro, V. A. (2015). Stochastic Cash Flow Management Models: A Literature Review Since the 1980s. In P. Guarnieri (Ed.), Decision Models in Engineering and Management (pp. 11-28). New York: Springer.
Das, S. (2017). Corporate cash management: A study on retail sector. Accounting, 3, 23-40. doi:10.5267/j.ac.2016.5.003
Davenport, T. H. (2006). Competing on analytics. Harvard business review, 84(1), 98.
Delen, D., & Zolbanin, H. M. (2018). The analytics paradigm in business research. Journal of Business Research, 90, 186-195.
den Hertog, D., & Postek, K. (2016). Bridging the gap between predictive and prescriptive analytics-new optimization methodology needed. (Tilburg University, Netherlands) Retrieved from http://www.optimization-online.org/DB_HTML/2016/12/5779.html
Duan, Y., Cao, G., & Edwards, J. S. (2020). Understanding the impact of business analytics on innovation. European Journal of Operational Research, 281(3), 673-686.
Ferstl, R., & Weissensteiner, A. (2010). Cash management using multi-stage stochastic programming. Quantitative Finance, 10(2), 209-219. Retrieved from https://doi.org/10.1080/14697680802637908. doi:10.1080/14697680802637908
Fischer, T., & Krauss, C. (2018). Deep learning with long short-term memory networks for financial market predictions. European Journal of Operational Research, 270(2), 654-669.
Frazzetto, D., Nielsen, T. D., Pedersen, T. B., & Šikšnys, L. (2019). Prescriptive analytics: a survey of emerging trends and technologies. The VLDB Journal, 28(4), 575-595. Retrieved from https://doi.org/10.1007/s00778-019-00539-y doi:10.1007/s00778-019-00539-y
Gardner, J. R., Kusner, M. J., Xu, Z. E., Weinberger, K. Q., & Cunningham, J. P. (2014). Bayesian Optimization with Inequality Constraints. Paper presented at the ICML.
Goyal, A., Aprilia, E., Janssen, G., Kim, Y., Kumar, T., Mueller, R., . . . Xiong, J. (2016). Asset health management using predictive and prescriptive analytics for the electric power grid. IBM Journal of Research and Development, 60(1), 4: 1-4: 14.
Grossberg, S., & Merill, J. W. (1992). A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. Cognitive Brain Research, 1(1), 3-38. doi:10.1016/0926-6410(92)90003-A
Grossberg, S., & Merrill, J. W. (1992). A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. Cognitive brain research, 1(1), 3-38.
Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
Hwang, K.-S., Lin, C.-J., Wu, C.-J., & Lo, C.-Y. (2007). Cooperation Between Multiple Agents Based on Partially Sharing Policy. In: Huang DS., Heutte L., Loog M. (eds) Advanced Intelligent Computing Theories and Applications. With Aspects of Theoretical and Methodological Issues. ICIC 2007. Lecture Notes in Computer Science, vol 4681. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74171-8_42
Krumeich, J., Werth, D., & Loos, P. (2016). Prescriptive control of business processes. Business & Information Systems Engineering, 58(4), 261-280.
López Lázaro, J., Barbero Jiménez, Á., & Takeda, A. (2018). Improving cash logistics in bank branches by coupling machine learning and robust optimization. Expert Systems with Applications, 92, 236-255. Retrieved from http://www.sciencedirect.com/science/article/pii/S0957417417306474 doi:https://doi.org/10.1016/j.eswa.2017.09.043
Lepenioti, K., Bousdekis, A., Apostolou, D., & Mentzas, G. (2020). Prescriptive analytics: Literature review and research challenges. International Journal of Information Management, 50, 57-70. Retrieved from http://www.sciencedirect.com/science/article/pii/S0268401218309873 doi:https://doi.org/10.1016/j.ijinfomgt.2019.04.003
Li, Y. (2017). Deep reinforcement learning: An overview. arXiv preprint arXiv:1701.07274.
Lilian,W (2018, April 8).Policy Gradient Algorithms. Retrieved November 2, 2020, from https://lilianweng.github.io/lil-log/2018/04/08/policy-gradient-algorithms.html#actor-critic
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., . . . Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971. Retrieved from https://arxiv.org/abs/1509.02971
Lin, C.-T., & Jou, C.-P. (1999). Controlling chaos by GA-based reinforcement learning neural network. IEEE Transactions on Neural Networks, 10(4), 846-859.
Lin, L.-J. (1992). Self-improving reactive agents based on reinforcement learning, planning and teaching. Machine learning, 8(3-4), 293-321.
Miller, M. H., & Orr, D. (1966). A Model of the Demand for Money by Firms. The Quarterly Journal of Economics, 80(3), 413-435.
Mishra, N., & Silakari, S. (2012). Predictive analytics: A survey, trends, applications, oppurtunities & challenges. International Journal of Computer Science and Information Technologies, 3(3), 4434-4438.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., . . . Ostrovski, G. (2015). Human-level control through deep reinforcement learning. nature, 518(7540), 529-533.
Moro, S., Cortez, P., & Rita, P. (2015). Business intelligence in banking: A literature analysis from 2002 to 2013 using text mining and latent Dirichlet allocation. Expert Systems with Applications, 42(3), 1314-1324. Retrieved from http://www.sciencedirect.com/science/article/pii/S0957417414005636. doi:https://doi.org/10.1016/j.eswa.2014.09.024
Mortenson, M. J., Doherty, N. F., & Robinson, S. (2015). Operational research from Taylorism to Terabytes: A research agenda for the analytics age. European Journal of Operational Research, 241(3), 583-595. Retrieved from http://www.sciencedirect.com/science/article/pii/S037722171400664X
Osorio, A. F., & Toro, H. (2012). An MIP model to optimize a Colombian cash supply chain. International Transactions in Operational Research, 19(5), 659-673.
Padmanaban, R., & Mukesh, R. (2020). HadoopSec: Sensitivity-aware Secure Data Placement Strategy for Big Data/Hadoop Platform using Prescriptive Analytics. GSTF Journal on Computing (JoC), 6(1).
Ramanathan, R., Mathirajan, M., & Ravindran, A. R. (2017). Big data analytics using multiple criteria decision-making models.Boca Raton: CRC Press. Retrieved from https://doi.org/10.1201/9781315152653
Ramírez, C., & Acuña, G. (2011). Forecasting Cash Demand in ATM Using Neural Networks and Least Square Support Vector Machine. In C. S. Martin & S. W. Kim (Eds.), Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications (pp. 515-522). Springer.
Reddy, B. K., & Delen, D. (2018). Predicting hospital readmission for lupus patients: An RNN-LSTM-based deep-learning methodology. Computers in biology and medicine, 101, 199-209.
Rundo, F. (2019). Deep LSTM with Reinforcement Learning Layer for Financial Trend Prediction in FX High Frequency Trading Systems. Applied Sciences, 9(20), 4460.
Shroff, G., Agarwal, P., Singh, K., Kazmi, A. H., Shah, S., & Sardeshmukh, A. (2014, 7-10 July 2014). Prescriptive information fusion. Paper presented at the 17th International Conference on Information Fusion (FUSION).
Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., & Riedmiller, M. (2014). Deterministic policy gradient algorithms. Paper presented at the Proceedings of the 31st International Conference on International Conference on Machine Learning, Bejing, China.
Simutis, R., Dilijonas, D., Bastina, L., Friman, J., & Drobinov, P. (2007). OPTIMIZATION OF CASH MANAGEMENT FOR ATM NETWORK. Information Technology and Control, 36.
Soltanpoor, R., & Sellis, T. (2016). Prescriptive Analytics for Big Data, Cham.
Srinivasan, V., & Kim, Y. H. (1986). Deterministic cash flow management: State of the art and research directions. Omega, 14(2), 145-166. Retrieved from http://www.sciencedirect.com/science/article/pii/0305048386900174. doi:https://doi.org/10.1016/0305-0483(86)90017-4
Sun, G.-D., Wu, Y.-C., Liang, R.-H., & Liu, S.-X. (2013). A survey of visual analytics techniques and applications: State-of-the-art research and future challenges. Journal of Computer Science and Technology, 28(5), 852-867.
Sutton, R. S., McAllester, D. A., Singh, S. P., & Mansour, Y. (2000). Policy gradient methods for reinforcement learning with function approximation. Paper presented at the Advances in neural information processing systems.
Tsai, C.-W., Lai, C.-F., Chao, H.-C., & Vasilakos, A. V. (2015). Big data analytics: a survey. Journal of Big Data, 2(1), 21. Retrieved from https://doi.org/10.1186/s40537-015-0030-3 doi:10.1186/s40537-015-0030-3
Tsitsiklis, J. N., & Van Roy, B. (1997). Analysis of temporal-diffference learning with function approximation. IEEE Transactions on Automatic Control, 42(5), 674-690. doi:10.1109/9.580874
Wang, Z. (2018). The Dynamic Analysis of the Cash Flows on ATM. Journal of Computer and Communications, 6(04), 38.
Weytjens, H., Lohmann, E., & Kleinsteuber, M. (2019). Cash flow prediction: MLP and LSTM compared to ARIMA and Prophet. Electronic Commerce Research,1-21. doi:10.1007/s10660-019-09362-7
Zapranis, A., & Alexandridis, A. (2009). Forecasting Cash Money WithdrawalsUsing Wavelet Analysis and Wavelet Neural Networks. International Journal of Financial Economics and Econometrics.ISSN 0975-2072.