一個可靠又精確的價格預測方法通常是難以企及的，而投資者又往往會受到各種因素的影響，例如感情、歷史紀錄、偏見，又或是難以控制的大事件等。因此投資者需要機器學習演算法，將不必要的情緒因素排除，並透過演算法建立一個單純、精確又可靠的策略使用。本論文著重在機器學習的強化學習領域來探討機器學習於今日投資領域上的不可或缺，本文使用Q學習，一種無模型的深度強化學習方法，憑之建立一台交易機器人在指數及股票市場上進行操作。機器人在與歷史環境互動之下不斷學習並持續進行買入及賣出操作，最後以期間內獲得的總報酬多寡來審視自己並加以改進。操作的種類範圍為四種與生物科技相關的金融工具及S&P500來佐證此機器人在多領域之下的表現；時間範圍依照新型冠狀病毒(Covid-19)前後劃分為2016~2019以及2020~2021兩種時段，用以表現出機器人在劇烈地交易環境變動之下會有何種表現。本文更以機器人的表現對比指數平滑異同移動平均線(MACD)和買入持有策略(BH)，發現Q學習明顯比其他兩種方法都優越，在納斯達克生科指數和莫德納的幾何平均報酬之下尤其顯著。

A reliable and precise method of price prediction is always hard to achieve, and it is what people have been strived for. Investors are affected by kinds of factors such as emotions, history records, prejudices, uncontrollable incidents. Therefore, investors need machine learning (ML) algorithms to exclude emotion impacts and calculate all factors to help them set up a simple yet accurate strategy which investors can rely on. This article sheds a light on the deep reinforcement learning (Abbreviation as DRL) to demonstrate the importance of ML algorithms. We focus on the model-free DRL branch (Q-Learning to be specific) with a build-up moving average convergence/divergence (MACD) algorithm from “Backtrader”, a Python computer language framework used for trading purposes, and the basic Buy-and-Hold (BH) strategy for comparison. DRL is expected to be sensitive to market dynamics while swiftly adapts to the changes, and is expected to have better results than the two strategies mentioned above. This article uses Q-Learning to maximize the rewards and compares two periods of “biotech instruments”, the first period is from 2016 to 2019, while the other is from 2020 to 2021, showing that Q-Learning is keen and quick enough to adapt drastic environment changes. We find out the results of Q-Learning is significantly better than other two strategies, especially for Nasdaq Biotech Index and Moderna stock in geometric mean percentages. Since DRL is growing nonstop, the better the technology the mellower the algorithms, hence the reason of using MLs on investing is even more sounded.

Adams, P. A., and Adams, J. K., 1960. Confidence in the Recognition and Reproduction of Words Difficult to Spell. The American Journal of Psychology, 73(4), 544-552.

Bao, D., and Yang, Z., 2008. Intelligent Stock Trading System by Turning Point Confirming and Probabilistic Reasoning. International Journal of Expert Systems with Applications, 34 (1), 620-627.

Baron, J., 2006. Chapter 2, The Study of Thinking. In: Thinking and Deciding, Cambridge Press; 1st Edition.

Das, T. K., Gosavi, A., Mahadevan, S., and Marchalleck, N., 1998. Solving Semi-Markov Decision Problems using Average Reward Reinforcement Learning. Management Science, 45(4), 560-574.

El-Baz, H. S., Lasfer, A., and Awadhi, I. A., 2013. SMA and MACD Combinations for Stock Investment Decisions in Frontier Markets: Evidence from Dubai Financial Market. International Journal of Financial Engineering and Risk Management, 1(2), 113-128

Friedman, J. H., 1998. Data Mining and Statistics: What's the connection? Working Paper.

Giamouridis, D., and Vrontos, I. D., 2007. Hedge Fund Portfolio Construction: A Comparison of Static and Dynamic Approaches. Journal of Banking & Finance, 31(1), 199-217

Grønager, C. S., and Vestergaard, K. J. V., Applying Machine Learning in Equity Trading. Master's Thesis, Copenhagen Business School.

Huang, B., Kim, Y. S., 2006. A Test of MACD Trading Strategy. Master's Thesis, Simon Fraser University.

Kahneman, D., and Tversky, A., 1979. Prospect Theory: An Analysis of Decision under Risk. Econometrica, 47(2), 263-292.

Kang, B. K., 2021. Improving MACD Technical Analysis by Optimizing Parameters and Modifying Trading Rules: Evidence from the Japanese Nikkei 225 Futures Market. Journal of Risk and Financial Management, 14(1), 37.

Kanwar, N., 2019. Deep Reinforcement Learning-based Portfolio Management. Master's Thesis, The University of Texas at Arlington.

Le, A. T., 2014. Dynamic Order Placement Strategies and Stock Market Quality: Further Evidence from a New Approach. Working Paper.

Markowitz, H., 1952. Portfolio Selection. The Journal of Finance, 7(1), 77-91.

Mikhniuk, A., 2020. The Impact of Covid-19 Related Restrictions on Returns Financial Stock Market Indices. Master's Thesis, Kyiv School of Economics.

Millea, A., 2021. Deep Reinforcement Learning for Trading—A Critical Survey. Data, 6(11), 119 (Data is an online open access journal and therefore the number 119 means its web URL).

Ororbia, M., and Warn, G., 2021. Design Synthesis Through a Markov Decision Process and Reinforcement Learning Framework. Journal of Computing and Information Science in Engineering, 22(2), 1-19.

Otterlo, M. V., and Wiering, M. A., 2012. Reinforcement Learning and Markov Decision Processes. Reinforcement Learning: State-of-the-Art, 1, 3-42.

Özdurak, C., Alcan, G., and Güvenbaş, S. D., 2020. The Impact of Covid-19 to Global Pharmaceuticals and Biotechnology Company Stocks Returns. Pressacademia, 9(2), 68-79.

Pandian, J. B., and Noel, M. M., 2018. Control of a Bioreactor using a New Partially Supervised Reinforcement Learning Algorithm. Journal of Process Control, 69(1), 16-29

Piñeiro-Chousa, J., López-Cabarcos, M. A., Quiñoá-Piñeiro, L., and Pérez-Pico, A. M., 2021. US Biopharmaceutical Companies' Stock Market Reaction to the Covid-19 Pandemic. Understanding The Concept of the 'Paradoxical Spiral' from a Sustainability Perspective. Technological Forecasting and Social Change, 175, 121365.

Popa, V., 2017. Q-Learning: an intelligent technique for financial trading systems implementation. Master's Thesis, Ca' Foscari University of Venice.

Remorov, A., 2016. Dynamic Trading and Behavioral Finance. Master's Thesis, Massachusetts Institute of Technology.

Ricciardi, V., and Simon, H. K., 2020. What is Behavioral Finance? Business, Education and Technology Journal, 2(2), 1-9.

Savaş, M. C., 2017. Algorithmic Trading Strategies using Dynamic Mode Decomposition: Applied to Turkish Stock Market. Master's Thesis, Middle East Technical University.

Spooner, T., 2021. Algorithmic Trading and Reinforcement Learning: Robust Methodologies for AI in Finance. Master's Thesis, University of Liverpool.

Sutton, R. S., 1999. Reinforcement Learning. Working Paper.

Sutton, R. S., and Barto, A. G., 2014. Finite Markov Decision Processes, Dynamic Programming, Planning and Learning with Tabular Methods. In: Reinforcement Learning: An Introduction, MIT Press; 2nd Edition.

Tan, F., Yan, P., and Guan, X., Deep Reinforcement Learning: From Q-Learning to Deep Q-Learning. Conference Paper, International Conference on Neural Information Processing.

Tanveer, J., Malik, A. H., Ali, R., and Kim, A., 2022. An Overview of Reinforcement Learning Algorithms for Handover Management in 5G Ultra-Dense Small Cell Networks. Applied Sciences, 12(1), 1-25

Waheed, A., 2013. Analysis of Moving Average Convergence Divergence (MACD) as a Tool of Equity Trading at the Karachi Stock Exchange. Master's Thesis, Blekinge Institute of Technology.

Wang, Y., Wang, D., Zhang, S., Feng, Y., Li, S., and Zhou, Q., 2017. Deep Q-trading. Technical Report, CSLT TECHNICAL REPORT-20160036.

Weijs, L., 2018. Reinforcement Learning in Portfolio Management and its Interpretation. Master's Thesis, Erasmus School of Economics.

Yang, S. Y., Kirilenko, A., and Hayes, R. L., 2012. Behavior Based Learning in Identifying High Frequency Trading Strategies. Conference Paper, Computational Intelligence for Financial Engineering & Economics (CIFEr)

Zeng, Y., and Klabjan, D., 2018. Portfolio Optimization for American Options. Journal of Computational Finance, 22(3), 37-64.