文字預測一直是人們感興趣的熱門課題之一，也在近年來越來越流行。近期的許多研究使用深度學習進行財報的預測，也都各自獲得了不同的成果。以此為啟發，本研究使用深度學習(Deep Learning)自然語言處理(Natural Language Processing，NLP)建立一個財報的文字分析。利用卷積神經網路構成的深度強化學習來預測財報文字分析，並使用深度強化學習演算法來訓練。本研究選擇了金融預訓練模型(Financial Sentiment Analysis with BERT, FinBERT)、長短期記憶模型(Long Short-Term Memory ,LSTM) 、簡單循環網路 (Simple Recurrent Neural Network , SimpleRNN)、門控循環單元 (Gated Recurrent Unit, GRU)等多種深度學習演算法訓練模型來預測財報的感情指數以及準確率。本研究使用了多個深度學習模型框架，針對英文的財報進行檢測，並且根據不同的文字特徵選擇進行比較，透過使模型完整的學習每一類支持與反駁的文字特徵，進而準確的檢測是否支持。

隨後分別使用Task Force on Climate-Related Financial Disclosures (TCFD)數據集，以及使用資料平衡過後的數據集進行模擬。結果在使用FinBERT訓練的可達到了最高78%的準確率，另一方面SimpleRNN在資料平衡過後比為平衡前的準確率提高了10%。最後發現比較四個模型遞歸神經網路模型、長短期記憶模型、門控循環單元模型以及FinBERT，它的準確率為最高並且在未平衡數據集和平衡後的數據集中都有著更好的表現。本研究驗證了深度強化學習在文字預測上的潛力，提出一種可行的強化學習環境。在未來可以設計更貼合實務的模型，做為未來進一步發展人工智慧用於金融市場分析的參考。

This research proposes deep learning to predict financial forecast, and it has become more and more popular in recent years. This research uses Deep Learning (DL) Natural Language Processing (NLP) to establish a text analysis of financial statements. The deep learning composed of convolutional neural network is used to predict financial report text analysis, and the deep learning algorithm is used to train. In this research, it selected a variety of in-deep learning algorithm training models, such as Financial Sentence Analysis with BERT (FinBERT), Long Short-Term Memory (LSTM), Simple Recurrent Neural Network (SimpleRNN), Gated Recurrent Unit (GRU), to predict the emotional index and accuracy of financial reports.
This research uses multiple deep learning model frameworks to detect English financial statements and compares them according to different text features. By making the model complete to learn each type of supporting and refuting text features, it can accurately detect whether they are supported.
Subsequently, the simulation was performed using TCFD data sets and datasets after data balance. Finally, it proves that FinBERT has the highest accuracy when comparing the four models and has better performance in both datasets.

[1] MIKOLOV, Tomas, et al. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013.
[2] Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
[3] Board, F. S. (2019). Task Force on Climate-Related Financial Disclosures: 2019 status report.
[4] Steffen, B. (2021). A comparative analysis of green financial policy output in OECD countries. Environmental Research Letters, 16(7), 074031.
[5] Europeia, C. (2008). EUR Lex. Access to European Union law.
[6] Friederich, D., Kaack, L. H., Luccioni, A., & Steffen, B. (2021). Automated identification of climate risk disclosures in annual corporate reports. arXiv preprint arXiv:2108.01415.
[7] House, W. (2021). Executive order on climate-related financial risk. White House. Available at: https://www. whitehouse. gov/briefing-room/presidential-actions/2021/05/20/executive-order-on-climate-related-financial-risk/(accessed 28 May 2021).
[8] Luccioni, A., & Palacios, H. (2019). Using natural language processing to analyze financial climate disclosures. In Proceedings of the 36th International Conference on Machine Learning, Long Beach, California.
[9] Bingler, J. A., Kraus, M., Leippold, M., & Webersinke, N. (2022). Cheap talk and cherry-picking: What climatebert has to say on corporate climate risk disclosures. Finance Research Letters, 102776.
[10] Kölbel, J., Leippold, M., Rillaerts, J., & Wang, Q. (2020). Does the CDS market reflect regulatory climate risk disclosures?. SSRN, (3616324).
[11] Sautner, Z., van Lent, L., Vilkov, G., & Zhang, R. (2020). Firm-level climate change exposure. European Corporate Governance Institute–Finance Working Paper, (686).
[12] Luccioni, A., Baylor, E., & Duchene, N. (2020). Analyzing sustainability reports using natural language processing. arXiv preprint arXiv:2011.08073.Long Papers). 2018.
[13] El Mokhtari, K., Maidens, J., & Bener, A. (2019, May). Predicting commentaries on a financial report with recurrent neural networks. In Canadian Conference on Artificial Intelligence (pp. 531-542). Springer, Cham.
[14] O'Dwyer, B., & Unerman, J. (2020). Shifting the focus of sustainability accounting from impacts to risks and dependencies: Researching the transformative potential of TCFD reporting. Accounting, Auditing & Accountability Journal, 33(5), 1113-1141.
[15] Doi: https://proj.ftis.org.tw/isdn/Message/MessageView/245?mid=47&page=1
[16] Chai, J., & Li, A. (2019, July). Deep learning in natural language processing: A state-of-the-art survey. In 2019 International Conference on Machine Learning and Cybernetics (ICMLC) (pp. 1-6). IEEE.
[17] Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. (2011). Natural language processing (almost) from scratch. Journal of machine learning research, 12(ARTICLE), 2493-2537.Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image processing, 15(12), 3736-3745.
[18] He, K., Zhang, X., Ren, S., and Sun, J. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.
[19] Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
[20] Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780.
[21] Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555.
[22] Zhang, J., Luan, H., Sun, M., Zhai, F., Xu, J., Zhang, M., & Liu, Y. (2018). Improving the transformer translation model with document-level context. arXiv preprint arXiv:1810.03581.
[23] Bingler, J. A., Kraus, M., Leippold, M., & Webersinke, N. (2022). Cheap talk in corporate climate commitments: The effectiveness of climate initiatives. Available at SSRN 3998435.
[24] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
[25] Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., ... & Stoyanov, V. (2019). Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692.
[26] Webersinke, N., Kraus, M., Bingler, J. A., & Leippold, M. (2021). Climatebert: A pretrained language model for climate-related text. arXiv preprint arXiv:2110.12010.
[27] Araci, D. (2019). Finbert: Financial sentiment analysis with pre-trained language models. arXiv preprint arXiv:1908.10063.
[28] Loughran, T., & McDonald, B. (2011). When is a liability not a liability? Textual analysis, dictionaries, and 10‐Ks. The Journal of finance, 66(1), 35-65.
[29] Mathias Kraus and Stefan Feuerriegel. 2017. Decision support from financial disclosures with deep neural networks and transfer learning. Decision Support Systems 104 (2017), 38–48. https://doi.org/10.1016/j.dss.2017.10.001 arXiv:1710.03954
[30] Sahar Sohangir, Dingding Wang, Anna Pomeranets, and Taghi M Khoshgoftaar. 2018. Big Data: Deep Learning for financial sentiment analysis. Journal of Big Data 5, 1 (2018). https://doi.org/10.1186/s40537-017-0111-6
[31] Macedo Maia, Andr Freitas, and Siegfried Handschuh. 2018. FinSSLx: A Sentiment Analysis Model for the Financial Domain Using Text Simplification. In 2018 IEEE 12th International Conference on Semantic Computing (ICSC). IEEE, 318–319. https://doi.org/10.1109/ICSC.2018.00065
[32] Ozbayoglu, A. M., Gudelek, M. U., & Sezer, O. B. (2020). Deep learning for financial applications: A survey. Applied Soft Computing, 93, 106384.Li, X., Li, Y., Zhan, Y., and Liu, X. Y. "Optimistic bull or pessimistic bear: Adaptive deep reinforcement learning for stock portfolio allocation." arXiv preprint arXiv:1907.01503, 2019.