Environment Social Governance (ESG) 投資是以企業環境、社會和治理表現當作評鑑基礎的投資方法，在投資回報的同時，促使企業落實社會責任、環境保護以及治理效率。許多國內外的研究報告表明，ESG 與公司的財務表現存在著正相關，隨著ESG越來越受到關注，有些企業組織開始建立起ESG的評分機構。本研究使用深度學習(Deep Learning)、自然語言處理(Natural Language Processing，NLP)，並透過大型語言模型 (Large Language Model，LLM)進行文本句子生成，利用卷積神經網路構成的深度強化學習來預測ESG評級文本之分析。
本研究蒐集各臺灣半導體相關公司公開發布的Task Force on Climate-Related Financial Disclosures (TCFD) 與 Corporate Sustainability Reports (CSR) 企業永續報告書，且使用MSCI-ESG Rating進行文本標記，並透過大語言模型進行文本生成。實驗模型分別為ESG預訓練模型 (Environmental, Social, and Governance with BERT,ESG-BERT)、長短期記憶模型(Long Short-Term Memory , LSTM)、簡單循環神經網路(Simple Recurrent Neural Network , SimpleRNN)、門控循環單元(Gated Recurrent Unit, GRU)、一維卷積神經網路(1D Convolutional Neural Networks, 1D-CNN)等多種深度學習模型來進行訓練並預測ESG評級文本等級及透過Gemma LLM進行文本生成任務。比較各種實驗組合的預測準確率與評估指標後，使用本研究提出的ESG-BERT文本過濾方法並結合LSTM、GRU、1D CNN深度學習模型訓練加上大語言模型文本生成的組合準確率達到最高的87 %，進行文本資料過濾與大語言模型文本生成後資料集準確率在所有的實驗模型都有達到相當幅度的提升。本研究驗證了使用深度學習進行蒐集ESG相關的公開報告書預測評級ESG評級有著良好的準確率，可做為未來投資者的一份ESG評級市場分析的參考依據。

Environment Social Governance (ESG) investment is an investment approach that evaluates companies based on their environmental, social, and governance performance, aiming to promote corporate social responsibility, environmental protection, and governance efficiency while achieving investment returns. Numerous domestic and international studies have indicated a positive correlation between ESG and corporate financial performance. As ESG gains increasing attention, some organizations have begun to establish ESG rating agencies. This study employs deep learning, natural language processing (NLP), and large language models (LLM) for text sentence generation, utilizing deep reinforcement learning composed of convolutional neural networks to analyze and predict ESG rating texts.
The study collects Task Force on Climate-Related Financial Disclosures (TCFD) and Corporate Sustainability Reports (CSR) publicly released by various Taiwanese semiconductor companies, using MSCI-ESG Ratings for text labeling, and employs large language models for text generation. The experimental models include multiple deep learning architectures such as ESG-BERT (Environmental, Social, and Governance with BERT), Long Short-Term Memory (LSTM), Simple Recurrent Neural Network (SimpleRNN), Gated Recurrent Unit (GRU), and 1D Convolutional Neural Networks (1D-CNN). These models are used for training and predicting ESG rating text levels, as well as text generation tasks using Gemma LLM. After comparing the prediction accuracy and evaluation metrics of various experimental combinations.The ESG-BERT text filtering method proposed in this study, combined with LSTM, GRU, and 1D CNN deep learning models, and supplemented by text generated from large language models,achieved a maximum accuracy rate of 87%.Filtering the text data and generating text with large language models significantly improved the accuracy across all experimental models.
This study validates that using deep learning to collect and predict ESG ratings from publicly available ESG-related reports achieves high accuracy. This approach can serve as a reference for future investors in ESG rating market analysis.

[1] 林楷翊（2021）。運用商業新聞與文字探勘技術預測公司永續治理表現。﹝碩士論文。國立臺灣大學﹞臺灣博碩士論文知識加值系統https://hdl.handle.net/11296/9b925h。
[2] Ahmad, N., Mobarek, A., & Roni, N. N. (2021). Revisiting the impact of ESG on financial performance of FTSE350 UK firms: Static and dynamic panel data analysis. Cogent Business & Management, 8(1), 1900500.
[3] Albuquerque, R., Koskinen, Y., & Zhang, C. (2019). Corporate Social Responsibility and Firm Risk: Theory and Empirical Evidence. Management Science, 65(10), 4451-4469.
[4] Anil, G. T. G. R., Borgeaud, S., Wu, Y., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., Silver, D., Petrov, S., Johnson, M., Antonoglou, I., Schrittwieser, J., Glaese, A., Chen, J., Pitler, E., Lillicrap, T. P., . . . Vinyals, O. (2023). Gemini: A Family of Highly Capable Multimodal Models. ArXiv, abs/2312.11805.
[5] Araci, D. (2019). FinBERT: Financial Sentiment Analysis with Pre-trained Language Models. ArXiv, abs/1908.10063.
[6] Berg, F., Heeb, F., & Kölbel, J. F. (2022). The economic impact of ESG ratings. Available at SSRN 4088545.
[7] Bingler, J. A., Kraus, M., Leippold, M., & Webersinke, N. (2022). Cheap talk and cherry-picking: What ClimateBert has to say on corporate climate risk disclosures. Finance Research Letters, 47, 102776.
[8] Board, F. S. (2019). Task Force on Climate-related Financial Disclosures: 2019 status report.
[9] Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., Arx, S. v., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., Brynjolfsson, E., Buch, S., Card, D., Castellon, R., Chatterji, N. S., Chen, A. S., Creel, K. A., Davis, J., Demszky, D., . . . Liang, P. (2021). On the Opportunities and Risks of Foundation Models. ArXiv, abs/2108.07258.
[10] Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., . . . Amodei, D. (2020). Language Models are Few-Shot Learners. ArXiv, abs/2005.14165.
[11] Cahuantzi, R., Chen, X., & Güttel, S. (2023). A Comparison of LSTM and GRU Networks for Learning Symbolic Sequences. Intelligent Computing, Cham.
[12] CAI, Z.-Y., DU, H.-Y., WANG, H.-Q., ZHANG, J.-W., & ZHU, L. (2024). 1D CNN and BiSRU Based Intrusion Detection Method for Industrial Control Systems. Journal of Information Science and Engineering, 40(1), 107-123.
[13] Chatterji, A. K., Durand, R., Levine, D. I., & Touboul, S. (2016). Do ratings of firms converge? Implications for managers, investors and strategy researchers. Strategic Management Journal, 37(8), 1597-1614.
[14] Chung, J., Gülçehre, Ç., Cho, K., & Bengio, Y. (2014). Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. ArXiv, abs/1412.3555.
[15] Dai, H., Liu, Z., Liao, W., Huang, X., Cao, Y., Wu, Z., Zhao, L., Xu, S., Liu, W., Liu, N., Li, S., Zhu, D., Cai, H., Sun, L., Li, Q., Shen, D., Liu, T., & Li, X. (2023). AugGPT: Leveraging ChatGPT for Text Data Augmentation.
[16] Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. North American Chapter of the Association for Computational Linguistics.
[17] Dilling, P. F. A., Harris, P., & Caykoylu, S. (2024). The Impact of Corporate Characteristics on Climate Governance Disclosure. Sustainability, 16(5), 1962.
[18] Doi: https://www.msci.com/our-solutions/esg-investing/esg-ratings/esg-ratings-key-issue-framework
[19] Doi: https://www.twse.com.tw/staticFiles/csr/download/2020_zh.pdf
[20] Elman, J. L. (1990). Finding Structure in Time. Cognitive Science, 14(2), 179-211.
[21] Friede, G., Busch, T., & Bassen, A. (2015). ESG and financial performance: aggregated evidence from more than 2000 empirical studies. Journal of Sustainable Finance & Investment, 5(4), 210-233.
[22] Gibson Brandon, R., Krueger, P., & Schmidt, P. S. (2021). ESG Rating Disagreement and Stock Returns. Financial Analysts Journal, 77(4), 104-127.
[23] Graves, A. (2013). Generating Sequences With Recurrent Neural Networks. ArXiv, abs/1308.0850.
[24] Gupta, B., Prakasam, P., & Velmurugan, T. (2022). Integrated BERT embeddings, BiLSTM-BiGRU and 1-D CNN model for binary sentiment classification analysis of movie reviews. Multimedia Tools and Applications, 81(23), 33067-33086.
[25] Hardin, P., & Shumway, J. M. (1997). Statistical significance and normalized confusion matrices. Photogrammetric Engineering and Remote Sensing, 63.
[26] HARTZMARK, S. M., & SUSSMAN, A. B. (2019). Do Investors Value Sustainability? A Natural Experiment Examining Ranking and Fund Flows. The Journal of Finance, 74(6), 2789-2837.
[27] Hawley, J. (2017). ESG Ratings and Rankings. All over the Map. What Does it Mean? TruValue Labs.
[28] Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780.
[29] Horn, M. (2023). The Influence of ESG Ratings On Idiosyncratic Stock Risk: The Unrated, the Good, the Bad, and the Sinners. Schmalenbach Journal of Business Research, 75(3), 415-442.
[30] Kalyan, K. S. (2024). A survey of GPT-3 family large language models including ChatGPT and GPT-4. Natural Language Processing Journal, 6, 100048.
[31] Khan, M. A. (2022). ESG disclosure and Firm performance: A bibliometric and meta analysis. Research in International Business and Finance, 61, 101668.
[32] Kim, Y. (2014). Convolutional Neural Networks for Sentence Classification. Conference on Empirical Methods in Natural Language Processing.
[33] Kiranyaz, S., Avci, O., Abdeljaber, O., Ince, T., Gabbouj, M., & Inman, D. J. (2021). 1D convolutional neural networks and applications: A survey. Mechanical Systems and Signal Processing, 151, 107398.
[34] Lee, H., Choi, J., Kwon, S., & Jung, S. (2023). EaSyGuide : ESG Issue Identification Framework leveraging Abilities of Generative Large Language Models. ArXiv, abs/2306.06662.
[35] Lertvittayakumjorn, P., & Toni, F. (2019). Human-grounded Evaluations of Explanation Methods for Text Classification. ArXiv, abs/1908.11355.
[36] Li, S., Yin, P., & Liu, S. (2022). Evaluation of ESG Ratings for Chinese Listed Companies From the Perspective of Stock Price Crash Risk [Original Research]. Frontiers in Environmental Science, 10.
[37] Mehra, S., Louka, R., & Zhang, Y. (2022). ESGBERT: Language Model to Help with Classification Tasks Related to Companies Environmental, Social, and Governance Practices. ArXiv, abs/2203.16788.
[38] Mesnard, G. T. T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Riviere, M., Kale, M., Love, J. C., Tafti, P. D., Hussenot, L. e., Chowdhery, A., Roberts, A., Barua, A., Botev, A., Castro-Ros, A., Slone, A., H'eliou, A. e., Tacchetti, A., . . . Kenealy, K. (2024). Gemma: Open Models Based on Gemini Research and Technology. ArXiv, abs/2403.08295.
[39] Miralles-Quirós, M. M., Miralles-Quirós, J. L., & Valente Gonçalves, L. M. (2018). The Value Relevance of Environmental, Social, and Governance Performance: The Brazilian Case. Sustainability, 10(3), 574.
[40] Olah, C. (2015). Understanding LSTM Networks. Retrieved from http://colah.github.io/posts/2015-08-Understanding-LSTMs/
[41] Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training.
[42] Sahoo, P., Singh, A. K., Saha, S., Jain, V., Mondal, S. S., & Chadha, A. (2024). A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications. ArXiv, abs/2402.07927.
[43] Sahut, J.-M., & Pasquini-Descomps, H. (2015). ESG Impact on Market Performance of Firms: International Evidence. Management international / International Management / Gestiòn Internacional, 19(2), 40-63.
[44] Seo, H. J., Jo, D. H., & Pan, Z. (2024). ESG News Analysis Using News Big Data: Focusing on Topic Modeling Analysis. In R. Lee (Ed.), Software Engineering and Management: Theory and Application: Volume 16 (pp. 15-27). Springer Nature Switzerland.
[45] Shanaev, S., & Ghimire, B. (2022). When ESG meets AAA: The effect of ESG rating changes on stock returns. Finance Research Letters, 46, 102302.
[46] Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research, 15(1), 1929-1958.
[47] Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to Sequence Learning with Neural Networks. ArXiv, abs/1409.3215.
[48] Vaswani, A., Shazeer, N. M., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is All you Need. Neural Information Processing Systems.
[49] Webersinke, N., Kraus, M., Bingler, J. A., & Leippold, M. (2021). ClimateBert: A Pretrained Language Model for Climate-Related Text. ArXiv, abs/2110.12010.
[50] Ye, Q., Axmed, M., Pryzant, R., & Khani, F. (2023). Prompt Engineering a Prompt Engineer. ArXiv, abs/2311.05661.
[51] Yoo, K. M., Park, D., Kang, J., Lee, S.-W., & Park, W. (2021). GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation. Conference on Empirical Methods in Natural Language Processing.