在過去幾年來，隨著整體能源消耗的不斷增加，資源的有限性已成為全球面臨的重大挑戰。世界上能耗占比最大的類別為建築物類，其中又以冰水系統的能耗占最大宗。隨著能源消耗的增加，相關的環境汙染和全球溫室氣體的排放量亦隨之上升。因此，制定有效的冰水系統能源管理策略，成為節能減碳戰略中重要的一環。
本研究首先利用深度學習中的長短期記憶模型 (Long Short-Term Memory，簡稱LSTM ) 來預測未來冰水系統的製冷能力。接著，將預測下一階段的製冷能力與當前製冷能力及室外濕球溫度結合作為強化學習中的環境特徵，進行策略學習，其中行動為調整冰水主機的設定溫度。通過每個時間步長所獲得的獎勵 ( Reward ) ，我們自適應地修正冰水主機設定溫度的權重，以最大化冰水系統的性能係數( COP )。
採用H公司過去一年廠務冰水系統的運轉資料進行實證分析，結果表明，結合長短期記憶模型 ( LSTM ) 與強化學習 ( Reinforcement Learning，簡稱RL )的方法顯著優於傳統基於規則的控制，這種混合方法不僅能更精確預測未來製冷的需求，還能有效地調整冰水主機的運行設置，實現更高的能源效率。

In recent years, with the continuous increase in overall energy consumption, the limited availability of resources has become a significant global challenge. Buildings constitute the largest category of energy consumption worldwide, with the highest energy consumption attributed to chiller systems. As energy consumption increases, associated environmental pollution and global greenhouse gas emissions also rise. Therefore, formulating effective energy management strategies for chiller systems has become a crucial aspect of energy-saving and carbon reduction strategies.
First, this study employs the Long Short-Term Memory (LSTM) model in deep learning to predict the future cooling capacity of the chiller system. Subsequently, the predicted cooling capacity, along with the current cooling capacity and outdoor wet-bulb temperature, are used as environmental features in reinforcement learning (RL) for policy learning. The action involves adjusting the set temperature of the chiller. By obtaining rewards at each time step, we adaptively modify the weights of the chiller's set temperature to maximize the Coefficient of Performance (COP) of the chiller systems.
Using operational data from the chiller system of Company H over the past year for empirical analysis, the results indicate that the combined LSTM and RL method significantly outperforms traditional rule-based control. This hybrid approach not only more accurately predicts future cooling demands but also effectively adjusts the chiller's operating settings, achieving higher energy efficiency.

Crawley, D.B., Lawrie, L.K., Winkelmann, F.C., Buhl, W.F., Huang, Y.J., Pedersen, C.O., Strand, R.K., Liesen, R.J., Fisher, D.E. and Witte, M.J. (2001), ‘EnergyPlus: Creating a new-generation building energy simulation program’, Energy Build, Vol. 33, pp. 319-331.
Coulomb, D. (2016), ‘IIR challenges for 2016’, International Journal of Refrigeration, Vol. 62, IV.
Esrafilian-Najafabadi, M. and Haghighat, F. (2022), ‘Towards self-learning control of HVAC systems with the consideration of dynamic occupancy patterns: Application of model-free deep reinforcement learning’, Building and Environment, Vol. 226, 109747.
Fu, Q., Chen, X.Y., Ma, S.A., Fang, N.W., Xing, B. and Chen, J.P. (2022), ‘Optimal control method of HVAC based on multi-agent deep reinforcement learning’, Energy Build, Vol. 270, 112284.
Gupta, R. and Puri, I.K. (2021), ‘Waste heat recovery in a data center with an adsorption chiller: Technical and economic analysis’, Energy Conversion and Management, Vol. 245, 114576.
Henze, G.P. and Schoenmann, J. (2003), ‘Evaluation of reinforcement learning control for thermal energy storage systems’, HVAC&R Research, Vol. 9, pp. 259-275.
He, K., Fu, Q.M., Lu, Y., Wang, Y.Z., Luo, J., Wu, H.J. and Chen, J.P. (2023), ‘Predictive control optimization of chiller plants based on deep reinforcement learning’, Journal of Building Engineering, Vol. 76, 107158.
Ho, W.T. and Yu, F.W. (2021), ‘Chiller system optimization using k nearest neighbour regression’, Journal of Cleaner Production, Vol. 303, 127050.
Karami, M. and Wang, L.P. (2018), ‘Particle Swarm optimization for control operation of an all-variable speed water-cooled chiller plant’, Applied Thermal Engineering, Vol. 130, pp. 962-978.
Liu, S.M. and Henze, G.P. (2006), ‘Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory Part 2: Results and analysis’, Energy and Buildings, Vol. 38, pp. 148-161.
Lu, L., Cai, W.J., Xie, L.H., Li, S.J. and Soh, Y.C. (2005), ‘HVAC system optimization—in-building section’, Energy and Buildings, Vol. 37, pp. 11-22.
Mnih, V., Kavukcuoglu, K. and Silver, D. (2015), ‘Human-level control through deep reinforcement learning’, Nature, Vol. 518, pp. 529-533.
Qiu, S., Li, Z. and Li, Z. (2021), Model-Free Optimal Control Method for Chilled Water Pumps Based on Multi-objective Optimization: Engineering Application. In Proceedings of the 2021 ASHRAE Virtual Conference, Phoenix, AZ, USA.
Qiu, S.N., Li, Z.H., Li, Z.W. and Wu, Q. (2022), ‘Comparative evaluation of different Multi-Agent reinforcement learning mechanisms in condenser water system control’, Buildings, Vol. 12, 12081092.
Qiu, S.N., Li, Z.H., Fan, D.L., He, R.K., Dai, X.H. and Li, Z.W. (2022), ‘Chilled water temperature resetting using model-free reinforcement learning: Engineering application’, Energy and Buildings, Vol. 255, 111694.
Sutton, R. S. and Barto, A.G. (2018), ‘Reinforcement learning: An introduction second edition introduction’, MIT press, pp. 1-22.
Sala-Cardoso, E., Delgado-Prieto, M., Kampouropoulos, K. and Romeral, L. (2020), ‘Predictive chiller operation: a data-driven loading and scheduling approach’, Energy Build, Vol. 208, 109639.
Thu, K., Saththasivam, J., Saha, B.B., Chua, K.J., Murthy, S.S. and Ng, K.C (2017),‘Experimental investigation of a mechanical vapour compression chiller at elevated chilled water temperatures’, Applied Thermal Engineering, Vol. 123, pp. 226-233.
Wang, L., Lee, E.W.M. and Yuen, R.K.K. (2018), ‘A practical approach to chiller plants optimisation’, Energy Build, Vol. 169, pp. 332-343.
Wang, Z. and Hong, T.Z. (2020), ‘Reinforcement learning for building controls: the opportunities and challenges’, Applied Energy, Vol. 269, 115036.