簡易檢索 / 詳目顯示

回結果列表
研究生： 蔡漢蒼
Tsai, Han-Tsang
論文名稱： 考慮熱效應與舒適度之增強式學習方法於最佳多空調需量反應控制系統
Reinforcement Learning Based Optimal Multi-Air-Conditioner DR Control System Considering Thermal Effect and Comfort
指導教授： 楊宏澤
Yang, Hong-Tzer
學位類別： 碩士
Master
系所名稱： 電機資訊學院 - 電機工程學系
Department of Electrical Engineering
論文出版年： 2021
畢業學年度： 109
語文別： 英文
論文頁數： 64
中文關鍵詞： 熱舒適度 、不滿意度指標 、需量反應 、空調溫度控制策略 、增強式學習 、Double Deep Q Network模型
外文關鍵詞： Air-conditioning temperature control strategy, demand response, double deep Q-network, predicted percentage dissatisfied index, reinforcement learning, thermal comfort
相關次數： 點閱：63  下載：4 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 由於近年的全球暖化影響，為維持建築物中人員的舒適度，空調設備裝設數量有增無減，然而大量地使用空調，除增加電網負擔外，若溫度控制不佳亦可能造成不必要的電能損失。透過空調設備參與需量反應可提供系統運營商有效地抑低尖峰負載，提高電網可靠性，然而在執行需量反應時，過度降低空調負載將可能導致人員須忍受不舒適的環境，減少參與需量反應的意願；反之當溫度控制過於寬鬆卻又無法提供足夠的卸載量，因此如何折衷以取得平衡為電能管理系統成為在控制空調負載時的一項挑戰。
    本研究提出基於綜合考量舒適度與卸載量之空調溫度控制模型，當受控場域欲執行需量反應時，將卸載量分配予各用戶後，執行基於舒適度指標與卸載需求之最佳控制策略。本研究提出以增強式學習(Reinforcement Learning)模型之特性訓練空調在需量反應時段運轉資訊及室內舒適度需求，尋找並提出最佳控制決策。本研究設計了一套獎勵機制，使模型的學習目標為滿足卸載量的同時，維持室內舒適度。除此之外，本研究亦使用神經網路(Neural Network)學習室內空間之熱模型，以使本模型可自行精進並適用於未知的環境資訊，以提高未來應用的準確性。
    本研究模擬二十個裝設有空調設備的房間共同參與需量反應，模擬結果顯示本研究提出的模型可使各種不同類型的房間維持環境舒適，同時可使聚合商滿足需量反應之卸載需求。

    The climate has changed substantially due to global warming in recent years. To maintain the comfort of building occupants, air-conditioning equipment is increasingly being used. Poor control of such equipment can cause unnecessary energy consumption. Applying demand response (DR) to air-conditioning equipment can effectively suppress peak loads and increase grid reliability for system operators. When executing DR, excessive reduction of the air conditioning load causes building occupants to feel uncomfortable; additionally, providing sufficient curtailment capacity when temperature control is inappropriate is impossible. The proper control of air conditioners during DR is an urgent challenge.
    This thesis proposes an air conditioner temperature control model based on comfort and demand curtailment. When DR adjustment is required, the optimal control strategy is implemented after energy curtailment has been applied fairly to each customer. Under the proposed method, the optimal control decision is determined by utilizing the characteristics of reinforcement learning to determine the indoor comfort requirements and operating information of air conditioners during the DR period. Furthermore, we design a set of reward mechanisms that constitute the learning objectives of the model: demand curtailment with the maintenance of indoor comfort. In addition, this thesis adapts the recurrent neural network to train the indoor space thermal model such that the model can learn by itself and employ unknown environmental information to improve its accuracy in future applications.
    DR for 20 rooms equipped with air conditioners is simulated. The results reveal that the proposed model maintains a comfortable environment in various types of rooms and enables the aggregator to achieve energy curtailment.

    摘要 I ABSTRACT III 誌謝 V CONTENTS VI LIST OF FIGURES X LIST OF TABLES XII ABBREVIATIONS XIII Chapter 1. INTRODUCTION 1 1.1 Background and Motivation 1 1.2 Literature Review 2 1.2.1 Comfort Estimation 2 1.2.2 Control Method 3 1.3 Methods and Study Contributions 5 1.4 Thesis Organization 6 Chapter 2. OVERVIEW OF THE CONTROL SYSTEM 7 2.1 System Architecture 7 2.2 Indoor Thermal Comfort 9 2.2.1 PMV Index 10 2.2.2 PPD Index 12 2.3 DR Rules 14 Chapter 3. PROBLEM FORMULATION AND THE PROPOSED CONTROL STRATEGY 16 3.1 System Operation Process 16 3.2 Reinforcement Learning 17 3.3 DRL 18 3.3.1 Learning Policies and the Value Function 19 3.3.2 Double DQN 21 3.3.3 DDQN Learning Method 26 3.3.4 MDP-Based Agent Learning Mechanism 26 3.3.5 Learning Process 30 3.3.6 Hyperparameters 31 3.4 Environment Construction 32 3.4.1 Variable-Frequency Air Conditioner Model 33 3.4.2 Equivalent Thermal Parameter Model for Air Conditioners 34 Chapter 4. ENVIRONMENT SIMULATION AND RESULTS 37 4.1 Simulation Settings 37 4.2 Environment Simulator 40 4.2.1 RNN Learning Results 40 4.2.2 Control Strategy for Optimal Comfort 42 4.3 Offline Learning Results 46 4.3.1 Distribution of the Curtailment Amount 46 4.3.2 Power Consumption Evaluation 48 4.3.3 Comfort Evaluation 53 Chapter 5. CONCLUSION AND FUTURE DIRECTIONS 58 5.1 Conclusion 58 5.2 Future Directions 59 REFERENCES 61

    [1] M. H. Chen, “The Study of Thermal Comfort and Energy Saving in the Classrooms,” M.S. thesis, EM, CYUT, Taichung, 2005.
    [2] R. L. Hwang, T. P. Lin, and N. J. Kuo, “Field Experiments on Thermal Comfort in Campus Classrooms in Taiwan,” Energy and Buildings, vol. 38, Issue 1, pp. 53-62, 2006.
    [3] C. C. Yu, “The Study on the Influence of Indoor Humidity and Temperature to Comfort Index,” M.S. thesis, EM, NPUST, Pingtung, 2010.
    [4] S. L. Lin, S. M. Wei, C. H. Huang, and W. K. Chen, “Thermal Comfort Study of an Air-Conditioned Presentation Room in Taiwan,” Journal of Architecture, no. 65, pp. 125-138, 2008, doi: 10.6377/JA.200809.0125.
    [5] Z. A. Shah, H. F. Sindi, A. Ul-Haq, and M. A. Ali, “Fuzzy Logic-Based Direct Load Control Scheme for Air Conditioning Load to Reduce Energy Consumption,” IEEE Access, vol. 8, pp. 117413-117427, 2020, doi: 10.1109/ACCESS.2020.3005054.
    [6] ANSI/ASHRAE Standard 55. (2010). Thermal Environmental Conditions for Human Occupancy. [Online]. Available: http://arco-hvac.ir/wp-content/uploads/2015/11/ASHRAE-55-2010.pdf, Accessed on: May. 21, 2021.
    [7] ISO7730. (2005). Ergonomics of the Thermal Environment — Analytical Determination and Interpretation of Thermal Comfort Using Calculation of the PMV And PPD Indices and Local Thermal Comfort Criteria. [Online]. Available: https://www.iso.org/standard/39155.html, Accessed on: May. 27, 2021.
    [8] W.H. Fang, “Application of PMV Fuzzy Control Algorithm in Pursuing Optimum Thermal Comfort,” M.S. thesis, EE, NSYSU, Kaohsiung, 2012.
    [9] R. Fan, Y. Li, Y. Cao, W. Xie, Y. Tan, and Y. Cai, “An Optimization Management Strategy for Energy Efficiency of Air Conditioning Loads in Smart Building,” 2016 IEEE 16th International Conference on Environment and Electrical Engineering (EEEIC), pp. 1-5, 2016, doi: 10.1109/EEEIC.2016.7555507.
    [10] M. Song, C. Gao, H. Yan, and J. Yang, “Thermal Battery Modeling of Inverter Air Conditioning for Demand Response,” IEEE Transactions on Smart Grid, vol. 9, no. 6, pp. 5522-5534, Nov. 2018, doi: 10.1109/TSG.2017.2689820.
    [11] S. Liu, C. Chen, W. Duan, Y. Dong, and C. Li, “The Research on Technology of Periodic Stopping of Central Air Conditioning Based on Modelling and Simulation of Demand Response,” 2012 China International Conference on Electricity Distribution, pp. 1-4, 2012, doi: 10.1109/CICED.2012.6508600.
    [12] A. G. Thomas et al., “Intelligent Residential Air-Conditioning System with Smart-Grid Functionality,” IEEE Transactions on Smart Grid, vol. 3, no. 4, pp. 2240-2251, Dec. 2012, doi: 10.1109/TSG.2012.2215060.
    [13] J. Kiljander et al., “Intelligent Consumer Flexibility Management with Neural Network-Based Planning and Control,” IEEE Access, vol. 9, pp. 40755-40767, 2021, doi: 10.1109/ACCESS.2021.3060871.
    [14] V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing Atari with Deep Reinforcement Learning,” arXiv preprint, 2013, doi: arXiv:1312.5602
    [15] K. H. Yu, Y. A. Chen, E. Jaimes, W. C. Wu, K. K. Liao, J. C. Liao, K. C. Lu, W. J. Sheu, and C. C. Wang, “Optimization of Thermal Comfort, Indoor Quality, and Energy-Saving in Campus Classroom Through Deep Q Learning,” Case Studies in Thermal Engineering, vol. 24, April, 2021.
    [16] W. Valladares, M. Galindo, J. Gutiérrez, W. C. Wu, K. K. Liao, J. C. Liao, K. C. Lu, and C. C. Wang, “Energy Optimization Associated with Thermal Comfort and Indoor Air Control Via A Deep Reinforcement Learning Algorithm,” Building and Environment, vol. 155, pp. 105-117, 2019.
    [17] P. O. Fanger, “Thermal Comfort: Analysis and Applications in Environmental Engineering.,” Danish Technical Press, Copenhagen, Denmark, 1970.
    [18] 台灣電力股份有限公司. (Mar. 2021). 需量反應負載管理措施. [Online]. Available: https://www.taipower.com.tw/upload/135/2021022611150421848.pdf, Accessed on: Jun. 9, 2021.
    [19] H. V. Hasselt, A. Guez, and D. Silver, “Deep Reinforcement Learning with Double Q-learning,” arXiv preprint, 2015, doi: arXiv:1509.06461.
    [20] V. Mnih, K. Kavukcuoglu, D. Silver, et al. “Human-Level Control Through Deep Reinforcement Learning,” Nature, vol. 518, pp. 529-533, 2015.
    [21] L. J. Lin, “Reinforcement Learning for Robots Using Neural Networks,” Ph.D. dissertation. CMU, Pittsburgh, PA, 1992.
    [22] J. R. Yang, K. Shi, X. Q. Cui, et al. “Peak Load Reduction Method of Inverter Air-Conditioning Group Under Demand Response.” Automation of Electric Power Systems, vol. 42, no. 24, pp. 44-52, 2018, doi:10.7500/AEPS20180517006.
    [23] H. Zhao et al., “Learning Based Compact Thermal Modeling for Energy-Efficient Smart Building Management,” 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 450-456, 2015, doi: 10.1109/ICCAD.2015.7372604.
    [24] N. Lu, “An Evaluation of the HVAC Load Potential for Providing Load Balancing Service,” IEEE Transactions on Smart Grid, vol. 3, no. 3, pp. 1263-1270, Sept. 2012, doi: 10.1109/TSG.2012.2183649.

    下載圖示 校內：2024-07-19公開
    校外：2024-07-19公開
    QR CODE