隨著全球對環境保護和可持續發展的關注日益加深，電動車（EV）的銷量在近幾年顯著增加。許多國家通過提供購車補助和建設充電設施來推動電動車市場的發展，使得電動車越來越受消費者的青睞。然而，電動車的普及也帶來了新的挑戰。如果讓電動車自由充電，容易導致電力系統的不平衡，特別是在高峰時段，電價會上漲，用電成本增加，甚至可能導致建築物的用電超過合約容量，從而產生高額罰款並大幅提升用電成本。無序的充電方式會加重此問題，進一步對電網造成壓力。

近年來，隨著車輛到電網技術（V2G）的進步，電動車不僅可以充電，還可以將電力回饋到電網。此外，研究顯示，多數私家車在大部分時間內處於閒置狀態。如果能夠有效利用這些閒置車輛的電力，不僅能滿足電動車使用者的需求，還能通過調整充電和放電行為來平衡建築物的電力負載。

本研究提出了一個基於分組的多智能體強化學習（GB-MARL）框架，旨在優化建築物的電力消耗。此框架整合了即時和歷史數據，通過智慧調控電動車的充電和放電行為，應對建築物負載的波動，避免在高峰時段用電超出合約容量。此外，透過在電價較低時充電，電價較高時放電，實現降低總電費的目標。

在實驗部分，我們將 GB-MARL 框架與其他基本方法進行比較。初步結果表明，GB-MARL 在降低能源成本和提升系統效率方面具有潛力。我們還探討了不同參數設置對 GB-MARL 性能的影響，從而評估其在各種情境下的適應性。

As global concern for environmental protection and sustainable development grows stronger, the sales of electric vehicles (EVs) have significantly increased in recent years. Many countries are promoting the development of the electric vehicle market by offering purchase subsidies and building charging infrastructure, making electric vehicles increasingly popular among consumers. However, the widespread adoption of electric vehicles also presents new challenges. Allowing electric vehicles to charge freely can easily lead to imbalances in the power system, especially during peak hours, causing electricity prices to rise, increasing electricity costs, and potentially leading to buildings exceeding their contracted power capacity, which could result in substantial penalties and significantly higher electricity expenses. Uncoordinated charging patterns would exacerbate this issue, putting additional pressure on the power grid.

In recent years, with advancements in Vehicle-to-Grid (V2G) technology, electric vehicles can not only charge but also feed power back into the grid.Moreover, studies show that most private cars remain idle for the majority of the time. If the electricity from these idle vehicles can be effectively utilized, it would not only meet the needs of electric vehicle users but also help balance the power load of buildings by adjusting charging and discharging behaviors.

This study proposes a Group-Based Multi-Agent Reinforcement Learning (GB-MARL) framework aimed at optimizing the power consumption of buildings. This framework integrates real-time and historical data to intelligently regulate the charging and discharging behaviors of electric vehicles, addressing fluctuations in building load and preventing electricity usage from exceeding contracted capacity during peak periods. Additionally, by charging during low electricity prices and discharging during high prices, it aims to reduce the overall electricity costs.

In the experimental section, we compare the GB-MARL framework with other baseline methods. Preliminary results indicate that GB-MARL has the potential to reduce energy costs and improve system efficiency. We also explore the impact of different parameter settings on the performance of GB-MARL, assessing its adaptability in various scenarios.

[1] R. Irle, “Global ev sales for 2023,” Jan. 2024. [Online]. Available: https: //ev-volumes.com/news/ev/global-ev-sales-for-2023/

[2] H. Engel, R. Hensley, S. Knupfer, and S. Sahdev, “The potential impact of electric vehicles on global energy systems,” Aug. 2018. [Online]. Avail- able: https://www.mckinsey.com/industries/automotive-and-assembly/our-insights/ the-potential-impact-of-electric-vehicles-on-global-energy-systems/

[3] ChargePoint, “Ev adoption creates more demand for workplace charging sta- tions,” May 2024. [Online]. Available: https://www.cbre.com/insights/articles/ ev-adoption-creates-more-demand-for-workplace-charging-stations/

[4] D. Romano, “Demand for ev charging at the workplace increases,” May 2024. [Online]. Available: https://csnews.com/demand-ev-charging-workplace-increases/

[5] D. Z. MORRIS, “Today’s cars are parked 95% of the time,” Mar. 2016. [Online]. Available: https://fortune.com/2016/03/13/cars-parked-95-percent-of-time/

[6] Z. Zhang, W. Li, and X. Li, “Electric vehicle load optimization model considering peak and valley electricity price time,” 2023 IEEE 2nd International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA), pp. 54–58, 2023.

[7] Z. Rostamnezhad, N. Mary, L.-A. Dessaint, and D. Monfet, “Electricity consumption op- timization using thermal and battery energy storage systems in buildings,” IEEE Trans- actions on Smart Grid, vol. 14, pp. 251–265, 2023.

[8] Y. Chen, X. Wang, J. Wang, T. Qian, and Q. Peng, “Power control strategy of battery energy storage system participating in power system peak load shifting,” 2020 5th Asia Conference on Power and Electrical Engineering (ACPEE), pp. 710–715, 2020.

[9] Z. Rostamnezhad and L.-A. Dessaint, “Power management in smart buildings using re- inforcement learning,” 2023 IEEE Power & Energy Society Innovative Smart Grid Tech- nologies Conference (ISGT), pp. 1–5, 2023.

[10] L. Yan, X. Chen, Y. Chen, and J. Wen, “A cooperative charging control strategy for electric vehicles based on multiagent deep reinforcement learning,” IEEE Transactions on Industrial Informatics, vol. 18, pp. 8765–8775, 2022.

[11] A. Shojaeighadikolaei and M. Hashemi, “An efficient distributed multi-agent reinforce- ment learning for ev charging network control,” 2023 59th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pp. 1–8, 2023.

[12] N. Mhaisen, N. Fetais, and A. M. Massoud, “Real-time scheduling for electric vehicles charging/discharging using reinforcement learning,” 2020 IEEE International Conference on Informatics, IoT, and Enabling Technologies (ICIoT), pp. 1–6, 2020.

[13] H. Li, Z. Wan, and H. He, “Constrained ev charging scheduling based on safe deep rein- forcement learning,” IEEE Transactions on Smart Grid, vol. 11, pp. 2427–2439, 2020.

[14] Y.-T. Chai, W.-N. Tan, M.-T. Gan, and S.-C. Yip, “An optimal charging and discharging schedule to maximize revenue for electrical vehicle,” 2019 IEEE Conference on Sustainable Utilization and Development in Engineering and Technologies (CSUDET), pp. 240–245, 2019.

[15] K. Chopra, M. kumar Shah, and K. R. Niazi, “Cost benefit analysis for electric vehicle charging infrastructure in parking lot,” 2023 IEEE IAS Global Conference on Renewable Energy and Hydrogen Technologies (GlobConHT), pp. 1–5, 2023.

[16] G. R. C. Mouli, M. Kefayati, R. Baldick, and P. Bauer, “Integrated pv charging of ev fleet based on energy prices, v2g, and offer of reserves,” IEEE Transactions on Smart Grid, vol. 10, pp. 1313–1325, 2019.

[17] C. Tsaknakis, C. D. Korkas, I. T. Michailidis, and E. B. Kosmatopoulos, “Multi-agent rl framework for ev charging scheduling driven by energy costs and user preferences,” 2023 International Conference on Electrical, Communication and Computer Engineering (ICECCE), pp. 1–6, 2023.

[18] H. Li, G. Li, S. Li, B. Han, K. Wang, and J. Xu, “Optimal ev charging scheduling consid- ering the lack of charging facilities based on deep reinforcement learning,” 2023 8th Asia Conference on Power and Electrical Engineering (ACPEE), pp. 1825–1829, 2023.

[19] J. Zhao, C. Wan, Z. Xu, and J. Wang, “Risk-based day-ahead scheduling of electric vehicle aggregator using information gap decision theory,” IEEE Transactions on Smart Grid, vol. 8, pp. 1609–1618, 2017.

[20] Y. Guo, J. Xiong, S. Xu, and W. Su, “Two-stage economic operation of microgrid-like electric vehicle parking deck,” IEEE Transactions on Smart Grid, vol. 7, pp. 1703–1712, 2016.

[21] J. Zhang, Y. Guan, L. Che, and M. Shahidehpour, “Ev charging command fast allocation approach based on deep reinforcement learning with safety modules,” IEEE Transactions on Smart Grid, vol. 15, pp. 757–769, 2024.

[22] H. Li, G. Li, T. T. Lie, X. Li, K. Wang, B. Han, and J. Xu, “Constrained large-scale real- time ev scheduling based on recurrent deep reinforcement learning,” International Journal of Electrical Power & Energy Systems, 2023.

[23] J. Dong, M. I. A. Yassine, A. Armitage, S. M. I. M. S. Hossain, and M. Shamim, “Multi- agent reinforcement learning for intelligent v2g integration in future transportation sys- tems,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, pp. 15974– 15 983, 2023.

[24] D. Wu, H. Zeng, C. Lu, and B. Boulet, “Two-stage energy management for office buildings with workplace ev charging and renewable energy,” IEEE Transactions on Transportation Electrification, vol. 3, pp. 225–237, 2017.

[25] R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multi-agent actor-critic for mixed cooperative-competitive environments,” ArXiv, vol. abs/1706.02275, 2017.

[26] M. Pipattanasomporn, G. Chitalia, J. Songsiri, and et al., “Cu-bems, smart building electricity consumption and indoor environmental sensor datasets,” Sci Data, vol. 7, p. 241, 2020. [Online]. Available: https://doi.org/10.1038/s41597-020-00582-3

[27] ComEd, “Comed hourly pricing,” 2024, accessed: 2024-07-18. [Online]. Available: https://hourlypricing.comed.com/hp-api/

[28] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. M. O. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” CoRR, vol. abs/1509.02971, 2015.

[29] M. Tan, “Multi-agent reinforcement learning: Independent versus cooperative agents,” in International Conference on Machine Learning, 1997.

[30] P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V. F. Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls, and T. Graepel, “Value-decomposition networks for cooperative multi-agent learning,” ArXiv, vol. abs/1706.05296, 2017.