近年來電動車數量逐漸增加，充電站選擇對於電動車駕駛來說變的愈發重要。電動車所有者關心的兩個關鍵因素是與充電站的距離和排隊時間。本論文嘗試使用強化學習來平衡距離以及排隊時間這兩個因素，提供了兩個有助於電動車用戶選擇理想充電站的決策方式。將考慮不同的環境場景，透過使用SUMO交通模擬器來進行模擬。提出的兩個決策方式以及兩個基準組將根據其減少電動車用戶花費在充電行為上的時間來進行評估。

Charging station selection policy for electric vehicle (EV) owners has received significant attentions as the number of EVs grows in recent years. Two key factors concerned by EV owners are the distance and the queuing time associated with a charging station (CS). Applying reinforcement learning method while trying to balance the two factors, this thesis presents two policies that help EV users to make a desired charging station selection. Different environment settings are being considered and simulated using the celebrated traffic simulator SUMO. The performance of the two proposed policies in comparison with two benchmark schemes policies are evaluated in terms of their ability to reduce the total amount of time EV owners need to spend on carrying out charging actions.

[1] O. Hafez and K. Bhattacharya, “Integrating ev charging stations as smart loads for demand response provisions in distribution systems,” IEEE Transactions on
Smart Grid, vol. 9, no. 2, pp. 1096–1106, 2016.
[2] K. Zhang, Y. Mao, S. Leng, Y. He, S. Maharjan, S. Gjessing, Y. Zhang, and D. H. Tsang, “Optimal charging schemes for electric vehicles in smart grid: A contract
theoretic approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 19, no. 9, pp. 3046–3058, 2018.
[3] H. Yang, S. Yang, Y. Xu, E. Cao, M. Lai, and Z. Dong, “Electric vehicle route optimization considering time-of-use electricity price by learnable partheno-genetic
algorithm,” IEEE Transactions on Smart Grid, vol. 6, no. 2, pp. 657–666, 2015.
[4] Z. Yi and M. Shirk, “Data-driven optimal charging decision making for connected and automated electric vehicles: A personal usage scenario,” Transportation Research Part C: Emerging Technologies, vol. 86, pp. 37–58, 2018.
[5] J. Tan and L. Wang, “Real-time charging navigation of electric vehicles to fast charging stations: A hierarchical game approach,” IEEE Transactions on Smart
Grid, vol. 8, no. 2, pp. 846–856, 2015.
[6] R. Ye, X. Huang, Z. Zhang, Z. Chen, and R. Duan, “A high-efficiency charging service system for plug-in electric vehicles considering the capacity constraint of
the distribution network,” Energies, vol. 11, no. 4, p. 911, 2018.
[7] Y. Cao, N. Wang, G. Kamel, and Y.-J. Kim, “An electric vehicle charging management scheme based on publish/subscribe communication framework,” IEEE
Systems Journal, vol. 11, no. 3, pp. 1822–1835, 2015.
[8] M. Gharbaoui, L. Valcarenghi, R. Brunoi, B. Martini, M. Conti, and P. Castoldi, “An advanced smart management system for electric vehicle recharge,” in Proc.
2012 IEEE International Electric Vehicle Conference, pp. 1–8.
[9] D. Isele, A. Cosgun, and K. Fujimura, “Analyzing knowledge transfer in deep qnetworks for autonomously handling multiple intersections,” arXiv preprint arXiv:
1705.01197, 2017.
[10] S. S. Mousavi, M. Schukat, and E. Howley, “Traffic light control using deep policygradient and value-function-based reinforcement learning,” IET Intelligent Transport Systems, vol. 11, no. 7, pp. 417–423, 2017.
[11] W. Genders and S. Razavi, “Using a deep reinforcement learning agent for traffic signal control,” arXiv preprint arXiv:1611.01142, 2016.
[12] C. Wu, A. Kreidieh, K. Parvate, E. Vinitsky, and A. M. Bayen, “Flow: Architecture and benchmarking for reinforcement learning in traffic control,” arXiv preprint arXiv:1710.05465, 2017.
[13] G. Stampa, M. Arias, D. Sanchez-Charles, V. Muntés-Mulero, and A. Cabellos, “A deep-reinforcement learning approach for software-defined networking routing
optimization,” arXiv preprint arXiv:1709.07080, 2017.
[14] S. S. Koh, B. Zhou, P. Yang, Z. Yang, H. Fang, and J. Feng, “Reinforcement learning for vehicle route optimization in sumo,” in Proc. 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS), pp. 1468–1473.
[15] A. Y. Lam, Y.-W. Leung, and X. Chu, “Electric vehicle charging station placement: Formulation, complexity, and solutions,” IEEE Transactions on Smart
Grid, vol. 5, no. 6, pp. 2846–2856, 2014.
[16] C. Luo, Y.-F. Huang, and V. Gupta, “Placement of ev charging stations—balancing benefits among multiple entities,” IEEE Transactions on Smart Grid, vol. 8,
no. 2, pp. 759–768, 2015.
[17] A. Hess, F. Malandrino, M. B. Reinhardt, C. Casetti, K. A. Hummel, and J. M. Barceló-Ordinas, “Optimal deployment of charging stations for electric vehicular
networks,” in Proc. the first workshop on Urban networking, pp. 1–6.
[18] Y. Sato, S. Ishikawa, T. Okubo, M. Abe, and K. Tamai, “Development of high response motor and inverter system for the nissan leaf electric vehicle,” SAE Technical Paper, Report 0148-7191, 2011.
[19] C. De Cauwer, J. Van Mierlo, and T. Coosemans, “Energy consumption prediction for electric vehicles based on real-world data,” Energies, vol. 8, no. 8, pp. 8573– 8593, 2015.
[20] M. C. Falvo, D. Sbordone, I. S. Bayram, and M. Devetsikiotis, “Ev charging stations and modes: International standards,” in Proc. 2014 International Symposium on Power Electronics, Electrical Drives, Automation and Motion, pp. 1134–1139.
[21] D. Krajzewicz, J. Erdmann, M. Behrisch, and L. Bieker, “Recent development and applications of sumo-simulation of urban mobility,” International Journal On
Advances in Systems and Measurements, vol. 5, no. 3&4, 2012.