隨著自動駕駛技術發展迅速，車用市場對自駕車需求也日漸增加，業界也開始投資研究自動駕駛技術，以自駕車完全自動化為目標，實現無人駕駛等技術。然而， 要實現無人駕駛技術仍存在不少困難與挑戰，例如在危險與未知的環境下，如何做出軌跡預測、規劃及車輛控制。有見及此，在本篇論文中提供了一個端到端的深度學習模型架構(Deep Imitative Model)， 結合了模仿學習(Imitation Learning, IL)及強化學習(Reinforcement Learning, RL) 的優點，從而解決軌跡預測及規劃的問題。 本論文研究首先使用模仿學習的概念，先收集專家在不同情況下所行駛的路線及輔助資訊作為訓練資料，訓練一個條件概率模型能與專家所行駛的路線分佈越接近越好且採樣出來的路線都能落在可行駛區域內，來預測不同路況時專家行駛路線的可能性並對於所預測出來的路線進行評估，之後使用強化學習的概念，把預先訓練好的條件概率模型當先驗機率，在設計目標函數後作最大化，讓機器選擇出一條最佳路線及動力模型作路線優化規劃，最後利用 PID 控制器作為反回饋控制，讓自駕車行駛更加穩定。

The development of autonomous driving and the demand for self-driving cars in the car selling market is increasing rapidly. Majority of the car product industries have begun to investigate autonomous driving technologies to increase their competitive. With the goal of autonomous driving as to achieve the fully automating self-driving technologies (Level 5). However, there are still many difficulties and challenges to achieve the level 5 standard of autonomous driving, such as how to make trajectory prediction, planning in a dangerous and unknown environment. In view of this, this paper provides a deep imitative learning model architecture (Deep Imitative Model), which combines the advantages of imitation learning (IL) and reinforcement learning (RL) to solve trajectory prediction and the problem of planning. In this research, we use imitation learning to train a conditional probability density model for inference, modeling the trajectory generated by experts under the conditions of the side information, using the trained conditional probability density model as trajectory prediction, and then with the concept of reinforcement learning, designing the objective function, optimizing the objective function by using maximum to allow the machine to choose an optimal trajectory and dynamics model, and finally applying the PID controller as feedback control to make the self-driving car drives more steady.

1. Louie, D. Inside Waymo's autonomous vehicles: Here's what it's like to ride in a car
with no driver behind the wheel. 2021; Available from: https://abc7news.com/waymosan-francisco-driverless-cars-autonomous-vehicles/10368250/.
2. Yeh, D., 百度自駕車 Apollo Go 在北京市區上路，開放乘客預約！累積測試距離
破 200 萬公里. 2020.
3. Oitzman, M. SAE clarifies autonomous driving level definitions. 2021; Available
from: https://www.therobotreport.com/sae-clarifies-autonomous-driving-leveldefinitions/.
4. Rhinehart, N., K.M. Kitani, and P. Vernaza. r2p2: A ReparameteRized Pushforward
Policy for Diverse, Precise Generative Path Forecasting. 2018. Cham: Springer
International Publishing.
5. Hussein, A., et al., Imitation learning: A survey of learning methods. ACM
Computing Surveys (CSUR), 2017. 50(2): p. 1-35.
6. Englert, P., et al. Model-based imitation learning by probabilistic trajectory matching.
in 2013 IEEE International Conference on Robotics and Automation. 2013. IEEE.
7. Abbeel, P. and A.Y. Ng, Apprenticeship learning via inverse reinforcement learning,
in Proceedings of the twenty-first international conference on Machine learning.
2004, Association for Computing Machinery: Banff, Alberta, Canada. p. 1.
8. Wulfmeier, M., et al., Large-scale cost function learning for path planning using deep
inverse reinforcement learning. The International Journal of Robotics Research, 2017.
36(10): p. 1073-1087.
9. Ho, J. and S. Ermon, Generative adversarial imitation learning. arXiv preprint
arXiv:1606.03476, 2016.
10. Li, Y., J. Song, and S. Ermon, Infogail: Interpretable imitation learning from visual
demonstrations. arXiv preprint arXiv:1703.08840, 2017.
11. Yi, F., W. Fu, and H. Liang. Model-based reinforcement learning: A survey. in
Proceedings of the International Conference on Electronic Business (ICEB), Guilin,
China. 2018.
12. Dinh, L., J. Sohl-Dickstein, and S. Bengio, Density estimation using real nvp. arXiv
preprint arXiv:1605.08803, 2016.
13. Gulrajani, I., et al., Improved training of wasserstein gans. arXiv preprint
arXiv:1704.00028, 2017.57
14. Lee, N., et al. Desire: Distant future prediction in dynamic scenes with interacting
agents. in Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition. 2017.
15. Salakhutdinov, R., Learning deep generative models. Annual Review of Statistics and
Its Application, 2015. 2: p. 361-385.
16. Blei, D.M., A. Kucukelbir, and J.D. McAuliffe, Variational inference: A review for
statisticians. Journal of the American statistical Association, 2017. 112(518): p. 859-
877.
17. Goodfellow, I.J., et al., Generative adversarial networks. arXiv preprint
arXiv:1406.2661, 2014.
18. Papamakarios, G., et al., Normalizing flows for probabilistic modeling and inference.
arXiv preprint arXiv:1912.02762, 2019.
19. Albawi, S., T.A. Mohammed, and S. Al-Zawi. Understanding of a convolutional
neural network. in 2017 International Conference on Engineering and Technology
(ICET). 2017.
20. Chung, J., et al., Empirical evaluation of gated recurrent neural networks on sequence
modeling. arXiv preprint arXiv:1412.3555, 2014.
21. Williams, R.J., Simple statistical gradient-following algorithms for connectionist
reinforcement learning. Machine learning, 1992. 8(3-4): p. 229-256.
22. Rhinehart, N., R. McAllister, and S. Levine, Deep imitative models for flexible
inference, planning, and control. arXiv preprint arXiv:1810.06544, 2018.
23. Rios, L.H.O. and L. Chaimowicz. A Survey and Classification of A* Based Best-First
Heuristic Search Algorithms. 2010. Berlin, Heidelberg: Springer Berlin Heidelberg.
24. Zidane, I. and K. Ibrahim. Wavefront and A-Star Algorithms for Mobile Robot Path
Planning. in AISI. 2017.
25. Caesar, H., nuscenes: A multimodal dataset for autonomous driving. 2020, IEEE. p.
11621.
26. Dosovitskiy, A., et al. CARLA: An open urban driving simulator. in Conference on
robot learning. 2017. PMLR.
27. "Online Barcode Generator," 20-Feb-2014. [Online]. Available: http://online-barcodegenerator.net/. [Accessed: 26-Sep-2018].