目前傾轉式旋翼機的事故多是發生在模式轉換期間，因為兩個模式之間轉換具有極大的不穩定性，因此如何穩定且安全地完成模式轉換，成為了傾轉式旋翼機實際應用的關鍵挑戰。本研究旨在探討如何設計模式轉換策略的最佳化方法，由於傾轉式旋翼機操控環境十分的複雜，使用傳統控制器來最佳化路徑，過程中會耗費大量時間在調整內部參數，讓飛行器在某一路徑中能夠穩定飛行。因此本文選用強化學習的DDPG Agent來探索模式轉換策略，經過神經網路的演算，既使是複雜又充滿不確定性的環境，強化學習也能藉由與環境的互動，以及獎勵函數的回饋， 探索出最佳的操作方法。
通過訓練，模型能夠在動態環境中實現穩定的模式轉換，並有效地完成短艙傾角的起飛操作。與傳統控制器的飛行結果進行了比較，強化學習方法在處理複雜環境中的轉換策略上具有明顯優勢，能夠更有效地實現模式轉換的探索，並顯著提高了控制系統的靈活性和適應性。研究結果證明了強化學習在最佳化傾轉式旋翼機轉換策略中的可行性，為未來在自主控制系統中的應用提供了新的方向和想法。

Tiltrotor aircraft accidents predominantly occur during mode transitions due to the significant instability between the two modes. Therefore, achieving stable and safe mode transitions has become a critical challenge in the practical application of tiltrotor aircraft. This study aims to explore the optimal methods for designing mode transition strategies. Given the complexity of the operating environment for tiltrotor aircraft, using traditional controllers to optimize the flight path often requires substantial time to adjust internal parameters, ensuring stable flight along a specific route.
To address this, the study employs a Deep Deterministic Policy Gradient (DDPG) agent in reinforcement learning to explore mode transition strategies. Through neural network computations, even in complex and uncertain environments, reinforcement learning can interact with the environment and leverage feedback from reward functions to discover the optimal operational methods. Through training, the model can achieve stable mode transitions in dynamic environments and effectively perform takeoff operations with nacelle angle adjustments.
A comparison with the flight results from traditional controllers demonstrates that the reinforcement learning approach has significant advantages in handling transition strategies within complex environments. It can more effectively explore mode transitions and significantly enhance the flexibility and adaptability of the control system. The research findings confirm the feasibility of using reinforcement learning to optimize mode transition strategies for tiltrotor aircraft, providing new directions and insights for future applications in autonomous control systems.

Kleinhesselink, K. M. (2007). Stability and Control Modeling of Tiltrotor Aircraft. (Master), University of Maryland, USA.
Sha, H. (2007). A Mathrmatical Model of Unmanned Tiltrotor Aircraft and Research on Attitude Control System Design. (Master), Nanjing University of Aeronautics and Astronautics, China.
Lee, C. H. (2018). Stability and Hovering Control Modeling of Tiltrotor Aircraft. (Master), National Cheng Kung University, Taiwan.
Anderson, J. D. (2015). Introduction to Flight (8th ed.): McGraw-Hill.
Cook, M. V. (2012). Flight Dynamics Principles: Butterworth-Heinemann.
Fang, C. C. (2022). Actuator Failure Detection and Control Law Reconfiguration for Tilt-Rotor Aircraft. (Master), National Cheng Kung University, Taiwan.
Dugan, D. C. (1980). The XV-15 Tilt Rotor Research Aircraft. NASA
Lillicrap, T. P., J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver and D. Wierstra (2015) Continuous control with deep reinforcement learning.
Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine Learning.
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller, M. (2013). Playing atari with deep reinforcement learning.
Plappert, M., R. Houthooft, P. Dhariwal, S. Sidor, R. Y. Chen, X. Chen, T. Asfour, P. Abbeel and M. Andrychowicz (2017) Parameter Space Noise for Exploration.
Heess, N., D. TB, S. Sriram, J. Lemmon, J. Merel, G. Wayne, Y. Tassa, T. Erez, Z. Wang, S. M. A. Eslami, M. Riedmiller and D. Silver (2017) Emergence of Locomotion Behaviours in Rich Environments.
Majhi, J. R. (2008). Modeling Helicopter Rotor Blade Flapping Motion Considering Nonlinear Aerodynamics. CMES, 27(Tech Science Press).
Mahapatra, D. (2011). Control System Design of Tilt Rotor Unmanned Aerial Vehical (UAV). (Bachelor), National Institute of Technology Rourkela.