隨著物聯網技術的發展，物聯網(IoT)在我們的生活中得到了廣泛的應用，其中，機器對機器間的通信是該領域中一個重要的運作概念。大量的各種數據，包括請求和響應，在不同物聯網節點中傳輸，由於每個應用層協議都有其特點，如重傳機制、和其基本特性等，皆可能會影響到傳輸性能，其中包括了在不同網路條件下和不同資料量大小下的訊息丟失率、傳輸延遲、能量消耗效率等。在我們先前的研究中，我們提出一種基於OM2M 架構的選擇機制，它允許OM2MMNCSE(GateWay)和OM2M INCSE(Server)在相互傳輸數據的同時選擇合適的應用層協議。然而，其選擇機制是一種基於靜態閾值的方法，當它在各種動態變動的網絡條件下使用時，找不到合適的應用層協議，可能會導致性能不佳，在本研究中，
我們提出使用DQN技術作為協議選擇策略，它可以與環境交互並選擇合適的應用層協議以獲得良好的性能，我們最終通過實驗結果並對其進行驗證。

With the development of IoT technology, IoT(Internet of Things) is widely applied in our life. M2M communication acts as an important operating concept among this domain. Lots of various data, including requests and responses is transmitted between diversified IoT nodes. Since each application-layer protocol has its characteristics, such as retransmission mechanism, etc. They are likely to affect the transmission performances including message loss rate, transmission delay, efficiency of energy consumption under different network conditions and different payload sizes. In our previous study, we proposed a selection mechanism based on OM2M architecture, which allows OM2M MN-CSE and OM2M IN-CSE to choose an appropriate application-layer protocol while they are transmitting data. However, the selection mechanism is a static-threshold based method, when it is used in dynamic network conditions it cannot find the suitable application-layer protocol, which may lead to poor performance.
In this thesis, we propose to use DQN technique as the protocol selection policy, which can interactive with the environment and choose the suitable application-layer protocol for obtaining good performance. We finally verified it with experiment results.

[1] P. Datta and B. Sharma, ”A survey on IoT architectures, protocols, security and smart city based applications”, 2017 8th International Conference on Computing,Communication and Networking Technologies (ICCCNT), Delhi, pp. 1-5, 2017
[2] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari and M. Ayyash, ”Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications,” in IEEE Communications Surveys Tutorials, vol. 17, no. 4, pp. 2347-2376, Fourthquarter 2015
[3] A. Alshamsi, Y. Anwar, M. Almulla, M. Aldohoori, N. Hamad and M. Awad,”Monitoring pollution: Applying IoT to create a smart environment,” 2017 International Conference on Electrical and Computing Technologies and Applications (ICECTA), Ras Al Khaimah, pp. 1-4, 2017
[4] S. Deshmukh and S. B. Vanjale, ”IOT Based Traffic Signal Control for Reducing Time Delay of an Emergency Vehicle Using GPS,” 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA),Pune, India, pp. 1-3, 2018
[5] N. Datta, A. Malik, M. Agarwal and A. Jhunjhunwala, ”Real Time Tracking and Alert System for Laptop through Implementation of GPS, GSM, Motion Sensor and Cloud Services for Antitheft Purposes,” 2019 4th International Conference on Internet of Things: Smart Innovation and Usages (IoT-SIU), Ghaziabad, India,pp. 1-6, 2019
[6] U. Tandale, B. Momin and D. P. Seetharam, ”An empirical study of application layer protocols for IoT,” 2017 International Conference on Energy, Communication, Data Analytics and Soft Computing (ICECDS), Chennai, pp. 2447-2451, 2017
[7] RFC 7252 - The Constrained Application Protocol (CoAP) Available:https://tools.ietf.org/html/rfc7252 (Accessed January 20, 2022)
[8] MQTT Available: https://mqtt.org/ (Accessed January 20, 2022)
[9] XMPP Availiable: https://xmpp.org/ (Accessed February 14, 2022)
[10] RFC 6455- The WebSocket Protocol Avaible: https://tools.ietf.org/html/rfc6455(Accessed February 14, 2022)
[11] H. W. Chen and F. J. Lin, ”Converging MQTT Resources in ETSI Standards Based M2M Platform,” 2014 IEEE International Conference on Internet of Things(iThings), and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom), Taipei, pp. 292-295,2014
[12] J. Toldinas, B. Lozinskis, E. Baranauskas and A. Dobrovolskis, ”MQTT Quality of Service versus Energy Consumption,” 2019 23rd International Conference Electronics, Palanga, Lithuania, pp. 1-4, 2019
[13] M. Pohl, J. Kubela, S. Bosse and K. Turowski, ”Performance Evaluation of Application Layer Protocols for the Internet-of-Things,” 2018 Sixth International Conference on Enterprise Systems (ES), Limassol, pp. 180-187, 2018
[14] I. Mobin, N. Mohammed and S. Momen, ”Optimal range estimation for energy efficient dynamic packet size,” 2017 International Conference on Electrical, Computer and Communication Engineering (ECCE), Cox’s Bazar, pp. 50-55, 2017
[15] S. R. J.Ramson, S. Vishnu and M.Shanmugam, ”Applications of Internet of Things (IoT) An Overview,” 2020 5th International Conference on Devices, Circuits and Systems (ICDCS), pp. 92-95, 2020
[16] Asma Abdalla Elmangoush, M.Sc, ”Application-derived Communication Protocol Selection in M2M Platforms for Smart Cities”, Engineering Doctorate Dissertation, Berlin 18. April 2016. Retrieved from https://depositonce.tuberlin.de/bitstream/11303/5445/4/elmangoush asma.pdf
[17] REST APIs. [Online]. Available: https://restfulapi.net/resource-naming/ (Accessed May 19, 2022)
[18] M.El Ouadghiri, B. Aghoutane, N.El Farissi, Communication model in the Internet Of Things, Procedia Computer Science,2020, Pages 72-77,2020
[19] D. Jienan, C. Xiangning and C. Shuai, ”Overview of Application Layer Protocol of Internet of Things,” 2021 IEEE 6th International Conference on Computer and Communication Systems (ICCCS), pp. 922-926, 2021
[20] S. Saritha and V. Sarasvathi, ”A study on application layer protocols used in IoT,” 2017 International Conference on Circuits, Controls, and Communications (CCUBE), pp. 155-159, 2017
[21] P. Gupta and I. O. P. M, ”A Survey of Application Layer Protocols for Internet of Things,” 2021 International Conference on Communication information and Computing Technology (ICCICT), pp. 1-6, 2021
[22] M. C¸ akır, T. Hackel, S. Reider, P. Meyer, F. Korf and T. C. Schmidt, ”A QoS Aware Approach to Service-Oriented Communication in Future Automotive Networks,” in 2019 IEEE Vehicular Networking Conference (VNC) (IEEEVNC 2019). Los Angeles, USA, Dec. 2019
[23] Chun-Hao Tang, ”OM2M Platform with Application-layer Protocol Management and Selection Mechanisms ”, Master’s thesis, National Cheng Kung University, Institute of Computer Science and Information Engineering, Jul. 2017.
[24] Tat-Hin Wong, ”Internet of Things Application-layer Protocol Evaluation Based on Efficiency Metric and Delay CDF”, Master’s thesis, National Cheng Kung University, Institute of Computer Science and Information Engineering, Jul. 2020.
[25] A. Elmangoush, R. Steinke, and T. Magedanz, ”AdM2M: adaptable machine-to machine transport framework,” Information Integration and Web-based Applications Services (iiWAS), pp. 331-335, 2015
[26] Gomes, Andr´e Macedo, Daniel Vieira, Luiz. ”Automatic MAC protocol selection in wireless networks based on reinforcement learning”. Computer Communications, pp 312-323, 2020
[27] oneM2M.org,“TS-0001-Functional Architecture-V2 10 0”,2016-Aug-30. https://www.onem2m.org/images/files/deliverables/Release2/TS-0001 %20Functional Architecture-V2 10 0.pdf
[28] WANem. [Online].Available: http://wanem.sourceforge.net (Accessed April 19,2022)
[29] Tshark. https://tshark.dev/ (Accessed April 19, 2022)
[30] MSE. https://en.wikipedia.org/wiki/Mean squared error (Accessed May 15, 2021)
[31] Kingma, D. and Ba, J., ”Adam: A Method for Stochastic Optimization”. Proceedings of the 3rd International Conference on Learning Representations(ICLR), pp. 202-216, 2015
[32] Agarap, Abien Fred. ”Deep Learning using Rectified Linear Units (ReLU)”(2018)
[33] NAT. [Online]. https://en.wikipedia.org/wiki/Network address translation (Accessed May 19, 2022)
[34] Openfire. [Online]. Available: https://www.igniterealtime.org/projects/openfire (Accessed May 19, 2022)
[35] Californium. [Online]. https://github.com/eclipse/californium (Accessed May 19, 2022)
[36] Tyrus. [Online]. Available: https://tyrus.java.net (Accessed May 19, 2022)
[37] Moquette. [Online]. Available: https://github.com/andsel/moquette (Accessed May 19, 2022)
[38] Flask. [Online]. Available: https://flask.palletsprojects.com/en/2.2.x/ (Accessed May 19, 2022)
[39] Matplotlib. [Online]. Available: https://matplotlib.org/stable/tutorials/introductory/pyplot.html (Accessed May 19, 2022)