在使用喚醒無線電的物聯網環境中，喚醒接收機作為一個輔助接收機，其功耗需遠低於主要接收機，所以可以不間斷地偵測通道，並且僅在收到喚醒碼時才喚醒高耗能的主要收發機，因此可以大幅降低平均功耗。由於以上特性，喚醒無線電可以有效的解決閒置監聽 (Idle Listening) 和過度監聽 (Overhearing) 的問題。但是喚醒接收機為了省電，其靈敏度會比主要收發機還要低，所以傳送端需要加大功率才能讓喚醒接收機正確的收到訊息。而加大功率代表需要花費更多能源，因此如果不能有效的安排每次傳送的時間，使喚醒碼的傳輸發生碰撞將會導致更多額外的能源浪費。儘管目前已經有許多基於喚醒接收機所設計的媒體存取控制協議，但是其大多只在輕流量的環境中有優勢，而在高流量的環境中則表現不佳。
為了解決上述問題，我們基於強化學習域中的演員-評論家(Actor Critic)設計了一套媒體存取控制協議。本機制可以使物聯網節點各自運算，而能達到有效錯開各個物聯網節點傳送時間，進而有效地降低傳輸碰撞率和額外耗能，同時提高網路吞吐量和保持傳輸公平性。本研究透過模擬，顯示出我們提出的機制有極佳的效能，相較其他方法最多能降低約30%的傳輸碰撞率和減少約15%的能源損耗，同時還保持相當高的公平性。

In wake up radio (WuR) enabled IoT networks, an IoT device can significantly save energy by using the low power wakeup receiver (WuRx) to monitor the channel. It only turns on its main radio for communication only when it receives a predefined wake up code (WuC) at the WuRx. However, the low power consumption of a WuRx is at the price of sacrificing its sensitivity. Thus, a transmitter needs significantly more transmission power to successfully send a message to a WuRx than to a main receiver. Therefore, there is a pressing need of designing a Media Access Control (MAC) protocol for WuR enabled IoT network. In this thesis, we have designed a MAC protocol based on the Actor-Critic algorithm, which is a model of reinforcement learning. The proposed scheme allows each IoT device computes its backoff counter independently while spreading out the backoff counters of all IoT devices more evenly to minimize the probability of transmission collision. The simulation results show that the proposed scheme can effectively reduce collision probability and dramatically reduce energy consumption while improving throughput and providing excellent fairness. Compared with other methods in the literature, the proposed scheme can reduce the transmission collision ratio by up to about 30%, and reduce the energy loss by about 15%, while still maintaining fairly high fairness.

[1] Akyildiz, Ian F. and Josep Miquel Jornet, "The internet of nano-things," IEEE Wireless Communications vol.17,no.6, pp.58-63, December 2010.
[2] Kawamoto, Yuichi, et al., "A novel access control scheme to construct fresh database of ambient information in internet of things," 2015 IEEE Wireless Communications and Networking Conference (WCNC). IEEE, 2015.
[3] Atzori, Luigi, Antonio Iera, and Giacomo Morabito. "The internet of things: A survey." Computer networks vol.54,no.15, pp.2787-2805, 2010.
[4] Al-Fuqaha, Ala, et al. "Internet of things: A survey on enabling technologies, protocols, and applications." IEEE communications surveys & tutorials vol.17,no.4, pp.2347-2376, June 2015.
[5] Newes, IOT ANALYTICS
Available:https://blog.viewneo.com/blog/iot-devices-how-they-are-changing-the-world-as-we-know-it/
[6] A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, M. Ayyash, “Internet of things: A survey on enabling technologies protocols and applications”, IEEE Commun. Surveys Tuts., vol. 17, no. 4, pp. 2347-2376, 4th Quart 2015
[7] P. Datta, B. Sharma, “A Survey on IoT Architectures, Protocols, Security and Smart City based Applications,” IEEE International Conference on Computing, Communication and Networking Technologies (ICCCNT), 2017.
[8] T. Adegbija, A. Rogacs, C. Patel, and A. Gordon-Ross, “Microprocessor optimizations for the Internet of things: A survey,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2018
[9] Wu, Miao, et al., "Research on the architecture of Internet of Things," 2010 3rd International Conference on Advanced Computer Theory and Engineering (ICACTE). Vol. 5. IEEE, September 2010.
[10] Piyare, Rajeev, et al., "Ultra low power wake-up radios: A hardware and networking survey," IEEE Communications Surveys & Tutorials vol.19,no.4, pp.2117-2157, July 2017.
[11] Spenza, Dora, et al., "Beyond duty cycling: Wake-up radio with selective awakenings for long-lived wireless sensing systems," IEEE Conference on Computer Communications (INFOCOM). IEEE, May 2015.
[12] Oller, Joaquim, et al., "Has time come to switch from duty-cycled MAC protocols to wake-up radio for wireless sensor networks?." IEEE/ACM Transactions on Networking vol.24,no.2, pp. 674-687, January 2015.
[13] Oller, Joaquim, et al., "Performance evaluation and comparative analysis of subcarrier modulation wake-up radio systems for energy-efficient wireless sensor networks," Sensors, pp.22-51, Jun 2014.
[14] Le, Trong Nhan, Alain Pegatoquet, and Michele Magno., "Asynchronous on demand MAC protocol using wake-up radio in wireless body area network," International Workshop on Advances in Sensors and Interfaces (IWASI). IEEE, June 2015.
[15] Zhang, Min, Debasish Ghose, and Frank Y. Li., "Does wake-up radio always consume lower energy than duty-cycled protocols?," 2017 IEEE 86th Vehicular Technology Conference (VTC-Fall). IEEE, Sept. 2017.
[16] Ghose, Debasish, Frank Y. Li, and Vicent Pla., "MAC protocols for wake-up radio: principles, modeling and performance analysis," IEEE Transactions on Industrial Informatics vol.14,no.5, pp.2294-2306, February 2018.
[17] Tang, Suhua, and Sadao Obana., "Tight integration of wake-up radio in wireless LANs and the impact of wake-up latency," 2016 IEEE Global Communications Conference (GLOBECOM). IEEE, Dec. 2016.
[18] Ghose, Debasish, and Frank Y. Li., "Enabling backoff for SCM wake-up radio: protocol and modeling," IEEE Communications Letters vol.21,no.5, pp.1031-1034, January 2017.
[19] Sanabria-Russo, Luis, et al., "A high efficiency MAC protocol for WLANs: Providing fairness in dense scenarios," IEEE/ACM Transactions on Networking vol.25,no.1, pp.492-505, July 2016.
[20] M. Fang, D. Malone, K. R. Duffy, and D. J. Leith, “Decentralised learning MACs for collision-free access in WLANs,” Wireless Netw., vol. 19, no. 1, pp. 83–98, Jan. 2013
[21] Salmani, Vahid, and Pai H. Chou., "Resilient Round Robin: A Lightweight Deterministic MAC Primitive," ACM Transactions on Sensor Networks (TOSN) vol.11,no.2, pp.1-38, December 2014.
[22] Djiroun, Fatima Zahra, Miloud Bagaa, and Tarik Taleb., "A Markov decision process-based collision avoidance in IoT applications," 2017 IEEE Global Communications Conference. IEEE, Dec. 2017.
[23] Pegatoquet, Alain, Trong Nhan Le, and Michele Magno., "A wake-up radio-based MAC protocol for autonomous wireless sensor networks," IEEE/ACM Transactions on Networking vol.27,no1, pp. 56-70, November 2018.
[24] Mazloum, Nafiseh Seyed, and Ove Edfors., "DCW-MAC: An energy efficient medium access scheme using duty-cycled low-power wake-up receivers," 2011 IEEE Vehicular Technology Conference (VTC Fall). IEEE, December 2011.
[25] Zhang, Min, Debasish Ghose, and Frank Y. Li. "Collision Avoidance in Wake-Up Radio Enabled WSNs: Protocol and Performance Evaluation." 2018 IEEE International Conference on Communications (ICC). IEEE, May 2018.
[26] Buettner, Michael, et al., "X-MAC: a short preamble MAC protocol for duty-cycled wireless sensor networks," Proceedings of the 4th international conference on Embedded networked sensor systems. October 2006.
[27] Gamm, Gerd Ulrich, et al., "Low power wake-up receiver for wireless sensor nodes," 2010 Sixth International Conference on Intelligent Sensors, Sensor Networks and Information Processing. IEEE, Dec. 2010.
[28] Texas Instruments. CC1101 Low-Cost Low-Power Sub-1 GHz RF Transceiver Datasheet. Accessed: 2018. [Online]. Available: http://www. ti.com/product/CC1101
[29] AS3932, available: https://ams.com/documents/20143/36005/AS3932_DS000340_1-00.pdf/d907e7ae-476a-0071-e7a1-5fb0948ae09a#G1258432
[30] Gamm, Gerd Ulrich, and Leonhard Michael Reindl., "Range extension for wireless wake-up receivers," International Multi-Conference on Systems, Signals & Devices. IEEE, March 2012.
[31] Sutton, R. S., and Barto, A. G., “Reinforcement learning: An introduction”, Cambridge, MA: MIT Press, 1998.
[32] D. Silver et al., "Mastering the game of Go with deep neural networks and tree search," Nature, vol. 529, no. 7587, pp. 484-489, January 2016.
[33] Deepmind, [Online]. Available: https://deepmind.com/, 2019.
[34] Thrun, Sebastian., "Monte Carlo pomdps," Advances in neural information processing systems. 2000.
[35] Sutton, Richard S., "Learning to predict by the methods of temporal differences," Machine learning vol.3, no.1, pp. 9-44 August 1998.
[36] Jiang, Chunxiao, et al., "Machine learning paradigms for next-generation wireless networks," IEEE Wireless Communications vol.24, no.2 , pp.98-105, December 2016.
[37] OMNET++ Document.
Available: https://doc.omnetpp.org/omnetpp4/manual/usman.html
[38] C. Jiang, H. Zhang, Y. Ren, Z. Han, K.-C. Chen, and L. Hanzo, “Machine learning paradigms for next-generation wireless networks,” IEEE Wireless Commun., vol. 24, no, pp. 98-105, Apr. 2017,
[39] Bellman, Richard. "A Markovian decision process." Journal of mathematics and mechanics vol.6, no.5, pp.679-684, 1957.
[40] Bellman, Richard., "Dynamic programming," Science vol.153, no.3731, pp.34-37, Jul 1966.
[41] Konda, Vijay R., and John N. Tsitsiklis., "Actor-critic algorithms," Advances in neural information processing systems. 2000.
[42] Jain, Rajendra K., Dah-Ming W. Chiu, and William R. Hawe., "A quantitative measure of fairness and discrimination," Eastern Research Laboratory, Digital Equipment Corporation, Hudson, MA, 1984.
[43] Ali, Rashid, et al., "Deep reinforcement learning paradigm for performance optimization of channel observation–based MAC protocols in dense WLANs," IEEE Access vol.7 , pp. 3500-3511, December 2018.
[44] Pressas, Andreas, et al., "Contention-based learning MAC protocol for broadcast vehicle-to-vehicle communication," 2017 IEEE Vehicular Networking Conference (VNC), Nov. 2017.
[45] Ye, Dayong, and Minjie Zhang., "A self-adaptive sleep/wake-up scheduling approach for wireless sensor networks," IEEE transactions on cybernetics vol.48, no.3, pp. 979-992, March 2017.
[46] Karlik, Bekir, and A. Vehbi Olgac. "Performance analysis of various activation functions in generalized MLP architectures of neural networks." International Journal of Artificial Intelligence and Expert Systems vol.1, no.4 , pp.111-122, 2011.