簡易檢索 / 詳目顯示

回結果列表
研究生： 吳威廷
Wu, Wei-Ting
論文名稱： 基於機器學習演算法之D2D通訊系統資源分配
Machine Learning Based Resource Allocation in Device-to-Device Communications
指導教授： 陳曉華
Chen, Hsiao-Hwa
學位類別： 碩士
Master
系所名稱： 工學院 - 工程科學系
Department of Engineering Science
論文出版年： 2023
畢業學年度： 111
語文別： 英文
論文頁數： 257
中文關鍵詞： 裝置對裝置 、深度Q網絡 、移動感知 、資源分配
外文關鍵詞： D2D, DQN, Mobility-Aware, Resource Allocation
相關次數： 點閱：121  下載：0 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
    • 摘要 iv Abstract v Acknowledgements vii Table of Contents ix List of Figures xiii List of Tables lix List of Abbreviations lxi List of Symbols lxv Dedication lxvii 1 Introduction 1 1.1 Background and Motivation 1 1.2 Related Works 2 1.2.1 The Papers on D2D Resource Allocation with Deep Learning between 2018 to 2022 3 1.2.2 Paper Summary for 2019 to 2022 9 1.2.3 The Papers on Adaptive Resource Allocation in D2D 10 1.3 Thesis Organization 14 2 Fundamentals of D2D Communications 15 2.1 Fundamentals of D2D Communications 15 2.1.1 Classification of D2D Communications 15 2.1.2 D2D Communication Scenarios 17 2.1.3 Mode Selection 17 2.1.4 Peerdiscovery 19 2.1.5 Interference Management 20 2.1.6 Use Cases and Application Business Opportunities 21 2.2 Artificial Intelligence 23 2.2.1 Evolutionalgorithm 24 2.2.2 Expertsystem 24 2.3 Machine Learning 24 2.3.1 Supervised Learning 25 2.3.2 Unsupervised Learning 26 2.3.3 Reinforcement Learnig 26 2.4 Deep Learning 27 3 Discuss the Area where SINR of CUE can be Guaranteed and the Area where SINR of D2D receiver can be Guaranteed in Uplink D2D Communication 31 3.1 The Area Where SINR of CUE can be Guaranteed and the Area Where SINR of D2D Receiver can not be Guaranteed in Uplink D2D Communication 32 3.2 Discussion on Protected Area of CUE can Forbidden Area of CUE in D2D Uplink Communication 39 3.2.1 Protected Area of CUE and Forbidden Area of CUE do not Overlap 42 3.2.2 Protected Area of CUE and Forbidden Area of CUE Overlap 46 3.3 Simulation of Overlapping Situation of Protected Area of CUE and Forbidden Area of CUE 50 4 Performance Analysis of Resource Allocation in D2D Communication System Based on Artificail Intelligence 57 4.1 Machine Learning 58 4.1.1 Supervised Learning 58 4.1.2 Unsupervised Learning 59 4.1.3 Reinforcement Learning 61 4.1.4 Why Use Reinforcement Learning to Solve D2D Communication Resource Allocation Problem 63 4.2 Deep Learning 64 4.2.1 Neural Network 66 4.2.2 Input Layer 67 4.2.3 Output Layer 67 4.2.4 Hidden Layer 67 4.2.5 Neuron 68 4.2.6 Activation Function 68 4.2.7 Backpropagation 71 4.2.8 Gradient Descent 79 4.3 Why Use Deep Q-Learning to Solve D2D Resource Allocation Problem 80 4.4 Introduction of DeepQ-Learning 81 5 Mobility-Aware Resource Allocation in D2D with Deep Q Learning 89 5.1 Cellular Users are Moving but D2D Users are Stationary in the Cellular 90 5.1.1 D2D Mobile Aware Simulation 92 5.2 Mobility-Aware Resource Allocation when D2D is Stationary and CUE is Moving with Deep Q Learning Algorithm 111 5.3 D2D Resource Allocation with Stationary CUEs 116 5.3.1 D2D on the Boundary of the Cellular 116 5.3.2 Comparison of Performance Analysis with other Thesis on Stationary CUE 123 5.4 D2D Resource Allocation with Mobile CUEs 126 5.4.1 Comparison of Performance with Training Rounds per Epoch 128 5.4.2 CUE1 and CUE2 Angle of 180 Degrees 130 5.4.3 Convegence Time of CUE 1 and CUE 2 with Angle of 180 Degrees 198 6 Conclusions and Future Works 249 6.1 Conclusions 249 6.2 Future Works 250 Bibliography 251

    [1] A. Asadi, Q. Wang, and V. Mancuso,” A survey on device-to-device communication in cellular networks,” IEEE Communications Surveys Tutorials,pp. 1801–1819, April 2014.
    [2] B. Kaufman and B. Aazhang,” Cellular networks with an overlaid device to device network,” in Asilomar Conference on Signals, Systems and Computers, pp. 1537–1541, Oct 2008.
    [3] Shahid Mumtaz, Kazi Mohammed Saidul Huq, and Jonathan Rodriguez,” Direct mobile-to- mobile communication: Paradigm for 5G,” IEEE Wireless Communications, pp. 14-23, Oct 2014.
    [4] Jeffrey G. Andrews, Stefano Buzzi, Wan Choi, Stephen V. Hanly, Angel Lozano, and Anthony C. K. Soo,” What will 5G be ?,” IEEE Journal on Selected Areas in Communications, pp. 1065- 1082, June 2014.
    [5] Mehyar Najla, David Gesbert, Zdenek Becvar, and Pavel Mach,” Machine Learning for Power Control in D2D Communication based on Cellular Channel Gains,” 2019 IEEE Globecom Workshops, March 2020.
    [6] Chenfei Gao, Jian Tang, Xiang Sheng, Weiyi Zhang, Shihong Zou, and Mohsen Guizani,” En- abling Green Wireless Networking With Device-to-Device Links: A Joint Optimization Ap- proach,” IEEE Transactions on Wireless Communications, pp.2770-2779, December 2015.
    [7] Yung-Fa Huang; Tan-Hsu Tan, Neng-Chung Wang, Young-Long Chen, and Yu-Ling Li,” Re- source Allocation For D2D Communications With A Novel Distributed Q-Learning Algorithm In Heterogeneous Networks,” 2018 International Conference on Machine Learning and Cyber- netics (ICMLC), pp.533-537, November 2018.
    [8] Dan Wang, Hao Qin, Bin Song, Xiaojiang Du, and Mohsen Guizani,” Resource Allocation in Information-Centric Wireless Networking With D2D-Enabled MEC: A Deep Reinforcement Learning Approach,” IEEE Access, pp. 114935-114944, August 2019.
    [9] Woongsup Lee, Minhoe Kim, and Dong-Ho Cho,” Deep Learning Based Transmit Power Con- trol in Underlaid Device-to-Device Communication,” IEEE Systems Journal, pp. 2551-2554, September 2018.
    [10] Amal Ali Algedir and Hazem H. Refai,” Energy Efficiency Optimization and Dynamic Mode Selection Algorithms for D2D Communication Under HetNet in Downlink Reuse,” IEEE Ac- cess, pp. 95251-95265, May 2020.
    [11] Khoi Khac Nguyen, Trung Q. Duong, Ngo Anh Vien, Nhien-An Le-Khac, and Long D. Nguyen,” Distributed Deep Deterministic Policy Gradient for Power Allocation Control in D2D- Based V2V Communications,” IEEE Access, pp.164533-164543, November 2019.
    [12] Ying Luo, Min Zeng, and Hong Jiang,” Learning to Tradeoff Between Energy Efficiency and Delay in Energy Harvesting-Powered D2D Communication: A Distributed Experience-Sharing Algorithm,” IEEE Internet of Things Journal, pp. 5585-5594, March 2019.
    [13] XueWang,TaoJin,LiangshuaiHu,andZhihongQian,”Energy-EfficientPowerAllocationand Q-Learning-Based Relay Selection for Relay-Aided D2D Communication,” IEEE Transactions on Vehicular Technology, pp. 6452-6462, April 2020.
    [14] A.K.Jain,R.P.W.DuinandJianchangMao,”Statisticalpatternrecognition:areview,”inIEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 4-37, Jan. 2000, doi: 10.1109/34.824819.
    [15] M. Usama et al., ”Unsupervised Machine Learning for Networking: Techniques, Applications and Research Challenges,” in IEEE Access, vol. 7, pp. 65579-65615, 2019, doi: 10.1109/AC- CESS.2019.2916648.
    [16] AndreaOrtiz,ArashAsadi,MaxEngelhardt,AnjaKlein,andMatthiasHollick,”CBMoS:Com- binatorial Bandit Learning for Mode Selection and Resource Allocation in D2D Systems,” IEEE Journal on Selected Areas in Communications, pp. 2225 - 2238, August 2019.
    [17] T. -W. Ban, ”An Autonomous Transmission Scheme Using Dueling DQN for D2D Communi- cation Networks,” in IEEE Transactions on Vehicular Technology, vol. 69, no. 12, pp. 16348- 16352, Dec. 2020, doi: 10.1109/TVT.2020.3041458.
    [18] Federico Librino; Giorgio Quer,” Channel, Mode and Power Optimization for Non-Orthogonal D2D Communications: A Hybrid Approach,” IEEE Transactions on Cognitive Communications and Networking, pp. 657-668, December 2019.
    [19] S.Mumtaz and J. Rodriguez, ”Smart Device to Smart Device Communication: Architecture and Security Issues,” J.Netw.Comput. Appl., vol. 78, pp.17-18, Jan.2017
    [20] Woongsup Lee, Minhoe Kim, and Dong-Ho Cho,” Transmit Power Control Using Deep Neu- ral Network for Underlay Device-to-Device Communication,” IEEE Wireless Communications Letters, pp. 141-144, August 2018.
    [21] Mehyar Najla, Pavel Mach, and Zdenek Becvar ” Deep Learning for Selection Between RF and VLC Bands in D2D Communications,” IEEE Wireless Communications Letters, pp. 1763 - 1767, June 2020.
    [22] Z. Ji, A. K. Kiani, Z. Qin and R. Ahmad, ”Power Optimization in Device-to-Device Communications: A Deep Reinforcement Learning Approach With Dynamic Reward,” in IEEE Wireless Communications Letters, vol. 10, no. 3, pp. 508-511, March 2021, doi: 10.1109/LWC.2020.3035898.
    [23] TaoZhang,KunZhu,andJunhuaWang,”Energy-EfficientModeSelectionandResourceAllo- cation for D2D-Enabled Heterogeneous Networks: A Deep Reinforcement Learning Approach, ” IEEE Transactions on Wireless Communications, pp.1175 - 1187, February 2021.
    [24] F. Jameel, Z. Hamid, F. Jabeen, S. Zeadally and M. A. Javed, ”A Survey of Device-to-Device Communications: Research Issues and Challenges,” in IEEE Communications Surveys and Tu- torials, vol. 20, no. 3, pp. 2133-2168, thirdquarter 2018, doi: 10.1109/COMST.2018.2828120.
    [25] B. Gu, X. Zhang, Z. Lin and M. Alazab, ”Deep Multiagent Reinforcement-Learning-Based Resource Allocation for Internet of Controllable Things,” in IEEE Internet of Things Journal, vol. 8, no. 5, pp. 3066-3074, 1 March1, 2021, doi: 10.1109/JIOT.2020.3023111.
    [26] J. Tan, Y. -C. Liang, L. Zhang and G. Feng, ”Deep Reinforcement Learning for Joint Channel Selection and Power Control in D2D Networks,” in IEEE Transactions on Wireless Communi- cations, vol. 20, no. 2, pp. 1363-1378, Feb. 2021.
    [27] Z. Li and C. Guo, ”Multi-Agent Deep Reinforcement Learning Based Spectrum Allocation for D2D Underlay Communications,” in IEEE Transactions on Vehicular Technology, vol. 69, no. 2, pp. 1828-1840, Feb. 2020.
    [28] X. Zhang, M. Peng, S. Yan and Y. Sun, ”Deep-Reinforcement-Learning-Based Mode Selec- tion and Resource Allocation for Cellular V2X Communications,” in IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6380-6391, July 2020, doi: 10.1109/JIOT.2019.2962715.
    [29] F. Tang, Y. Zhou and N. Kato, ”Deep Reinforcement Learning for Dynamic Uplink/Downlink Resource Allocation in High Mobility 5G HetNet,” in IEEE Journal on Selected Areas in Com- munications, vol. 38, no. 12, pp. 2773-2782, Dec. 2020, doi: 10.1109/JSAC.2020.3005495.
    [30] J. Jang and H. J. Yang, ”Deep Reinforcement Learning-Based Resource Allocation and Power Control in Small Cells With Limited Information Exchange,” in IEEE Transactions on Vehicular Technology, vol. 69, no. 11, pp. 13768-13783, Nov. 2020, doi: 10.1109/TVT.2020.3027013.
    [31] H. Liang, X. Zhang, J. Zhang, Q. Li, S. Zhou and L. Zhao, ”A Novel Adaptive Resource Al- location Model Based on SMDP and Reinforcement Learning Algorithm in Vehicular Cloud System,” in IEEE Transactions on Vehicular Technology, vol. 68, no. 10, pp. 10018-10029, Oct. 2019, doi: 10.1109/TVT.2019.2937842.
    [32] D.Kwon,J.Kim,D.A.MohaisenandW.Lee,”Self-adaptivepowercontrolwithdeepreinforce- ment learning for millimeter-wave Internet-of-vehicles video caching,” in Journal of Communi- cations and Networks, vol. 22, no. 4, pp. 326-337, Aug. 2020, doi: 10.1109/JCN.2020.000022.
    [33] G. Liu, S. Salehi, C. -C. Shen and L. J. Cimini, ”TDD Massive MISO Piloting Strategy With User Information: A Reinforcement Learning Approach,” in IEEE Wireless Communications Letters, vol. 10, no. 2, pp. 349-352, Feb. 2021, doi: 10.1109/LWC.2020.3030728.
    [34] M. Najla, Z. Becvar, P. Mach and D. Gesbert, ”Predicting Device-to-Device Channels From Cellular Channel Measurements: A Learning Approach,” in IEEE Transactions on Wireless Communications, vol. 19, no. 11, pp. 7124-7138, Nov. 2020, doi: 10.1109/TWC.2020.3008303.
    [35] X. Zhang, M. Peng, S. Yan and Y. Sun, ”Deep-Reinforcement-Learning-Based Mode Selec- tion and Resource Allocation for Cellular V2X Communications,” in IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6380-6391, July 2020, doi: 10.1109/JIOT.2019.2962715.
    [36] H. Zhou, T. Wu, H. Zhang and J. Wu, ”Incentive-Driven Deep Reinforcement Learning for Content Caching and D2D Offloading,” in IEEE Journal on Selected Areas in Communications, vol. 39, no. 8, pp. 2445-2460, Aug. 2021, doi: 10.1109/JSAC.2021.3087232.
    [37] Kuang-Chih, Chou,”Power Allocation in downlink Non-Orthogonal Multiple Acess based on Artifical intelligence,” Aug. 2021.
    [38] Z. Ji, A. K. Kiani, Z. Qin and R. Ahmad, ”Power Optimization in Device-to-Device Communications: A Deep Reinforcement Learning Approach With Dynamic Reward,” in IEEE Wireless Communications Letters, vol. 10, no. 3, pp. 508-511, March 2021, doi: 10.1109/LWC.2020.3035898.
    [39] J. Huang, Y. Yang, G. He, Y. Xiao and J. Liu, ”Deep Reinforcement Learning-Based Dynamic Spectrum Access for D2D Communication Underlay Cellular Networks,” in IEEE Communica- tions Letters, vol. 25, no. 8, pp. 2614-2618, Aug. 2021, doi: 10.1109/LCOMM.2021.3079920.
    [40] J. Huang, Y. Yang, Z. Gao, D. He and D. W. K. Ng, ”Dynamic Spectrum Access for D2D- Enabled Internet-of-Things: A Deep Reinforcement Learning Approach,” in IEEE Internet of Things Journal, March. 2022, doi: 10.1109/JIOT.2022.3160197.
    [41] M. F. Pop and N. C. Beaulieu, ”Limitations of sum-of-sinusoids fading channel simulators,” in IEEE Transactions on Communications, vol. 49, no. 4, pp. 699-708, April 2001, doi: 10.1109/26.917776.
    [42] ”StudyonLTEDevicetoDeviceProxmityServices;RadioAspects,”3GPPTR36.843,release 12, Mar. 2014.
    [43] M. V. da Silva, S. Montejo-Sa ́nchez, R. D. Souza, H. Alves and T. Abra ̃o, ”D2D Assisted Q- Learning Random Access for NOMA-Based MTC Networks,” in IEEE Access, vol. 10, pp. 30694-30706, 2022, doi: 10.1109/ACCESS.2022.3160156
    [44] X.Wang,T.Jin,L.HuandZ.Qian,”Energy-EfficientPowerAllocationandQ-Learning-Based Relay Selection for Relay-Aided D2D Communication,” in IEEE Transactions on Vehicular Technology, vol. 69, no. 6, pp. 6452-6462, June 2020, doi: 10.1109/TVT.2020.2985873.
    [45] N. C. Luong et al., ”Applications of Deep Reinforcement Learning in Communications and Networking: A Survey,” in IEEE Communications Surveys and Tutorials, vol. 21, no. 4, pp. 3133-3174, Fourthquarter 2019, doi: 10.1109/COMST.2019.2916583.
    [46] Volodymyr, Mnih Koray, Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra and Martin Riedmiller., ”Playing Atari with Deep Reinforcement Learning.
    [47] Mofan,”UsingPytorchtoimplementdeepneuralnetworksforregressionfittingofdatascatters, ” https://mofanpy.com/tutorials/machine-learning/torch/regression.
    [48] H. Xiang, Y. Yang, G. He, J. Huang and D. He, ”Multi-Agent Deep Reinforcement Learning-Based Power Control and Resource Allocation for D2D Communications,” in IEEE Wireless Communications Letters, vol. 11, no. 8, pp. 1659-1663, Aug. 2022, doi: 10.1109/LWC.2022.3170998.
    [49] I. Budhiraja, N. Kumar and S. Tyagi, ”Deep-Reinforcement-Learning-Based Proportional Fair Scheduling Control Scheme for Underlay D2D Communication,” in IEEE Internet of Things Journal, vol. 8, no. 5, pp. 3143-3156, 1 March1, 2021, doi: 10.1109/JIOT.2020.3014926.
    [50] Y. Yuan, Z. Li, Z. Liu, Y. Yang and X. Guan, ”Double Deep Q-Network Based Distributed Resource Matching Algorithm for D2D Communication,” in IEEE Transactions on Vehicular Technology, vol. 71, no. 1, pp. 984-993, Jan. 2022, doi: 10.1109/TVT.2021.3130159.
    [51] D.RonandJ.-R.Lee,”DRL-BasedSum-RateMaximizationinD2DCommunicationUnderlaid Uplink Cellular Networks,” in IEEE Transactions on Vehicular Technology, vol. 70, no. 10, pp. 11121-11126, Oct. 2021, doi: 10.1109/TVT.2021.3106398.
    254
    [52] X. Xu, Y. Zhang, Z. Sun, Y. Hong and X. Tao, ”Analytical Modeling of Mode Selection for Moving D2D-Enabled Cellular Networks,” in IEEE Communications Letters, vol. 20, no. 6, pp. 1203-1206, June 2016, doi: 10.1109/LCOMM.2016.2552171.
    [53] M. Waqas et al., ”A Comprehensive Survey on Mobility-Aware D2D Communications: Princi- ples, Practice and Challenges,” in IEEE Communications Surveys and Tutorials, vol. 22, no. 3, pp. 1863-1886, thirdquarter 2020, doi: 10.1109/COMST.2019.2923708.
    [54] W. Li, X. Qin, Z. Jia, J. Bi and X. Li, ”Resource Sharing for Cellular-Assisted D2D Communi- cations With Imperfect CSI: A Many-to-Many Strategy,” in IEEE Systems Journal, vol. 16, no. 3, pp. 4454-4465, Sept. 2022, doi: 10.1109/JSYST.2022.3145398.
    [55] D.RonandJ.-R.Lee,”DRL-BasedSum-RateMaximizationinD2DCommunicationUnderlaid Uplink Cellular Networks,” in IEEE Transactions on Vehicular Technology, vol. 70, no. 10, pp. 11121-11126, Oct. 2021, doi: 10.1109/TVT.2021.3106398.
    [56] Q.Wu,G.Y.Li,W.ChenandD.W.K.Ng,”Energy-EfficientD2DOverlayingCommunications With Spectrum-Power Trading,” in IEEE Transactions on Wireless Communications, vol. 16, no. 7, pp. 4404-4419, July 2017, doi: 10.1109/TWC.2017.2698032.
    [57] B. V. R. Gorantla and N. B. Mehta, ”Interplay Between Interference-Aware Resource Allo- cation Algorithm Design, CSI, and Feedback in Underlay D2D Networks,” in IEEE Trans- actions on Wireless Communications, vol. 21, no. 5, pp. 3452-3463, May 2022, doi: 10.1109/TWC.2021.3121874.
    [58] L. Wang, H. Tang, H. Wu and G. L. Stu ̈ber, ”Resource Allocation for D2D Communications Underlay in Rayleigh Fading Channels,” in IEEE Transactions on Vehicular Technology, vol. 66, no. 2, pp. 1159-1170, Feb. 2017, doi: 10.1109/TVT.2016.2553124.
    [59] W. Lee and K. Lee, ”Resource Allocation Scheme for Guarantee of QoS in D2D Communi- cations Using Deep Neural Network,” in IEEE Communications Letters, vol. 25, no. 3, pp. 887-891, March 2021, doi: 10.1109/LCOMM.2020.3042490.

    無法下載圖示 校內：2028-08-22公開
    校外：2028-08-22公開
    電子論文尚未授權公開，紙本請查館藏目錄
    QR CODE