資料異質性是聯邦學習的主要挑戰之一，非獨立同分佈的資料分佈使得聯邦學習的訓練難以收斂，為解決此挑戰，許多研究從不同角度提出各種辦法，如用戶選擇、分群方法、聚合策略等。階層式聯邦學習藉由分群方法展現處理資料異質性的優越學習性能，階層式聯邦學習首先根據邊緣裝置的標籤分佈進行分群，使得具有相似資料分布的邊緣裝置會被歸納入同一群內，接著使用兩個階段的模型聚合，儘管每個群內的資料分佈相似，但群與群之間的資料分佈仍存在異質性。為了解決上述問題，本篇論文提出基於階層式聯邦學習架構之選擇性階層式聯邦學習，透過選擇性地執行階層式聯邦學習並引入微調模型步驟以解決群與群之間的資料異質性問題，此方法雖可加速模型收斂但也使得訓練時間急遽提升。為改善訓練時間增加的問題，本論文提出一個基於最小包圍圓中心(Minimum Enclosing Circle, MEC)的模型聚合策略，取代原本的微調模型步驟以提供公平的聚合結果，並更進一步地提出基於最小包圍圓的階層式聯邦學習。根據大量的實驗結果顯示，相較於其他經典聯邦學習演算法，本論文提出之最小包圍圓的階層式聯邦學習方法具有優越的學習性能，此外，基於最小包圓圓的階層式聯邦學習更進一步地提高模型收斂速度，相較於先前提出的選擇性階層式聯邦學習有著更好的學習效率。

Data heterogeneity is one of the key challenges in federated learning (FL). The non-independent and identically distributed (non-IID) data distribution leads to the difficulty in training convergence. To address this challenge, various approaches have been proposed from different perspectives, such as client selection, clustering approach, and aggregation strategy. Hierarchical federated learning (HFL) demonstrates a superior learning performance in dealing with data heterogeneity by clustering. On the basis of clients’ label distribution, HFL groups devices with a similar data distribution into the same cluster and then introduces two-phase model aggregation. Although data distribution in each cluster is similar, it is still heterogeneous between clusters. To address inter-cluster data heterogeneity, the thesis proposed a new approach based on Selective Hierarchical Federated Learning (SHFL). SHFL enhances the existing HFL framework by selectively performing HFL training and incorporating a model fine-tuning step, which better handles inter-cluster data heterogeneity and improves overall learning performance. Although the SHFL accelerates model convergence, it increases training time. To mitigate the high training time, the thesis proposed a novel aggregation strategy based on the Minimum Enclosing Circle (MEC). The proposed MEC-based HFL replaces the fine-tuning step with the center of MEC to provide a fair aggregation result. Experimental results showed that the proposed MEC-based HFL yields a superior learning performance compared to other classic FL algorithms. Moreover, MEC-based HFL further improves model convergence speed and achieves better training efficiency than the previously proposed SHFL.

[1] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE Communications Surveys Tutorials, vol. 19, no. 4, pp. 2322–2358, 2017.
[2] H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 54, pp. 1273–1282, April 2017.
[3] M. Paulik, M. S. Seigel, H. Mason, D. Telaar, J. Kluivers, R. C. van Dalen, C. W. Lau, L. Carlson, F. Granqvist, C. Vandevelde, S. Agarwal, J. Freudiger, A. Byde, A. Bhowmick, G. Kapoor, S. Beaumont, ́A. Cahill, D. Hughes, O. Javidbakht, F. Dong, R. Rishi, and S. Hung, “Federated evaluation and tuning for on-device personalization: System design & applications,” ArXiv, vol. abs/2102.08503, 2021.
[4] S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K. Chan, “Adaptive federated learning in resource constrained edge computing systems,” IEEE journal on selected areas in communications, vol. 37, no. 6, pp. 1205–1221, 2019.
[5] J. Ren, W. Ni, and H. Tian, “Toward communication-learning trade-off for federated learning at the network edge,” IEEE Communications Letters, vol. 26, no. 8, pp. 1858–1862, 2022.
[6] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, April 2020.
[7] L. Liu, J. Zhang, S. H. Song, and K. B. Letaief, "Client-Edge-Cloud Hierarchical Federated Learning," 2020 IEEE International Conference on Communications (ICC), Dublin, Ireland, June 7-11, 2020, pp. 1–6.
[8] A. Z. Tan, H. Yu, L. Cui, and Q. Yang, “Towards personalized federated learning,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 12, pp. 9587–9603, 2023.
[9] S. Luo, X. Chen, Q. Wu, Z. Zhou, and S. Yu, “Hfel: Joint edge association and resource allocation for cost-efficient hierarchical federated edge learning,” IEEE Transactions on Wireless Communications, vol. 19, no. 10, pp. 6535–6548, 2020.
[10] H. Chai, S. Leng, Y. Chen, and K. Zhang, “A hierarchical blockchain-enabled federated learning algorithm for knowledge sharing in internet of vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 7, pp. 3975–3986, 2021.
[11] M. Ankerst, M. M. Breunig, H.-P. Kriegel, and J. Sander, “Optics: Ordering points to identify the clustering structure,” ACM Sigmod record, vol. 28, no. 2, pp. 49–60, June 1999.
[12] M. Ester, H.-P. Kriegel, J. Sander, X. Xu, “A density-based algorithm for discovering clusters in large spatial databases with noise,” Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, Portland, Oregon, August 2-4, 1996, pp. 226–231.
[13] J. Reyes, L. Di Jorio, C. Low-Kam, and M. Kersten-Oertel, “Precision-weighted federated learning,” arXiv preprint arXiv:2107.09627, 2021.
[14] K. Pillutla, S. M. Kakade, and Z. Harchaoui, “Robust aggregation for federated learning,” IEEE Transactions on Signal Processing, vol. 70, pp. 1142–1154, 2022.
[15] J. Xu, S. Wang, L. Wang, and A. C.-C. Yao, “Fedcm: Federated learning with client-level momentum,” arXiv preprint arXiv:2106.10874, 2021.
[16] X. Ma, J. Zhang, S. Guo, and W. Xu, “Layer-wised model aggregation for personalized federated learning,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2022, pp. 10092–10101.
[17] S. Chen, C. Shen, L. Zhang, and Y. Tang, “Dynamic aggregation for heterogeneous quantization in federated learning,” IEEE Transactions on Wireless Communications, vol. 20, no. 10, pp. 6804–6819, 2021.
[18] K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. S. Quek, and H. Vincent Poor, “Federated learning with differential privacy: Algorithms and performance analysis,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 3454–3469, 2020.
[19] James Joseph Sylvester. "A question in the geometry of situation", Quarterly Journal of Mathematics, 1: 79, 1857.
[20] D. Cheng, X. Hu, and C. Martin, “On the smallest enclosing balls,” Communications in Information and Systems, vol. 6, 2006.
[21] T. Nishio and R. Yonetani, “Client selection for federated learning with heterogeneous resources in mobile edge,” 2019 IEEE International Conference on Communications (ICC), Shanghai, China, May 20-24, 2019, pp. 1–7.
[22] S. Abdulrahman, H. Tout, A. Mourad, and C. Talhi, “Fedmccs: Multicriteria client selection model for optimal iot federated learning,” IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4723–4735, March 2021.
[23] Y. Jee Cho, J. Wang, and G. Joshi, "Towards understanding biased client selection in federated learning," Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (AISTATS), vol. 151, pp.10351–10375, March 2022.
[24] R. Balakrishnan, T. Li, T. Zhou, N. Himayat, V. Smith, and J. Bilmes, “Diverse client selection for federated learning via submodular maximization,” International Conference on Learning Representations, 2022.
[25] X. Wang, Y. Chen, Y. Li, X. Liao, H. Jin, and B. Li, “Fedmos: Taming client drift in federated learning with double momentum and adaptive selection,” IEEE INFOCOM 2023 - IEEE Conference on Computer Communications, 2023, pp. 1–10.
[26] F. Lai, X. Zhu, H. V. Madhyastha, and M. Chowdhury, “Oort: Efficient federated learning via guided participant selection,” 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21), Jul. 2021, pp. 19–35.
[27] C. Briggs, Z. Fan, and P. Andras, “Federated learning with hierarchical clustering of local updates to improve training on non-iid data,” 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, July 19-24, 2020, pp. 1-9.
[28] H. Lee and D. Seo, “Fedlc: Optimizing federated learning in non-iid data via label-wise clustering,” IEEE Access, vol. 11, pp. 42082–42095, 2023.
[29] Fan-Hsun Tseng and Yu-Teng Lai. "SHFL: Selective Hierarchical Federated Learning for Non-IID Data Distribution." 2024 IEEE 99th vehicular technology conference (VTC Spring), Singapore, June 24-27, 2024, pp. 1-6.
[30] H. Zhang, Q. Cheng, and X. Chen, "Aggregation Mechanisms in Federated Learning for Enhancing Robotic Visual Obstacle Avoidance," 2023 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence (RICAI), Hangzhou, China, 2023, pp. 143-148.
[31] V. Hassija, V. Chawla, V. Chamola, and B. Sikdar, “Incentivization and aggregation schemes for federated learning applications,” IEEE Transactions on Machine Learning in Communications and Networking, vol. 1, pp. 185–196, 2023.
[32] A. B. Mansour, G. Carenini, A. Duplessis and D. Naccache, "Federated Learning Aggregation: New Robust Algorithms with Guarantees," 2022 21st IEEE International Conference on Machine Learning and Applications (ICMLA), Nassau, Bahamas, 2022, pp. 721-726.
[33] J. Lee and H. Ko, “Energy and distribution-aware cooperative clustering algorithm in internet of things (iot)-based federated learning,” IEEE Transactions on Vehicular Technology, vol. 72, no. 10, pp. 13799–13804, October 2023.
[34] Welzl, Emo. "Smallest enclosing disks (balls and ellipsoids)." New Results and New Trends in Computer Science: Graz, Austria, June 20–21, 1991 Proceedings. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005.
[35] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE, vol. 86, no. 11, pp. 2278–2324, Nov. 1998.
[36] S. L. Smith, P.-J. Kindermans, and Q. V. Le, “Don’t decay the learning rate, increase the batch size,” International Conference on Learning Representations (ICLR), Vancouver, BC, Canada, April 30–May 3, 2018.