隨著近年來人工智慧技術的迅速發展，許多研究機構和業界投入大量資源來發展生技醫療、工業4.0、智慧城市等人工智慧相關應用領域的產品與服務，並陸續開發出多種先進技術，而在這些中，電腦視覺關鍵技術扮演著相當重要的角色。除此之外，大家對個人資料隱私越來越受到重視，各國也對資料保護制定出許多相關的法規來限制，而聯邦式學習就是用來達成個人資料隱私保護的分散式機器學習方法。如何在不傳送個人資料到伺服器，並彙整多個客戶端的資訊來獲得好的模型，在聯邦式學習是相當重要的問題。在現實生活中，客戶端的資料會因習慣或偏好不同，導致客戶端間的資料分佈有所差異，例如，客戶端使用手機拍照，而因喜好不同，有的客戶端手機的相簿中都是狗，或者都是貓，這種非獨立同分布問題則會直接影響模型訓練的結果。目前有許多研究嘗試解決這個問題，但並沒有突破性的進展。現有的方法為了衡量其是否可以有效的整合客戶端間的資訊，他們設計的實驗會將資料集平分給所有客戶端，在這種情況下，當客戶端數量變多時，客戶端所擁有的資料量將會遞減，整體學習到的資料量不變，這樣的實驗設計無法表現出聯邦式學習的另一個特性－越多客戶端參與，資料越多，故這樣的實驗設計完全不符合聯邦式學習的應用情境。此外，目前的方法都會假設客戶端有大量的標記資料，但在現實生活中人們根本沒有興趣標記大量資料，真實情況下標籤資料是相當稀少的，故假設參與者有足夠多的標記資料是不符合實際情況的。
在本論文中，我們提出一種具高度擴充性與強非獨立同分布適應力之少樣本聯邦式學習方法，使得在少樣本情況下非獨立同分布準確度可以達到與獨立同分布差不多的準確度。並提出更符合真實聯邦式學習情境的實驗設計。我們提出新的聯邦式學習流程，結合了客戶端資料特徵向量傳送、模型平均以及伺服器訓練來解決非獨立同分布的問題，並能夠確切整合大量客戶端之資訊。我們的方法結合了度量學習與情景訓練，讓我們的模型可以適用在少樣本情況下。而我們的論文提出的實驗設計更符合真實的聯邦式學習情境，更真實的評估了真實世界人們可能標記的資料量，並讓聯邦式學習情境之特點得以完全呈現，能夠做出更為準確的評估。
最後，我們提供實驗結果來驗證我們所提出方法的效能。而為了與先前的方法比較，我們不僅實驗於新設計的實驗設計，我們同時也採用了與前人一樣的實驗設計來進行實驗，在傳統的實驗設計中，本論文所提出的方法不論是在多個客戶端或是非獨立同分布情況下，都可以得到相當優異的準確度。在新的實驗設計下真實的模擬了客戶端對於標註資料的情況，我們的方法能夠完善的整合所有客戶端的資訊，而於巨量使用者時更是能夠得到非常出色的成果。

With the rapid development of artificial intelligence (AI) in recent years, many research institutes and companies invest lots of resources to develop AI-enabled technologies for biotechnology, industry 4.0 and smart city, etc. Among these technologies, computer vision plays a very significant role. Additionally, people pay more attention to the privacy issue, and various countries lay down the laws and regulations to protect personal data. Federated learning is a decentralized machine learning technique proposed to protect personal data privacy. In federated learning, it is a vital problem to obtain a well-trained model by gathering the information from multiple clients without accessing the personal data of clients. In the real world, the data of clients are very different because of various habits or preferences. It leads to the data distributions of clients more diverse. For example, the clients take the pictures by smartphone, but due to their different preferences, the pictures that one owns might all be dogs while another might all be cats. This non-independent and identically distributed (Non-IID) problem will directly impact the results. A lot of researches try to solve this problem, but no breakthrough progress has been made. In the existing methods, to evaluate whether they can effectively integrate the information between clients, their experiment design will divide the dataset to the clients equally. In this situation, when the number of clients increases, the amount of data of the client will decrease, and the whole data quantity will remain the same. This experiment design cannot show the characteristic of federated learning that the more clients participate, the more data server will receive. Due to this problem, the design of the experiment is incompatible with the federated learning scenario. Furthermore, in contrast to the assumption of the existing methods that the clients have a lot of labeled data, people are not interested in labeling a great amount of data, leading to the scarcity of labeled data in reality. Hence, the hypothesis that all the clients participate in federated learning having enough labeled data does not meet the real situation.
In this thesis, a few-shot federated learning method with strong Non-IID adaptability and high-scalability is proposed. It could gain almost the same accuracy between IID and Non-IID in the case of few samples. We propose a novel federated learning process that combines the feature vectors transmitting, model averaging, and server training and distributing to solve the Non-IID problem. In addition, our method can integrate the information of a huge number of clients exactly with the new training process. We also combine metric learning and episodic training in our method to apply our model in the few-shot situation. Furthermore, our paper proposes a novel experiment design to fit the realistic federated learning scenario that can evaluate the amount of the labeled data in the real world. It fully presents the characteristics of federated learning and makes a more accurate estimation.
Finally, we provide experimental results to verify the effectiveness of the proposed method. To compare with the previous methods, the novel experiment design and the traditional experiment design of the existing methods are both conducted to verify the effectiveness of the proposed method. In the traditional experiment design, the proposed method gets excellent accuracy regardless of the situations of multiple clients and Non-IID. In the novel experiment design, we simulate the scenario of the labeled data of clients, and our method can perfectly integrate all the information of clients and get outstanding results in a huge number of clients.

[BAG 19] E. Bagdasaryan, O. Poursaeed, and V. Shmatikov, “Differential Privacy Has Disparate Impact on Model Accuracy”, Advances in Neural Information Processing Systems, 2019.
[BUC 06] C. Buciluǎ, R. Caruana, and A. Niculescu-Mizil, “Model compression”, Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 535–541, 2006.
[CHE 18] X. Chen, J. Ji, C. Luo, W. Lia, and P. Li, “When Machine Learning Meets Blockchain: A Decentralized”, IEEE International Conference on Big Data, pp. 117-1187, 2018.
[CHE 19] K. Cheng, T. Fan, Y. Jin, Y. Liu, T. Chen, and Q. Yang, “SecureBoost: A lossless federated”, arXiv preprint arXiv: 1901.08755, 2019.
[CUR 09] J. Curto, and J. Pinto, “The coefficient of variation asymptotic distribution in the case of non-iid random variables”, Journal of Applied Statistics, 2009.
[DEA 08] J. Dean and S. Ghemawat, “MapReduce: simplified data processing on large clusters”, Communications of the ACM, pp. 107–113, 2008.
[DEN 09] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. FeiFei, “ImageNet: A
large-scale hierarchical image database”, IEEE Conference on Computer Vision and Pattern Recognition, 2009.
[DEN 12] J. Deng, A. Berg, S. Satheesh, H.Su, A. Khosla, and L. FeiFei, ”ImageNet Large Scale Visual Recognition Competition”, 2012.
[DON 21] K. Donahue, and J. Kleinberg, “Optimality and Stability in Federated Learning: A Game-theoretic Approach”, arXiv preprint arXiv: 2106.09580, 2021.
[FAC 21] Facebook–Cambridge Analytica data scandal
https://en.wikipedia.org/wiki/Facebook%E2%80%93Cambridge_Analytica_data_scandal
[FAD 15] Facebook AI Director Yann LeCun on His Quest to Unleash Deep Learning and Make Machines Smarter https://spectrum.ieee.org/automaton/robotics/artificial-intelligence/facebook-ai-director-yann-lecun-on-deep-learning/
[FAN 21] S. H. Fan, M. H. Lin, J. Y. Jiang and Y. H. Kuo, “A Few-Shot Learning Method Using Feature Reparameterization and Dual-Distance Metric Learning for Object Re-identification”, IEEE ACCESS, pp. 133650 - 133662, 2021.
[FAN 21] C. Fan, and J. Huang, “Federated Few-Shot Learning with Adversarial Learning”, arXiv preprint arXiv: 2104.00365, 2021.
[FIN 17] C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks”, In International Conference on Machine Learning, 2017.
[GIR 15] R. Girshick, “Fast R-CNN”, IEEE International Conference on Computer Vision, pp. 1440-1448, 2015.
[HAR 18] A. Hard, K. Rao, R. Mathews, S. Ramaswamy, F. Beaufays, S. Augenstein, H. Eichner, C. Kiddon, and D. Ramage, “Federated Learning for Mobile Keyboard Prediction”, arXiv preprint arXiv:1811.03604, 2018.
[HE 16] K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770-778, 2016.
[HE 20] C. He, M. Annavaram, S. Avestimehrh, “Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge”, In Advances in Neural Information Processing Systems, 2020.
[HEL 20] H. Hellström, J. M. B. da Silva Jr., V. Fodor, and C. Fischione, “Wireless
for Machine Learning”, arXiv preprint arXiv:2008.13492, 2020.
[HIN 15] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network”, arXiv preprint arXiv:1503.02531, 2015.
[HUA 19] X. Huang, J. Guan, B. Zhang, S. Qi, X. Wang, and Q. Liao, “Differentially Private Convolutional Neural Networks with Adaptive Gradient Descent”, IEEE Fourth International Conference on Data Science in Cyberspace, pp. 642-648, 2019.
[HUA 20] X. Huang, Y. Ding, Z. L. Jiang, S. Qi, X. Wang, and Q. Liao, “DP-FL: a novel differentially private federated learning framework for the unbalanced data”, World Wide Web, 2020.
[JAM 21] H. Jamali-Rad, M. Abdizadeh, and A. Szabo, “Federated Learning with Taskonomy for Non-IID Data”, arXiv preprint arXiv: 2103.15947, 2021.
[KAN 20] R. Kanagavelu, Z. Li, J. Samsudin, Y. Yang, F. Yang, R. S. M. Goh, M. Cheah, P. Wiwatphonthana, K. Akkarajitsakul, and S. Wang, “Two-Phase Multi-Party Computation Enabled Privacy-Preserving Federated Learning” IEEE/ACM International Symposium on Cluster, Cloud and Internet Computing (CCGRID), 2020.
[KAR 20] S. P. Karimireddy, S. Kale, M. Mohri, S. J. Reddi, S. U. Stich, and A. T. Suresh, “SCAFFOLD: Stochastic Controlled Averaging for Federated Learning”, Proceedings of the 37th International Conference on Machine Learning (PMLR), 2020.
[KON 16] J. Konečny, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated learning: Strategies for improving communication efficiency”, NIPS Workshop on Private Multi-Party Machine Learning, 2016.
[KRI 12] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ¨ImageNet Classification with Deep Convolutional Neural Networks”, Conference and Workshop on Neural Information Processing Systems, pp. 1097-1105, 2012.
[LEC 98] Y. LeCun, L. Bottou, and Y. Bengio, “Gradient-based learning applied to document recognition”, Proceedings of the IEEE, 1998.
[LI 21] C. Li, D.Niu, B. Jiang, X. Zuo, and J.Yang, “Meta-HAR: Federated Representation Learning for Human Activity Recognition”, Proceedings of the Web Conference, pp. 912–922, 2021.
[Luo 19] J. Luo, X. Wu, Y. Luo, A. Huang, Y. Huang, Y. Liu, and Q. Yang. “Real-world image datasets for federated learning”, International Workshop on Federated Learning for Data Privacy and Confidentiality in Conjunction with NeurIPS (FL-NeurIPS), 2019.
[MCM 17] H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data”, In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
[MIC 21] U. Michieli, and M. Ozay, “Prototype Guided Federated Learning of Visual Feature Representations”, arXiv preprint arXiv: 2105.08982, 2021.
[MUC 13] L. Muchnik, S. Pei, L. C. Parra, S. D.S. Reis, J. S. Andrade Jr., S. Havlin, and H. A. Makse, “Origins of power-law degree distribution in the heterogeneity of human activity in social networks”, Scientific Reports, 2013.
[ROT 20] D. Rothchild, A. Panda, E. Ullah, N. Ivkin, I. Stoica, V. Braverman, J. Gonzalez, and R. Arora, “FetchSGD: Communication-Efficient Federated Learning with Sketching”, International Conference on Machine Learning, pp. 8253–8265, 2020.
[SAI 13]T. N. Sainath, A. R. Mohamed, B. Kingsbury, and B. Ramabhadran, “Deep convolutional neural networks for LVCSR”, International Conference on Acoustics, Speech and Signal Processing, 2013.
[SAN 16] A.Santoro, S. Bartunov, and M. Botvinick, “One-shot learning with memory augmented neural networks”, arXiv preprint arXiv:1605.06065, 2016.
[SAT 19] F. Sattler, S. Wiedemann, K.-R. Müller, and W. Samek, “Robust and communication-efficient federated learning from non-iid data”, arXiv preprint arXiv:1903.02891, 2019.
[SHY 21] S. K. Shyn, D. Kim, and K. Kim, “FedCCEA : A Practical Approach of Client Contribution Evaluation for Federated Learning”, arXiv preprint arXiv: 2106.02310, 2021.
[SNE 17] J. Snell, K. Swersky, and R. S. Zemel, “Prototypical networks for few-shot learning”, Conference on Neural Information Processing Systems (NIPS), 2017.
[VAH 21] S. Vahidian, M. Morafah, and B. Lin, “Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity”, arXiv preprint arXiv: 2105.00562, 2021.
[VIN 16] O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, “Matching networks for one shot learning”, Conference on Neural Information Processing Systems (NIPS), pp. 3630–3638, 2016.
[WAN 19] Y. Wang, Q. Yao, J. Kwok, L. M. Ni, “Generalizing from a Few Examples: A Survey on Few-Shot Learning”, ACM Computing Surveys (CSUR), 2019
[WAN 20] H. Wang, M. Yurochkin, Y. Sun, D. Papailiopoulos, and Y. Khazaeni, “Federated learning with matched averaging”, International Conference on Learning Representations, 2020.
[XIA 20] W. Xia, T. Q. S. Quek, K. Guo, W. Wen, H. H. Yang, and H. Zhu, “Multi-armed bandit based client scheduling for federated learning”, IEEE Transactions on Wireless Communications, 2020.
[YAN 19] Q. Yang, Y. Liu, Y. Cheng, Y. Kang, T. Chen, and H. Yu, “Federated learning”, Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 13, no. 3, pp. 1–207, 2019.
[YU 20] T. Yu, E. Bagdasaryan, and V. Shmatikov, “Salvaging Federated Learning by Local Adaptation”, arXiv preprint arXiv:2002.04758, 2020.
[ZHA 21] H. Zhang, and J. Kim, “Towards a Federated Learning Framework for Heterogeneous Devices of Internet of Things”, arXiv preprint arXiv: 2105.14675, 2021.