有限的對齊資料為垂直式聯邦學習的主要挑戰之一，由於該框架僅能依賴對齊的資料進行模型訓練，其表現往往受限於對齊資料的數量，這種依賴性導致模型在面對資料不足或不平衡的情況下難以有效學習，進而影響其泛化能力和實際應用潛力。為解決上述問題，許多方法被提出，例如對比學習、自監督學習以及資料蒸餾等，旨在提升模型在有限資料條件下的表現。本論文基於利用未對齊資料並訓練高效特徵萃取模型，提出適應式垂直聯邦學習之預訓練擴散模型。此方法透過使用對齊資料與未對齊資料共同預訓練擴散模型，顯著地提升資料使用效率，突破傳統垂直聯邦學習僅依賴對齊資料訓練的瓶頸。預訓練過程中，擴散模型能夠提取高品質的圖片特徵，並將其細調後的編碼器作為底部模型，為後續訓練的堅實基礎；此外，傳統垂直聯邦學習的頂部模型通常以平等的方式處理每個參與方的特徵輸出，忽視各參與方特徵品質與重要程度的差異，這在資料異質性高的場景中將導致性能下降。因此，本論文進一步提出參與方權重注意力機制，該機制藉由注意力機制的核心概念，即並非所有輸入特徵同等重要，模型應聚焦於關鍵特徵，透過動態調整參與方特徵的權重，該機制能夠根據特徵的重要性動態地調整特徵權重，藉此提升頂部模型的分類精確度，在非獨立同分布場景中表現尤為突出。實驗結果顯示，本論文提出的演算法在異質資料的場景中，其準確度表現優於其他現有演算法，此外，該演算法的計算複雜度相對較低，這得益於預訓練階段的優化設計以及參數量的有效控制，因此在進行垂直聯邦學習時有備較低的時間成本。

Limited aligned data is one of the primary challenges in vertical federated learning (VFL) Since VFL can only rely on aligned data for model training, its performance is extremely constrained by the quantity of aligned data. The dependency often results in the model's inability to learn effectively when faced with insufficient or imbalanced data, thereby restricting its generalization capability and practical application possibility. To solve above-mentioned challenge, several studies have been conducted such as contrastive learning, self-supervised learning, and data distillation. The goal aims to enhance model performance under limited data conditions. This thesis presents a novel perspective by utilizing unaligned data adequately and by training feature extraction models efficiently, i.e., the proposed Adaptive Vertical Federated Learning with Pretrained Diffusion Models (AVFL-PreDM). This method significantly improves data utilization efficiency by jointly pretraining a diffusion model with both aligned and unaligned data, overcoming the conventional VFL limitation of relying exclusively on aligned data for training. During the pretraining process, the diffusion model effectively extracts high-quality image features, and its fine-tuned encoder serves as the bottom model, establishing a solid foundation for subsequent training phases. Furthermore, conventional VFL typically processes the feature outputs of each participating party equally in the top model, overlooking the varying quality and importance of features across parties, thereby results in performance degradation especially for highly heterogeneous data environment. To solve this issue, this thesis presents the Party Weight Attention (PWA) mechanism based on the concept of attention mechanisms as well as not all input features are equally important so that the model should focus on key features. By dynamically adjusting the weights of features from each party based on their importance, the proposed PWA mechanism significantly enhances the classification accuracy of the top model especially for non-independent and identically distributed (non-IID) scenarios. Experimental results demonstrates that the proposed algorithm outperforms existing methods in terms of accuracy in heterogeneous data environments. Additionally, the algorithm's computational complexity is relatively low, owing to the better design of the pretraining phase and effective parameter control, resulting in reduced time costs of VFL. This combination of high accuracy and low computational overhead achieves the algorithm as a promising solution for practical applications where efficiency and adaptability are critical.

[1] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE communications surveys & tutorials, vol. 19, no. 4, pp. 2322–2358, 2017.
[2] K. Wei, J. Li, M. Ding, C. Ma, H. H. Yang, F. Farokhi, S. Jin, T. Q. Quek, and H. V. Poor, “Federated learning with differential privacy: Algorithms and performance analysis,” IEEE transactions on information forensics and security, vol. 15, pp. 3454–3469, 2020.
[3] P. Voigt and A. Von dem Bussche, “The eu general data protection regulation (gdpr),” A practical guide, 1st ed., Cham: Springer International Publishing, vol. 10, no. 3152676, pp. 10–5555, 2017.
[4] P. Voigt and A. Von dem Bussche, “The policy effect of the general data protection regulation (gdpr) on the digital public health sector in the european union: An empirical investigation,” International Journal of Environmental Research and Public Health, vol. 16, no. 5, p. 858, 2019.
[5] S. H. Park, J. Kim, and M. Lee, “General data protection regulation: A study on attitude and emotional empowerment,” Journal of Global Information Technology Management, vol. 26, no. 3, pp. 203–218, 2023.
[6] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics. PMLR, 2017, pp. 1273–1282.
[7] W. Y. B. Lim, N. C. Luong, D. T. Hoang, Y. Jiao, Y. Liang, Q. Yang, D. Niyato, and C. Miao, “Federated learning in mobile edge networks: A comprehensive survey,” IEEE communications surveys & tutorials, vol. 22, no. 3, pp. 2031–2063, 2020.
[8] S. Niknam, H. S. Dhillon, and J. H. Reed, “Federated learning for wireless communications: Motivation, opportunities, and challenges,” IEEE Communications Magazine, vol. 58, no. 6, pp. 46–51, 2020.
[9] S. Wang, T. Tuor, T. Salonidis, K. K. Leung, C. Makaya, T. He, and K. Chan, “Adaptive federated learning in resource constrained edge computing systems,” IEEE journal on selected areas in communications, vol. 37, no. 6, pp. 1205–1221, 2019.
[10] M. Aledhari, R. Razzak, R. M. Parizi, and F. Saeed, “Federated learning: A survey on enabling technologies, protocols, and applications,” IEEE Access, vol. 8, pp. 140 699–140 725, 2020.
[11] B. Liu, N. Lv, Y. Guo, and Y. Li, “Recent advances on federated learning: A systematic survey,” Neurocomputing, p. 128019, 2024.
[12] P. Mohassel and Y. Zhang, “Secureml: A system for scalable privacy-preserving machine learning,” Proceedings of the IEEE Symposium on Security and Privacy (SP), pp. 19–38, 2017.
[13] Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, no. 2, pp. 1–19, 2019.
[14] Y. Cheng, Y. Liu, T. Chen, and Q. Yang, “Federated learning for privacy-preserving ai,” Communications of the ACM, vol. 63, no. 12, pp. 33–36, 2020.
[15] J. Wen, Z. Zhang, Y. Lan, Z. Cui, J. Cai, and W. Zhang, “A survey on federated learning: challenges and applications,” International Journal of Machine Learning and Cybernetics, vol. 14, no. 2, pp. 513–535, 2023.
[16] Q. Li, Z. Wen, Z. Wu, S. Hu, N. Wang, Y. Li, X. Liu, and B. He, “A survey on federated learning systems: Vision, hype and reality for data privacy and protection,”IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 4, pp. 3347–3366, 2021.
[17] L. Li, Y. Fan, M. Tse, and K. Lin, “A review of applications in federated learning,” Computers & Industrial Engineering, vol. 149, p. 106854, 2020.
[18] Z. Zhou, Y. Tian, and C. Peng, “Privacy-preserving federated learning framework with general aggregation and multiparty entity matching,” Wireless Communications and Mobile Computing, vol. 2021, no. 1, p. 6692061, 2021.
[19] M. Aledhari, R. Razzak, R. M. Parizi, and F. Saeed, “Private set intersection: A survey on enabling technologies, protocols, and applications,” IEEE Access, vol. 8, pp. 140 699–140 725, 2020.
[20] Y. Liu, X. Liu, X. Zhang, D. Wu, Y. Kang, W. Fang, T. Li, and Y. Tong, “Vertical federated learning for effectiveness, security, applicability: A survey,” ACM Computing Surveys, vol. 57, no. 4, pp. 1–42, 2025.
[21] X. Chen, Z. Li, J. Zhang, Y. Wang, and N. Xu, “Efficient vertical federated learning with secure aggregation,” Journal of Big Data, vol. 11, no. 1, pp. 1–25, 2024.
[22] Y. Kang, Y. Liu, and X. Liang, “Fedcvt: Semi-supervised vertical federated learning with cross-view training,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 13, no. 4, pp. 1–16, 2022.
[23] Y. Yang, X. Ye, and T. Sakurai, “Multi-view federated learning with data collaboration,” in Proceedings of the 2022 14th International Conference on Machine Learning and Computing, 2022, pp. 178–183.
[24] C. Huang, L. Wang, and X. Han, “Vertical federated knowledge transfer via representation distillation for healthcare collaboration networks,” in Proceedings of the ACM Web Conference 2023, 2023, pp. 4188–4199.
[25] Z. Ren, L. Yang, and K. Chen, “Improving availability of vertical federated learning: Relaxing inference on non-overlapping data,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 13, no. 4, pp. 1–20, 2022.
[26] W. Li, Q. Xia, J. Deng, H. Cheng, J. Liu, K. Xue, Y. Cheng, and S. Xia, “Vfed-ssd: Towards practical vertical federated advertising,” arXiv preprint arXiv:2205.15987, 2022.
[27] Y. He, Y. Kang, X. Zhao, J. Luo, L. Fan, Y. Han, and Q. Yang, “A hybrid self-supervised learning framework for vertical federated learning,” IEEE Transactions on Big Data, 2024.
[28] S. Feng, “Vertical federated learning-based feature selection with non-overlapping sample utilization,” Expert Systems with Applications, vol. 208, p. 118097, 2022.
[29] F.-H. Tseng and S.-H. Liao, “Vfldiff: Pretrained vertical federated learning with diffusion model,” Paper Submit to IEEE globecom, 2025.
[30] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840–6851, 2020.
[31] J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” in International Conference on Learning Representations, 2021.
[31] Z. Kong and W. Ping, “On fast sampling of diffusion probabilistic models,” in ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models, 2021.
[33] A. Q. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. Mcgrew, I. Sutskever, and M. Chen, “GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models,” in Proceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 162. PMLR, 17–23 Jul 2022, pp. 16 784–16 804.
[34] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695.
[35] Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score based generative modeling through stochastic differential equations,” in International Conference on Learning Representations, 2021.
[36] J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. Van Den Berg, “Structured denoising diffusion models in discrete state-spaces,” Advances in neural information processing systems, vol. 34, pp. 17 981–17 993, 2021.
[37] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans et al., “Photorealistic text-to-image diffusion models with deep language understanding,” Advances in neural information processing systems, vol. 35, pp. 36 479–36 494, 2022.
[38] Y. Liu, J. Yin, W. Zhang, C. An, Y. Xia, and H. Zhang, “Integration of federated learning and ai-generated content: A survey of overview, opportunities, challenges, and solutions,” IEEE Communications Surveys & Tutorials, 2024.
[39] Z. Zhao, F. Yang, and G. Liang, “Federated learning based on diffusion model to cope with non-iid data,” in Chinese Conference on Pattern Recognition and Computer Vision (PRCV). Springer, 2023, pp. 220–231.
[40] F. V. Stanley Jothiraj and A. Mashhadi, “Phoenix: A federated generative diffusion model,” in Companion Proceedings of the ACM Web Conference 2024, 2024, pp. 1568–1577.
[41] Y. Di, H. Shi, X. Wang, R. Ma, and Y. Liu, “Federated recommender system based on diffusion augmentation and guided denoising,” ACM Transactions on Information Systems, vol. 43, no. 2, pp. 1–36, 2025.
[42] W. Issa, N. Moustafa, B. Turnbull, and K. R. Choo, “Rve-pfl: Robust variational encoder-based personalised federated learning against model inversion attacks,” IEEE Transactions on Information Forensics and Security, 2024.
[43] J.-h. Duan, W. Li, D. Zou, R. Li, and S. Lu, “Federated learning with data-agnostic distribution fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 8074–8083.
[44] D. C. Nguyen, M. Ding, P. N. Pathirana, A. Seneviratne, and A. Y. Zomaya, “Federated learning for covid-19 detection with generative adversarial networks in edge cloud computing,” IEEE Internet of Things Journal, vol. 9, no. 12, pp. 10 257–10 271, 2021.
[45] J. Zhang, L. Zhao, K. Yu, G. Min, A. Y. Al-Dubai, and A. Y. Zomaya, “A novel federated learning scheme for generative adversarial networks,” IEEE Transactions on Mobile Computing, vol. 23, no. 5, pp. 3633–3649, 2023.
[46] I. Ceballos, V. Sharma, E. Mugica, A. Singh, A. Roman, P. Vepakomma, and R. Raskar, “Splitnn-driven vertical partitioning,” arXiv preprint arXiv:2008.04137, 2020.
[47] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
[48] A. G. Woodside and J. W. Davenport Jr, “The effect of salesman similarity and expertise on consumer purchasing behavior,” Journal of Marketing Research, vol. 11, no. 2, pp. 198–202, 1974.