一個好的推薦系統是電子商務成功發展的重要基石，藉由準確的商品推薦，使用者可以輕易的從眾多商品中找到自己所需。近年來，由於人們著重於提升推薦準確度，多數系統傾向做出一些使用者可能喜歡卻雷同的推薦。然而，已有研究表明，這些同質性過高的推薦容易讓使用者感到廣告疲勞並限縮了其對不同物品的探索，進而影響了他們的滿意度。因此，一個理想的推薦除了保持高準確度外亦應達到一定程度的多樣性。近年來，推薦系統大多採用圖神經網路的模型架構，藉由分析購物資料中使用者與商品之間的交互關係來達到高準確率。然而，由於圖神經網路僅分析二分圖形式的購物資料，如此單一的資料無法有效理解使用者多面向的喜好，進而導致推薦商品的多樣性不足。
因此，本論文提出了新的模型架構：多模態聚合圖神經網路，藉由在圖神經網路引入不同面向的資料來在維持一定推薦準確率的同時提升推薦的多樣性。多模態聚合圖神經網路主要解決三個問題：模態隔離問題、複合式協同學習的效率問題以及模型權重最佳化問題。首先，本論文提出關係感知模態轉換演算法，藉由分析求得朋友關係資訊並用來將圖、表格與文字三種不同模態的資料全都轉換成二分圖的形式以便後續學習，進而解決模態隔離問題。其次，利用聯合協同學習方法來共享模型參數，讓多個使用二分圖的圖神經網路模型之間可以有資訊流通，進一步提升訓練的效率。最後，基於注意力之多模態適配器可以用注意力機制來自動的決策不同資料間的權重，藉此增加模型準確率。
本論文進行了多項實驗來評估多模態聚合圖神經網路的表現。首先，在與基準方法比較時，多模態聚合圖神經網路能在其準確度與基準方法接近的同時得到更高的推薦多樣性。此外，在與其他考量多樣性的推薦方法比較時，在雙方有相同的推薦多樣性的基礎上，多模態聚合圖神經網路明顯在準確率上有更好的表現。由此可知，採用多模態聚合圖神經網路的推薦系統將因其推薦同時具有高準確度與多樣性而讓使用者更加滿意，進而提升電子商務的整體營收。

Generally, recommendation systems are an important cornerstone to develop a successful e-commerce business. Referring to an accurate recommendation, users can purchase what they like from numerous commodities. Recently, most recommendation systems dedicated to improving their accuracy which tend to make similar recommendations users might like. However, research indicated that these homogeneous recommendations make users feel ad fatigue and limit their exploration for various commodities leading to satisfaction drops. Therefore, an ideal system should consider recommendation diversity while improving accuracy. Existing recommendation systems with high accuracy usually apply the model of graph neural networks (GNN) which analyzes the relation between users and commodities in shopping data. However, since GNN only analyzes the shopping data in the form of bipartite graphs, such a single type of data cannot fully discover user's preference which usually is multi-dimensional. Hence, existing GNN-based systems fail to make recommendations with high diversity.
Therefore, this thesis proposes a new model: Multi-Modality Fusion Graph Neural Network (MMF-GNN), which introduces different type of data into the GNN to improve the recommendation diversity while maintaining high accuracy. MMF-GNN addresses three problems: modality isolation problem (MIP), the inefficiency problem of multiple collaborative learning (MCLIP), and weight optimization problem (WOP). First, Relationaware Modality Translation Algorithm (RMTA) analyzes the relationship information to convert data of three modalities, including graph, tabular and text, into bipartite graphs for further learning. Then, Joint Collaborative Learning Method (JCLM) is applied to share model parameters which helps those GNN models using bipartite graphs to communicate with each other, and thus the training efficiency can be improved. Finally, Attention based Multi-Modality Adaptor (AMMA) exploits the attention mechanism to automatically determine the weights of different data to increase the model accuracy.
Several experiments were conducted to evaluate the performance of MMF-GNN. First, comparing to the benchmark method, MMF-GNN obtains higher recommendation diversity with similar accuracy to the benchmark method. In addition, comparing to other recommendation methods considering diversity, the MMF-GNN has a significantly better performance in terms of accuracy under the same diversity. Clearly, the recommendation system applying MMF-GNN makes users more satisfied due to its recommendations with high accuracy and diversity, thereby increasing the overall revenue of e-commerce business.

1. Paul Resnick, Hal R. Varian, “Recommender systems” in Communications of the ACM, Volume 40, Issue 3, Pages 56–58, 1997
2. J. Ben Schafer, Joseph Konstan, John Riedl, “Recommender Systems in E-Commerce” in Proceedings of the 1st ACM conference on Electronic commerce, Pages 158–166, 01 Nov 1999
3. Keith Bradley, Barry Smyth, “Improving recommendation diversity”, in the 12th National Conference in Artificial Intelligence and Cognitive Science, page 75-84, 2001.
4. Mathias Niepert, Mohamed Ahmed, Konstantin Kutzkov. “Learning convolutional neural networks for graphs.” in ICML'16: Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, Pages 2014–2023, 19 Jun 2016
5. Cai-Nicolas Ziegler, Sean M. McNee, Joseph A. Konstan, Georg Lausen, “Improving Recommendation Lists Through Topic Diversification.” in the 14th international conference on World Wide Web, Pages 22–32, 10 May 2005
6. Shiwen Wu, Fei Sun, Wentao Zhang, Xu Xie, Bin Cui, “Graph Neural Networks in Recommender Systems: A Survey” in ACM Computing Surveys, Volume 55, Issue 5, Article No.: 97, pp 1–37, 03 Dec 2022
7. Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, Yoshua Bengio, “Graph Attention Networks” in 6th International Conference on Learning Representations, 16 Feb 2018.
8. Chen Gao, Yu Zheng, Nian Li, Yinfeng Li, Yingrong Qin, Jinghua Piao, Yuhan Quan, Jianxin Chang, Depeng Jin, Xiangnan He, Yong Li, “A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions” in ACM Transactions on Recommender Systems, Volume 1, Issue 1, 03 Mar 2023
9. Tien T. Nguyen, F. Maxwell Harper, Loren Terveen, Joseph A. Konstan, “User Personality and User Satisfaction with Recommender Systems” in Information Systems Frontiers, Volume 20, Issue 6, pp 1173–1189, 01 Dec 2018
10. Xiangnan He, Tat-Seng Chua, “Neural Factorization Machines for Sparse Predictive Analytics.” in Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Pages 355–364, 07 Aug 2017
11. Marco Gori, Augusto Pucci, “ItemRank: A Random-Walk Based Scoring Algorithm for Recommender Engines.” in IJCAI'07: Proceedings of the 20th international joint conference on Artifical intelligence, Pages 2766–2771, 06 Jan 2007
12. Thomas N. Kips, Max Welling., “Semi-supervised classification with graph convolutional networks” in Proceedings of the 5th International Conference on Learning Representations, 22 Feb 2017
13. Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, Tat-Seng Chua, “Neural Graph Collaborative Filtering” in Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Pages 165–174, 18 Jul 2019
14. Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, Meng Wang, “LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation” in Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Pages 639–648, 25 Jul 2020
15. Laming Chen, Guoxin Zhang, Hanning Zhou, “Fast Greedy MAP Inference for Determinantal Point Process to Improve Recommendation Diversity.” in the 32nd International Conference on Neural Information Processing Systems, Pages 5627–5638, 03 Dec 2018.
16. Insu Han, Prabhanjan Kambadur, Kyoungsoo Park, Jinwoo Shin, “Faster greedy MAP inference for determinantal point processes”, in Proceedings of the 34th International Conference on Machine Learning - Volume 70, Pages 1384–1393, 15 Sep 2017.
17. T. Baltrušaitis, C. Ahuja and L. -P. Morency, "Multimodal Machine Learning: A Survey and Taxonomy," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 2, pp. 423-443, 1 Feb. 2019
18. Hanghang Tong, Jingrui He, Mingjing Li, Changshui Zhang, Wei-Ying Ma2, “Graph Based Multi-Modality Learning” in Proceedings of the 13th annual ACM international conference on Multimedia, Pages 862–871, Nov 2005
19. Xi Wei, Tianzhu Zhang, Yan Li, Yongdong Zhang, Feng Wu, “Multi-Modality Cross Attention Network for Image and Sentence Matching” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 05 Aug 2020
20. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin, “Attention is all you need.” in Proceedings of the 31st International Conference on Neural Information Processing Systems, Pages 6000–6010, 04 Dec 2017
21. XiangWang, Xiangnan He, Yixin Cao, Meng Liu, Tat-Seng Chua., “KGAT: Knowledge Graph Attention Network for Recommendation.” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Pages 950–958, 25 Jul 2019