Recommender systems are information filtering mechanisms that seek to predict the user's preference toward a particular product, typically in the form of a numeric rating. In implementing recommender systems, collaborative filtering (CF) models are the most dominant approach. CF leverages the collaborative behaviors of all existing users, and designing a CF model involves learning user and item representations and modeling user-item interactions. The earliest CF models learned these representations solely based on user-given numeric ratings. However, employing ratings to learn such representations is a clear case of oversimplifying the latter that can actually harm accuracy.

In light of this, review texts have been utilized to alleviate this issue. Since users can freely discuss their reasons and experience with the items, reviews contain a large quantity of rich latent information that cannot be otherwise acquired exclusively from ratings. Moreover, when integrated with neural networks and deep learning, this has given rise to a new class of recommender systems, i.e., neural review-based collaborative filtering (NRCF), resulting in state-of-the-art recommendation performances.

Notwithstanding the revolutionary results granted by NN, there are serious issues that are compounding NRCF models. First is the black-box nature of NN. Despite widespread usage, they have largely remained black-boxes that opaque the explainability behind predictions. Providing explanations is necessary to improve user satisfaction and aid potential clients in making informed decisions. Second is the trade-off between accuracy and explainability, wherein the most accurate models are usually complicated, non-transparent, and unexplainable. The reverse is also true for explainable and straightforward methods that sacrifice accuracy. The third issue, which is rarely tackled, is the need for ground-truth explanation texts during training. A typical large dataset only consists of a user ID, item ID, rating, and review. It is unrealistic to expect target or ground-truth explanations in the dataset. Also, obtaining these targets from scratch is cumbersome, labor-intensive, and time-consuming.

To address the aforementioned issues, this dissertation formally establishes a novel CF pipeline that equally emphasizes accuracy and explainability for NRCF. While maintaining excellent rating prediction accuracy, we reformulate explainability as a non-supervised construction of explanation texts acceptable to humans in real life. As our contribution to the theory of explanation construction, we formally define two types of recommender explanations for the first time. Invariant explanations are fixed or the same for all users regardless of their preferences since they depend on the item information. In contrast, variant explanations are personalized and geared toward user preferences.

Specifically, we implemented three NRCF models in this dissertation that bridge the research gap between accuracy and explainability. extbf{BENEFICT} is one of the first recommender models to integrate BERT and CF. For explainability, it implements the fixed-length solution of the maximum subarray problem to produce token-based explanations.
extbf{ESCOFILT} is the first extractive summarization-based CF model; it uniquely integrates BERT, $K$-Means embedding clustering, and multilayer perceptron to learn sentence embeddings, summary-level explanations, and user-item interactions. Unlike other types of explanations, ESCOFILT-produced explanations closely resemble real-life explanations. Lastly, extbf{NEAR} pioneers the first complete non-supervised explainability architecture that can produce both invariant and variant explanations. Invariant explanations are summary-level texts derived from ESCOFILT. On the other hand, variant explanations are generated by our customized Transformer conditioned on every user-item-rating tuple and artificial ground-truth (self-supervised label) from one of the invariant explanation's sentences. All these models have resulted in state-of-the-art recommendation accuracy during the time of their publication. Moreover, further experiments and assessments show that our variant explanations are more personalized than those from other similar models, and invariant explanations are preferred over other contemporary models' texts in real life.

[1] Julian McAuley and Jure Leskovec. Hidden factors and hidden topics: understand- ing rating dimensions with review text. In Proceedings of the 7th ACM conference on Recommender systems, pages 165–172, 2013.
[2] Chong Chen, Min Zhang, Yiqun Liu, and Shaoping Ma. Neural attentional rating regression with review-level explanations. In Proceedings of the 2018 World Wide Web Conference, pages 1583–1592, 2018.
[3] Yongfeng Zhang, Guokun Lai, Min Zhang, Yi Zhang, Yiqun Liu, and Shaoping Ma. Explicit factor models for explainable recommendation based on phrase- level sentiment analysis. In Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 83–92, 2014.
[4] Lei Zheng, Vahid Noroozi, and Philip S Yu. Joint deep modeling of users and items using reviews for recommendation. In Proceedings of the 10th ACM International Conference on Web Search and Data Mining, pages 425–434, 2017.
[5] Le Wu, Xiangnan He, Xiang Wang, Kun Zhang, and Meng Wang. A sur- vey on accuracy-oriented neural recommendation: From collaborative filtering to information-rich recommendation. IEEE Transactions on Knowledge and Data Engineering, 2022.
[6] Xiangnan He, Xiaoyu Du, Xiang Wang, Feng Tian, Jinhui Tang, and Tat- Seng Chua. Outer product-based neural collaborative filtering. arXiv preprint arXiv:1808.03912, 2018.
[7] Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat-Seng Chua. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, pages 165–174, 2019.
[8] Andriy Mnih and Russ R Salakhutdinov. Probabilistic matrix factorization. In Advances in neural information processing systems, pages 1257–1264, 2008.
[9] Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37, 2009.
[10] Cataldo Musto, Marco de Gemmis, Giovanni Semeraro, and Pasquale Lops. A multi-criteria recommender system exploiting aspect-based sentiment analysis of users’ reviews. In Proceedings of the 11th ACM Conference on Recommender Systems, pages 321–325, 2017.
[11] Jobin Wilson, Santanu Chaudhury, and Brejesh Lall. Improving collaborative filtering based recommenders using topic modelling. In Proceedings of the 2014 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT)-Volume 01, pages 340–346, 2014.
[12] Li Chen, Guanliang Chen, and Feng Wang. Recommender systems based on user reviews: the state of the art. User Modeling and User-Adapted Interaction, 25(2):99–154, 2015.
[13] Xianchen Wang, Hongtao Liu, Peiyi Wang, Fangzhao Wu, Hongyan Xu, Wenjun Wang, and Xing Xie. Neural review rating prediction with hierarchical attentions and latent factors. In International Conference on Database Systems for Advanced Applications, pages 363–367. Springer, 2019.
[14] Hanna M Wallach. Topic modeling: beyond bag-of-words. In Proceedings of the 23rd International Conference on Machine learning, pages 977–984, 2006.
[15] David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3(Jan):993–1022, 2003.
[16] Mingmin Jin, Xin Luo, Huiling Zhu, and Hankz Hankui Zhuo. Combining deep learning and topic modeling for review understanding in context-aware recom- mendation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1605–1614, 2018.
[17] Xingjie Feng and Yunze Zeng. Neural collaborative embedding from reviews for recommendation. IEEE Access, 7:103263–103274, 2019.
[18] Xin Dong, Jingchao Ni, Wei Cheng, Zhengzhang Chen, Bo Zong, Dongjin Song, Yanchi Liu, Haifeng Chen, and Gerard de Melo. Asymmetrical hierarchical net- works with attentive interactions for interpretable review-based recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 7667–7674, 2020.
[19] Zhongxia Chen, Xiting Wang, Xing Xie, Tong Wu, Guoqing Bu, Yining Wang, and Enhong Chen. Co-attentive multi-task learning for explainable recommendation. In IJCAI, pages 2137–2143, 2019.
[20] Donghua Liu, Jing Li, Bo Du, Jun Chang, and Rong Gao. DAML: Dual atten- tion mutual learning between ratings and reviews for item recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 344–352, 2019.
[21] Sungyong Seo, Jing Huang, Hao Yang, and Yan Liu. Interpretable convolutional neural networks with dual local and global attention for review rating prediction. In Proceedings of the 11th ACM Conference on Recommender Systems, pages 297– 305, 2017.
[22] Xu Chen, Yongfeng Zhang, and Zheng Qin. Dynamic explainable recommendation based on neural attentive models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 53–60, 2019.
[23] Chuhan Wu, Fangzhao Wu, Junxin Liu, and Yongfeng Huang. Hierarchical user and item representation with three-tier attention for recommendation. In Proceed- ings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1818–1826, 2019.
[24] Yi Tay, Anh Tuan Luu, and Siu Cheung Hui. Multi-pointer co-attention networks for recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2309–2318, 2018.
[25] Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. Neural collaborative filtering. In Proceedings of the 26th International Conference on World Wide Web, pages 173–182, 2017.
[26] Georgina Peake and Jun Wang. Explanation mining: Post hoc interpretability of latent factor models for recommendation systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 2060–2069, 2018.
[27] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should I trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 1135–1144, 2016.
[28] Xiang Wang, Xiangnan He, Fuli Feng, Liqiang Nie, and Tat-Seng Chua. TEM: Tree-enhanced embedding model for explainable recommendation. In Proceedings of the 2018 World Wide Web Conference, pages 1543–1552, 2018.
[29] Yongfeng Zhang and Xu Chen. Explainable recommendation: A survey and new perspectives. arXiv preprint arXiv:1804.11192, 2018.
[30] Pegah Sagheb Haghighi, Olurotimi Seton, and Olfa Nasraoui. An explain- able autoencoder for collaborative filtering recommendation. arXiv preprint arXiv:2001.04344, 2019.
[31] Andrew Dillon. Collecting as routine human behavior: Personal identity and control in the material and digital world. Information & Culture, 54(3):255–280, 2019.
[32] Ali Faraji-Rad, Shiri Melumad, and Gita Venkataramani Johar. Consumer desire for control as a barrier to new product adoption. Journal of Consumer Psychology, 27(3):347–354, 2017.
[33] Zhiwei Liu, Lin Meng, Fei Jiang, Jiawei Zhang, and Philip S Yu. Deoscillated graph collaborative filtering. arXiv preprint arXiv:2011.02100, 2020.
[34] Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgcn: Simplifying and powering graph convolution network for recom- mendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 639–648, 2020.
[35] Mohammad Taher Pilehvar and Jose Camacho-Collados. WiC: the word-in- context dataset for evaluating context-sensitive meaning representations. In Pro- ceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 1267–1273, 2019.
[36] Qianqian Wang, Si Li, and Guang Chen. Word-driven and context-aware review modeling for recommendation. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pages 1859–1862, 2018.
[37] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre- training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
[38] Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. arXiv preprint arXiv:1908.02265, 2019.
[39] Qi Liu, Matt J Kusner, and Phil Blunsom. A survey on contextual embeddings. arXiv preprint arXiv:2003.07278, 2020.
[40] Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S Weld, Luke Zettlemoyer, and Omer Levy. Spanbert: Improving pre-training by representing and predicting spans. arXiv preprint arXiv:1907.10529, 2019.
[41] Lei Li, Yongfeng Zhang, and Li Chen. Personalized transformer for explainable recommendation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4947–4957, August 2021.
[42] Lei Li, Yongfeng Zhang, and Li Chen. Generate neural template explanations for recommendation. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pages 755–764, 2020.
[43] Li Dong, Shaohan Huang, Furu Wei, Mirella Lapata, Ming Zhou, and Ke Xu. Learning to generate product reviews from attributes. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Lin- guistics: Volume 1, Long Papers, pages 623–632, 2017.
[44] Piji Li, Zihao Wang, Zhaochun Ren, Lidong Bing, and Wai Lam. Neural rating regression with abstractive tips generation for recommendation. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 345–354, 2017.
[45] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, AidanNGomez,L?ukaszKaiser,andIlliaPolosukhin.Attentionisallyouneed. In Advances in neural information processing systems, pages 5998–6008, 2017.
[46] Alan Zakbik, Tanja Bergmann, and Roland Vollgraf. Pooled contextualized em- beddings for named entity recognition. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 724– 728, 2019.
[47] Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144, 2016.
[48] Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. How to fine-tune bert for text classification? arXiv preprint arXiv:1905.05583, 2019.
[49] Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 315–323, 2011.
[50] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
[51] Sung Eun Bae. Sequential and parallel algorithms for the generalized maximum subarray problem. 2007.
[52] Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in In- formation Retrieval, pages 43–52, 2015.
[53] Ruining He and Julian McAuley. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In Proceedings of the 25th International Conference on World Wide Web, pages 507–517, 2016.
[54] Hongtao Liu, Fangzhao Wu, Wenjun Wang, Xianchen Wang, Pengfei Jiao, Chuhan Wu, and Xing Xie. NRPA: Neural recommendation with personalized attention. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1233–1236, 2019.
[55] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[56] Anand Rajaraman and Jeffrey David Ullman. Mining of massive datasets. Cam- bridge University Press, 2011.
[57] Rada Mihalcea and Paul Tarau. Textrank: Bringing order into text. In Proceedings of the 2004 conference on empirical methods in natural language processing, pages 404–411, 2004.
[58] Ria Mae Borromeo and Motomichi Toyama. Automatic vs. crowdsourced senti- ment analysis. In Proceedings of the 19th International Database Engineering & Applications Symposium, pages 90–95, 2015.
[59] Reinald Adrian Pugoy and Hung-Yu Kao. BERT-based neural collaborative filtering and fixed-length contiguous tokens explanation. In Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, pages 143–153, 2020.
[60] Jess Drake. Introduction to Logic. ED-Tech Press, 2018.
[61] Sarah Andersen. Sentence types and functions. San Jose State University Writing Center, 2014.
[62] Sarang Gupta and Kumari Nishu. Mapping local news coverage: Precise location extraction in textual news content using fine-tuned bert based language model. In Proceedings of the 4th Workshop on Natural Language Processing and Computa- tional Social Science, pages 155–162, 2020.
[63] Derek Miller. Leveraging BERT for extractive text summarization on lectures. arXiv preprint arXiv:1906.04165, 2019.
[64] Nils Reimers and Iryna Gurevych. Sentence-BERT: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019.
[65] Shuyin Xia, Daowan Peng, Deyu Meng, Changqing Zhang, Guoyin Wang, Elisa- beth Giem, Wei Wei, and Zizhong Chen. A fast adaptive k-means with no bounds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020.
[66] Jeffrey C Zemla, Steven Sloman, Christos Bechlivanidis, and David A Lagnado. Evaluating everyday explanations. Psychonomic Bulletin & Review, 24(5):1488– 1500, 2017.
[67] J Richard Landis and Gary G Koch. The measurement of observer agreement for categorical data. Biometrics, pages 159–174, 1977.
[68] Hady Elsahar, Maximin Coavoux, Matthias Gall ́e, and Jos Rozen. Self- supervised and controlled multi-document opinion summarization. arXiv preprint arXiv:2004.14754, 2020.
[69] Laura Frances Bright. Consumer control and customization in online environ- ments: an investigation into the psychology of consumer choice and its impact on media enjoyment, attitude and behavioral intention. The University of Texas at Austin, 2008.
[70] Harald Steck. Evaluation of recommendations: rating-prediction and ranking. In Proceedings of the 7th ACM Conference on Recommender systems, pages 213–220, 2013.
[71] Mariette Awad and Rahul Khanna. Support vector regression. In Efficient learning machines, pages 67–80. Springer, 2015.
[72] Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019.
[73] Sanjeevan Sivapalan, Alireza Sadeghian, Hossein Rahnama, and Asad M Madni. Recommender systems in e-commerce. In 2014 World Automation Congress (WAC), pages 179–184. IEEE, 2014.
[74] Ilya Loshchilov and Frank Hutter. Fixing weight decay regularization in adam. 2018.
[75] Reinald Adrian Pugoy and Hung-Yu Kao. Unsupervised extractive summarization- based representations for accurate and explainable collaborative filtering. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 2981–2990, August 2021.