現代人為了方便，上網購物已經變成常態。消費者在購物平台上看到想購買的產品時，因為無法看到或實際試用該產品以做決定，通常會參考平台上該產品的顧客評論和摘要來做購買與否的決定。但通常平台上的顧客評論可能過於口語化，或是摘要過於簡略並沒有提到該產品的關鍵特徵及規格，導致消費者只知道產品很棒或是很差，但無法了解該產品的特徵是好還是壞，因此單單平台上的評論摘要通常無法滿足潛在顧客的要求。而本研究主要以亞馬遜網站的評論及摘要為資料，透過深度學習，針對不同類別的產品評論及摘要進行分析，並產生具關鍵特徵之摘要。本研究結合詞性標註、句法依賴及片語修飾關係找出評論中的關鍵詞，最後藉由機器學習文本關鍵詞與文本內容，從而理解產品評論中句子的語意，並生成簡單易懂的文本摘要，冀望能輔助消費者快速理解評論中的重要資訊。
本研究主要特點如下，1.針對評論文本設計文法及句法依賴規則，能針對不同類型評論語句提取關鍵詞，並可依據需求再進行規則擴充。2.修改原有的Attention機制改以加入Intra Attention機制之指標網路進行生成，使decoder在生成摘要詞彙時會重新考慮過去已生成序列所產生的temporal Attention scores，以避免模型在生成時過度關注相同的已生成詞彙。3.在原有Attention 的機制裡加入keyword的語意特徵，使計算出注意力權重比起原有的Attention機制更能集中在關鍵詞彙上。4.套用Self-Critical-Sequence-Training方法進一步優化Pointer-Base指標網路。本研究進行了十三個驗證，第一個驗證著重在關鍵字提取的準確性，第二到第四個驗證著重於分析模型加入不同Attention機制的詞彙分布，第五到第十一驗證則是與近年提出的抽取式和摘要式摘要方法比較準確性，第十二到第十三驗證則是以本論文最佳模型，在不同類型的產品評論進行評論摘要生成，並分別以Rouge、BLEU及METEOR三種方式進行準確性之評估。

Today shopping online has become a daily practice for many people. A customer, being interested in something on a web shop and not able to examine and try the real product, might turn to the custom reviews to see the opinions of buyers on the product before making the final decision. But the reviews may be too colloquial, or the summary may be too short to mention important key characteristics and the detail specs of the product. This makes the customer informed only that the product is good or bad, but not that important key characteristics of the product are good or bad. Such reviews of a web shop may not actually be helpful for its potential customers. This research mainly aims at Amazon review and summary information and uses deep learning approach to analyze and learn reviews and summaries of different classes of products and to produce summaries with import key characteristics of products for customers. The approach first uses part-of-speech (POS) tagging, syntactic dependence, and noun phrases to find the keywords in the reviews and summaries, and then uses reinforcement learning to learn the keywords and their review contents in order to understand the semantic of sentences in the customer reviews. Given a product review, the network produces a summary that is easy to read and contains information of important key characteristics of the product and is helpful for consumers to quickly understand the quality of the product and make the final decision.
The major research works of this study are as follows: 1. Design grammar and syntactic dependence rules of sentence and use them to extract keywords from different types of sentences in the reviews. Those rules can be expanded in the future when there are new requirements. 2. Replace the Attention mechanism of a Pointer-Generator with the Intra Attention mechanism to make the decoder reconsider the temporal Attention scores of previously generated sequences when generating summary vocabulary. This makes the model avoid focusing on a same word when the model is in generation mode. 3. Add extra keyword semantic information in the original Attention mechanism, which makes the generated attention weights focus more on the key words than the original Attention mechanism. 4. Apply the self-critical-sequence training method to optimize the Pointer-Generator network.
This study conducts totally thirteen experiments. The first experiment focuses on the accuracy of keyword extractions. The second to the fourth experiments focus on the analysis of the vocabulary distribution of different Attention mechanisms in the Pointer-Generator network. The fifth to the eleventh experiments are to compare the accuracy with some extractive and abstractive summary methods proposed in recent years. The twelfth to the thirteenth experiments, by using the best model in this study, generate summary for different types of product reviews and evaluate the accuracy of the model by three methods, namely Rouge, BLEU and METEOR.

[1] Hu, M., & Liu, B. (2004, August). Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 168-177).
[2] Khan, J., & Jeong, B. S. (2016, July). Summarizing customer review based on product feature and opinion. In 2016 international conference on machine learning and cybernetics (ICMLC) (Vol. 1, pp. 158-165). IEEE.
[3] Liu, Y., Li, X., & Wang, M. (2019, February). Quantifying Customer Review by Integrating Multiple Source of Knowledge. In Proceedings of the 2019 11th International Conference on Machine Learning and Computing (pp. 6-11).
[4] Somprasertsri, G., & Lalitrojwong, P. (2010). Mining Feature-Opinion in Online Customer Reviews for Opinion Summarization. J. UCS, 16(6), 938-955.
[5] Bengio, Y. (2008). Neural net language models. Scholarpedia, 3(1), 3881.
[6] Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems (pp. 3111-3119).
[7] Pennington, J., Socher, R., & Manning, C. D. (2014, October). Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1532-1543).
[8] Parkkinen, J., Selkäinaho, K., & Oja, E. (1990). Detecting texture periodicity from the cooccurrence matrix. Pattern Recognition Letters, 11(1), 43-50.
[9] Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135-146.
[10] Cavnar, W. B., & Trenkle, J. M. (1994, April). N-gram-based text categorization. In Proceedings of SDAIR-94, 3rd annual symposium on document analysis and information retrieval (Vol. 161175).
[11] Ng, K., & Zue, V. W. (2000). Subword-based approaches for spoken document retrieval. Speech Communication, 32(3), 157-186.
[12] Mani, I., & Maybury, M. T. (1999). Advances in Automatic Text Summarization. Cambridge.
[13] Jones, K. S. (2007). Automatic summarising: The state of the art. Information Processing & Management, 43(6), 1449-1481.
[14] Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information processing & management, 24(5), 513-523.
[15] Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan), 993-1022.
[16] Mihalcea, Rada, and Paul Tarau. "Textrank: Bringing order into text." Proceedings of the 2004 conference on empirical methods in natural language processing. 2004.
[17] Radev, D. R., Jing, H., Styś, M., & Tam, D. (2004). Centroid-based summarization of multiple documents. Information Processing & Management, 40(6), 919-938.
[18] Rossiello, G., Basile, P., & Semeraro, G. (2017, April). Centroid-based text summarization through compositionality of word embeddings. In Proceedings of the MultiLing 2017 Workshop on Summarization and Summary Evaluation Across Source Types and Genres (pp. 12-21).
[19] Erkan, Gunes, and Dragomir Radev. "Lexpagerank: Prestige in multi-document text summarization." Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing. 2004.
[20] Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems (pp. 3104-3112).
[21] Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473.
[22] Hinton, G., Deng, L., Yu, D., Dahl, G. E., Mohamed, A. R., Jaitly, N., ... & Kingsbury, B. (2012). Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal processing magazine, 29(6), 82-97.
[23] Rush, A. M., Chopra, S., & Weston, J. (2015). A neural attention model for abstractive sentence summarization. arXiv preprint arXiv:1509.00685.
[24] See, A., Liu, P. J., & Manning, C. D. (2017). Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368.
[25] Tu, Z., Lu, Z., Liu, Y., Liu, X., & Li, H. (2016). Modeling coverage for neural machine translation. arXiv preprint arXiv:1601.04811.
[26] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems (pp. 5998-6008).
[27] Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805.
[28] Kim, Y. H., Lewis, F. L., & Abdallah, C. T. (1997). A dynamic recurrent neural-network-based adaptive observer for a class of nonlinear systems. Automatica, 33(8), 1539-1543.
[29] Schuster, M., & Paliwal, K. K. (1997). Bidirectional recurrent neural networks. IEEE transactions on Signal Processing, 45(11), 2673-2681.
[30] Selvin, S., Vinayakumar, R., Gopalakrishnan, E. A., Menon, V. K., & Soman, K. P. (2017, September). Stock price prediction using LSTM, RNN and CNN-sliding window model. In 2017 international conference on advances in computing, communications and informatics (icacci) (pp. 1643-1647). IEEE.
[31] Graves, A., Mohamed, A. R., & Hinton, G. (2013, May). Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing (pp. 6645-6649). IEEE.
[32] Greff, K., Srivastava, R. K., Koutník, J., Steunebrink, B. R., & Schmidhuber, J. (2016). LSTM: A search space odyssey. IEEE transactions on neural networks and learning systems, 28(10), 2222-2232.
[33] Nallapati, R., Xiang, B., & Zhou, B. (2016). Sequence-to-sequence rnns for text summarization.
[34] Segaran, T., & Hammerbacher, J. (2009). Beautiful data: the stories behind elegant data solutions. " O'Reilly Media, Inc.".
[35] Ghahramani, Z. (2001). An introduction to hidden Markov models and Bayesian networks. In Hidden Markov models: applications in computer vision (pp. 9-41).
[36] Viterbi, A. (1967). Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE transactions on Information Theory, 13(2), 260-269.
[37] Lew, R., & Mitton, R. (2013). Online English learners’ dictionaries and misspellings: One year on. International Journal of Lexicography, 26(2), 219-233.
[38] Peter Norvig. 2007. How to write a spelling corrector.
http://norvig.com/spell-correct.html. [Online; accessed 12-November-2016]
[39] Loria, S. (2018). TextBlob Documentation. Release 0.15.
[40] De Smedt, T., & Daelemans, W. (2012). Pattern for python. The Journal of Machine Learning Research, 13(1), 2063-2067.
[41] Min, W. N. S. W., & Zulkarnain, N. Z. (2020). Comparative Evaluation of Lexicons in Performing Sentiment Analysis. JOURNAL OF ADVANCED COMPUTING TECHNOLOGY AND APPLICATION (JACTA), 2(1), 14-20.
[42] Tesnière, L. (1959). Eléments de syntaxe structurale.
[43] 刘海涛. (2007). 泰尼埃的结构句法理论 (Doctoral dissertation).
[44] Abbasi Moghaddam, S. (2013). Aspect-based opinion mining in online reviews (Doctoral dissertation, Applied Sciences: School of Computing Science).
[45] Jiang, X., Hu, P., Hou, L., & Wang, X. (2018, August). Improving pointer-generator network with keywords information for chinese abstractive summarization. In CCF International Conference on Natural Language Processing and Chinese Computing (pp. 464-474). Springer, Cham.
[46] Paulus, R. (2019). U.S. Patent No. 10,474,709. Washington, DC: U.S. Patent and Trademark Office.
[47] A. Lamb, A. Goyal, Y. Zhang, S. Zhang, A. Courville, and Y. Bengio. Professor Forcing: A New Algorithm for Training Recurrent Networks (2016), NeurIPS 2016.
[48] S. Wiseman, and A. Rush. Sequence-to-Sequence Learning as Beam-Search Optimization (2016), EMNLP 2016.
[49] Rennie, S. J., Marcheret, E., Mroueh, Y., Ross, J., & Goel, V. (2017). Self-critical sequence training for image captioning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 7008-7024).
[50] Yang, P., Ma, S., Zhang, Y., Lin, J., Su, Q., & Sun, X. (2018). A deep reinforced sequence-to-set model for multi-label text classification. arXiv preprint arXiv:1809.03118.
[51] Paulus, R., Xiong, C., & Socher, R. (2017). A deep reinforced model for abstractive summarization. arXiv preprint arXiv:1705.04304.
[52] Jianmo Ni. 2018. Amazon review data.
http://deepyeti.ucsd.edu/jianmo/amazon/index.html. [Online; accessed 12-November-2019]
[53] Boutkan, F., Ranzijn, J., Rau, D., & van der Wel, E. (2019). Point-less: More abstractive summarization with pointer-generator networks. arXiv preprint arXiv:1905.01975.
[54] Justifying recommendations using distantly-labeled reviews and fined-grained aspects, Jianmo Ni, Jiacheng Li, Julian McAuley
Empirical Methods in Natural Language Processing (EMNLP), 2019
[55] Ganapathibhotla, M., & Liu, B. (2008, August). Mining opinions in comparative sentences. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008) (pp. 241-248).
[56] Takase, S., & Okazaki, N. (2019). Positional encoding to control output sequence length. arXiv preprint arXiv:1904.07418.

[57] Shibata, Y., Kida, T., Fukamachi, S., Takeda, M., Shinohara, A., Shinohara, T., & Arikawa, S. (1999). Byte Pair encoding: A text compression scheme that accelerates pattern matching. Technical Report DOI-TR-161, Department of Informatics, Kyushu University.
[58] Liu, Y. (2019). Fine-tune BERT for extractive summarization. arXiv preprint arXiv:1903.10318.
[59] Brants, T. (2000, April). TnT: a statistical part-of-speech tagger. In Proceedings of the sixth conference on Applied natural language processing (pp. 224-231). Association for Computational Linguistics.
[60] Hunston, S. (2002). Pattern grammar, language teaching, and linguistic variation. Using corpora to explore linguistic variation, 167-183.
[61] Hoang, A., Bosselut, A., Celikyilmaz, A., & Choi, Y. (2019). Efficient adaptation of pretrained transformers for abstractive summarization. arXiv preprint arXiv:1906.00138.
[62] Lin, C. Y. (2004). Rouge: A package for automatic evaluation of summaries. In Text summarization branches out (pp. 74-81).
[63] Papineni, K., Roukos, S., Ward, T., & Zhu, W. J. (2002, July). BLEU: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (pp. 311-318). Association for Computational Linguistics.
[64] Miller, G. A. (1995). WordNet: A Lexical Database for English. Communications of the ACM, 38(11), 39-41. doi:10.1145/219717.219748
[65] Banerjee, S., & Lavie, A. (2005, June). METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization (pp. 65-72).
[66] Maite Taboada, J. B. (2011, 06). Lexicon-Based Methods for Sentiment Analysis. Computational Linguistics, 37(2), 267-307. doi:10.1162/COLI_a_00049
[67] wikiwand.隱藏式馬可夫模型狀態變遷圖.(2020)
https://www.wikiwand.com/zh-tw/%E9%9A%90%E9%A9%AC%E5%B0%94%E5%8F%AF%E5%A4%AB%E6%A8%A1%E5%9E%8B
[68] He, R., & McAuley, J. (2016, April). Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In proceedings of the 25th international conference on world wide web (pp. 507-517).
[69] McAuley, J., Targett, C., Shi, Q., & Van Den Hengel, A. (2015, August). Image-based recommendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 43-52).
[70] 小風英文教室- 連讀與縮寫表(2005)
https://iter01.com/372682.html
[71] 英文縮寫點的用法 & 常見的英文縮寫(2014) https://aixo.pixnet.net/blog/post/43232002
[72] wikiwand.維特比演算法. (2020)
https://www.wikiwand.com/zh-tw/%E7%BB%B4%E7%89%B9%E6%AF%94%E7%AE%97%E6%B3%95#/%E6%B3%A8%E9%87%8A.
[73] Forney Jr, G. D. (2005). The viterbi algorithm: A personal history. arXiv preprint cs/0504020.
[74] Fan and Fuel研究調查(2017)
https://fanandfuel.com/no-online-customer-reviews-means-big-problems-2017/
[75] MarketPlace研究調查(2016)
https://www.marketplacepulse.com/articles/how-many-retailers-are-selling-on-amazon-marketplaces
[76] Episerver Ascend研究調查(2019)
https://retailwire.com/discussion/who-is-winning-the-shopping-search-race-amazon-or-google/
[77] Raymond James研究調查(2014)
https://www.businessinsider.com/amazon-gains-share-in-ecommerce-searches-2017-1
[78] Julian McAuley研究調查(2014)
https://minimaxir.com/2014/06/reviewing-reviews/