| 研究生: |
張生翰 CHANG, SHENG-HAN |
|---|---|
| 論文名稱: |
基於深度學習與文本摘要之論文緒論產生器 Paper Introduction Generator based on Deep Learning and Text Summarization |
| 指導教授: |
王惠嘉
Wang, Hei-Chia |
| 學位類別: |
碩士 Master |
| 系所名稱: |
管理學院 - 資訊管理研究所 Institute of Information Management |
| 論文出版年: | 2018 |
| 畢業學年度: | 106 |
| 語文別: | 中文 |
| 論文頁數: | 56 |
| 中文關鍵詞: | 深度學習 、文本摘要 、修辭文步結構 、文本自動生成 |
| 外文關鍵詞: | Deep Learning, Rhetorical Move, Automatic Text Generation, Text Summarization |
| 相關次數: | 點閱:77 下載:1 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
學術研究對於科技發展與產業推動至關重要,在撰寫學術研究論文時,通常會以修辭文步結構陳述自己的想法,但這項任務對於初學者、非英語系學者而言都相當困難,為了解決此問題,部份過去研究以文本摘要方式產生論文,但此方法所產生的文本內容上下不連貫,缺乏實用性。
近年來,由於深度學習技術逐漸成熟且能處理自動文本生成的問題,因此本研究提出一種自動緒論生成方法,並結合深度學習和文步結構以達到更好的效果。首先,依文獻整理出緒論包含三個修辭文步結構,分別為「界定研究範圍」、「建立利基」、「佔據利基」,而三個文步共包含十三個步驟,實作時將過去文獻中原本十三個步驟濃縮為適合本研究的五步驟(核心主題、過去相關研究、研究缺口、研究方法與研究貢獻)來建出包含此三個文步的緒論。目前在深度學習的高複雜度運算下,利用序列相依性和出現機率的遞歸神經網絡進行序列記憶與生成,有不錯的表現,可正確產生序列並找到序列之間的相關性。若將用戶提供的關鍵詞和關鍵句導入到本研究中提出的演算法中,將更容易生成專屬於使用者的緒論。
實驗結果發現,將緒論進行文步標記分類後再進行緒論生成能夠產出更具意義與條理的緒論。利用本研究提出的重要性、主題性和連貫性檢測方法進行緒論句子篩選檢測,結果顯示產生的緒論閱讀時具有向心力包含重要內容且流暢通順。另外,摘要方法經實驗後發現相較於傳統方法可提升2%的正確率,顯示該方法有助於找出更好的摘要。而產出的緒論經評估後顯示有許多內容可供參考,相較於之前研究只能產出少量內容、閱讀不通順、用字語法錯誤、敘述方式、缺乏變化性等問題,本研究能夠產生完整的緒論,在字詞和語法方面有大幅度改善,讓使用者能更順暢的閱讀,且內容具有多樣性和創新性,有助於輔助使用者針對自己的研究進行緒論的撰寫。
Academic research is crucial for the development of science and technology and the promotion of industry. When writing academic research papers, it usually uses rhetorical structure to present its ideas, but this task is quite difficult for beginners and non-English scholars. To solve this problem, some past studies have produced papers in the form of text summaries, but the text content produced by this method is inconsistent and lacks practicality. In recent years, due to the maturity of deep learning technology and the ability to deal with the problem of automatic text generation, this study proposes an automatic introduction method, combined with deep learning and a new step structure to achieve better results. At present, under the high complexity operation of deep learning, sequence memory and generation using sequence correlation and probability recurrent neural network can correctly generate sequences and find correlations between sequences. If the keywords and key sentences provided by the user are imported into the algorithm proposed in this study, it will be easier to generate an introduction specific to the user. The experimental results show that it is better to mark rhetorical move first before generating the introduction. Using the detection method proposed in this study to screen the introduction content, the results show that the resulting introduction is more centripetal, important, and fluent. In addition, the abstract method of this study has been found to improve the accuracy of 2% compared with the traditional method. The introduction produced by this study has been evaluated to show that the study can produce a comprehensive introduction compared to previous studies, which helps to assist users in writing an introduction to their own research.
Ameur, M. S. H., Meziane, F., & Guessoum, A. (2017). Arabic machine transliteration using an attention-based encoder-decoder model. Procedia Computer Science, 117, 287-297.
Anthony, L., & Lashkia, G. V. (2003). Mover: A machine learning tool to assist in the reading and writing of technical papers. IEEE transactions on professional communication, 46(3), 185-193.
Baba, K., Nakatoh, T., & Minami, T. (2017). Plagiarism detection using document similarity based on distributed representation. Procedia Computer Science, 111, 382-387.
Barrios, F., López, F., Argerich, L., & Wachenchauzer, R. (2016). Variations of the similarity function of textrank for automated summarization. arXiv preprint arXiv:1602.03606.
Basu, T., & Murthy, C. A. (2015). A similarity assessment technique for effective grouping of documents. Information Sciences, 311, 149-162.
Batcha, N. K., Aziz, N. A., & Shafie, S. I. (2013). CRF based feature extraction applied for supervised automatic text summarization. Procedia Technology, 11, 426-436.
Bing, L., Li, P., Liao, Y., Lam, W., Guo, W., & Passonneau, R. J. (2015). Abstractive multi-document summarization via phrase selection and merging. arXiv preprint arXiv:1506.01597.
Chen, X., Qiu, X., & Huang, X. (2016). Neural sentence ordering. arXiv preprint arXiv:1607.06952.
Cho, K., Van Merriënboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078.
Costa-jussà, M. R., Allauzen, A., Barrault, L., Cho, K., & Schwenk, H. (2017). Introduction to the special issue on deep learning approaches for machine translation. Computer Speech & Language, 46, 367-373.
Cortes, V. (2013). The purpose of this study is to: Connecting lexical bundles and moves in research article introductions. Journal of English for academic purposes, 12(1), 33-43.
Cotos, E., & Pendar, N. (2016). Discourse classification into rhetorical functions for AWE feedback. calico journal, 33, 1.
Dušek, O., & Jurčíček, F. (2016). A context-aware natural language generator for dialogue systems. arXiv preprint arXiv:1608.07076.
Fang, H., Lu, W., Wu, F., Zhang, Y., Shang, X., Shao, J., & Zhuang, Y. (2015). Topic aspect-oriented summarization via group selection. Neurocomputing, 149, 1613-1619.
Irie, K., Tuske, Z., Alkhouli, T., Schluter, R., & Ney, H. (2016). LSTM, GRU, highway and a bit of attention: an empirical overview for language modeling in speech recognition. RWTH Aachen University Aachen Germany.
Hamedani, M. R., Kim, S. W., & Kim, D. J. (2016). SimCC: A novel method to consider both content and citations for computing similarity of scientific papers. Information Sciences, 334, 273-292.
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006). A fast learning algorithm for deep belief nets. Neural computation, 18(7), 1527-1554.
Jimeno Yepes, A. (2017). Word embeddings and recurrent neural networks based on Long-Short Term Memory nodes in supervised biomedical word sense disambiguation. Journal of Biomedical Informatics, 73, 137-147.
Kim, H. K., Kim, H., & Cho, S. (2017). Bag-of-Concepts: Comprehending Document Representation through Clustering Words in Distributed Representation. Neurocomputing. , 266, 336-352.
Kwan, B. S. (2017). A cross-paradigm macro-structure analysis of research articles in Information Systems. English for Specific Purposes, 45, 14-30.
Li, J., Li, J., Fu, X., Masud, M. A., & Huang, J. Z. (2016). Learning distributed word representation with multi-contextual mixed embedding. Knowledge-Based Systems, 106, 220-230.
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017). A survey of deep neural network architectures and their applications. Neurocomputing, 234, 11-26.
Liu, Y., Fu, J., Mei, T., & Chen, C. W. (2017, February). Let Your Photos Talk: Generating Narrative Paragraph for Photo Stream via Bidirectional Attention Recurrent Neural Networks. In AAAI(pp. 1445-1452).
Lloret, E., Romá-Ferri, M. T., & Palomar, M. (2013). Compendium: A text summarization system for generating abstracts of research papers. Data & Knowledge Engineering, 88, 164-175.
Louvigné, S., Shi, J., & Sharmin, S. (2014, August). A corpus-based analysis of the scientific RA genre and RA introduction. In Advanced Mechatronic Systems (ICAMechS), 2014 International Conference on (pp. 123-127). IEEE.
Martín, P., & Pérez, I. K. L. (2014). Convincing peers of the value of one’s research: A genre analysis of rhetorical promotion in academic texts. English for Specific Purposes, 34, 1-13.
Nallapati, R., Zhou, B., Gulcehre, C., & Xiang, B. (2016). Abstractive text summarization using sequence-to-sequence rnns and beyond. arXiv preprint arXiv:1602.06023.
Nguyen, N. T., Miwa, M., Tsuruoka, Y., & Tojo, S. (2015). Identifying synonymy between relational phrases using word embeddings. Journal of biomedical informatics, 56, 94-102.
Ouyang, Y., Li, W., Zhang, R., Li, S., & Lu, Q. (2013). A progressive sentence selection strategy for document summarization. Information Processing & Management, 49(1), 213-221.
Pagliardini, M., Gupta, P., & Jaggi, M. (2017). Unsupervised learning of sentence embeddings using compositional n-gram features. arXiv preprint arXiv:1703.02507.
Peng, C., Li, Y., Yu, Y., Zhou, Y., & Du, S. (2018, January). Multi-step-ahead Host Load Prediction with GRU Based Encoder-Decoder in Cloud Computing. In 2018 10th International Conference on Knowledge and Smart Technology (KST) (pp. 186-191). IEEE.
Qiang, J.-P., Chen, P., Ding, W., Xie, F., & Wu, X. (2016). Multi-document summarization using closed patterns. Knowledge-Based Systems, 99(Supplement C), 28-38.
Roemmele, M., & Intelligence, D. D. N. (2016, February). Writing Stories with Help from Recurrent Neural Networks. In AAAI (pp. 4311-4342).
Salton, G., Wong, A., & Yang, C.-S. (1975). A vector space model for automatic indexing. Communications of the ACM, 18(11), 613-620.
Shouzhong, T., & Minlie, H. (2016). Mining microblog user interests based on TextRank with TF-IDF factor. The Journal of China Universities of Posts and Telecommunications, 23(5), 40-46.
Sutskever, I., Martens, J., & Hinton, G. E. (2011). Generating text with recurrent neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11) (pp. 1017-1024).
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. In Advances in neural information processing systems (pp. 3104-3112)
Teufel, S., & Moens, M. (2002). Summarizing scientific articles: experiments with relevance and rhetorical status. Computational linguistics, 28(4), 409-445.
Thu, H. N. T. (2014). An optimization text summarization method based on naive bayes and topic word for single syllable language. Applied Mathematical Sciences, 8(3), 99-115.
Wielgosz, M., Skoczeń, A., & Mertik, M. (2017). Using LSTM recurrent neural networks for monitoring the LHC superconducting magnets. Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 867, 40-50.
Xiong, Y., & Zuo, R. (2016). Recognition of geochemical anomalies using a deep autoencoder network. Computers & Geosciences, 86(Supplement C), 75-82.
Yang, S., Lu, W., Zhang, Z., Wei, B., & An, W. (2016). Amplifying scientific paper’s abstract by leveraging data-weighted reconstruction. Information Processing & Management, 52(4), 698-719.
Yen, T. H., Wu, J. C., Chang, J., Boisson, J., & Chang, J. S. (2015, July). WriteAhead: Mining Grammar Patterns in Corpora for Assisted Writing. Proceedings of ACL-IJCNLP 2015 System Demonstrations, 139-144. ACL (System Demonstrations) (pp. 139-144).
Zuo, X., Zhang, S., & Xia, J. (2017, August). The enhancement of TextRank algorithm by using word2vec and its application on topic extraction. In Journal of Physics: Conference Series (887, No. 1, p. 012028). IOP Publishing.
國家實驗研究院。(民102)。讓數字說話!台灣學術能量提升與轉換。取自http://www-old.narlabs.org.tw/tw/news/news.php?news_id=1190
國家實驗研究院。(民104)。國研院分析我國科技競爭力。取自https://technews.tw/2015/10/13/narlabs-research-competitive-taiwan/