簡易檢索 / 詳目顯示

回結果列表
研究生： 林奕廷
Lin, Yi-Ting
論文名稱： 以聯合分析法探討大型語言模型使用因素
Investigating the Usage Factors of Large Language Models through Conjoint Analysis
指導教授： 黃瀞瑩
Huang, Ching-Ying
學位類別： 碩士
Master
系所名稱： 管理學院 - 企業管理學系
Department of Business Administration
論文出版年： 2024
畢業學年度： 112
語文別： 英文
論文頁數： 109
中文關鍵詞： 大型語言模型 、TAM 、UTAUT 、聯合分析
外文關鍵詞： Large Language Models, TAM, UTAUT, Conjoint Analysis
相關次數： 點閱：66  下載：9 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 大型語言模型（LLM）已成為現代科技重要的一部分，在文本生成、摘取訊息和對話輔助方面提供了先進的功能。盡管它們得到了非常廣泛的應用，但在理解是什麽驅使大眾使用這項人工智慧工具方面仍需探討。本論文通過探索一般大眾對LLM的偏好和優先事項來探討這一問題。通過對472名有效受訪者進行兩階段調查並采用聯合分析，研究揭示傳統的科技接受模型（TAM和UTAUT）無法全面理解大眾的偏好。相反的，使用者更重視回復準確性、對話記憶能力和個人化等產品特性，表明上述產品特性與交互體驗對使用者的使用意願至關重要。
    研究結果對LLM開發者具有重要意義。為滿足使用者期望並提高使用率，開發者應專注於提高LLM回應的自然度和連貫性，確保互動直觀且類人化。保持高準確性並提供與上下文相關的信息對建立使用者參與度有直接影響，這可以通過持續更新和實時數據集成來達成。產品個人化在其中也起著關鍵作用；結合自適應學習機制根據使用者偏好和歷史來定制交互可顯著提高滿意度。此外，調整回復速度對時間敏感的應用尤為重要。最後，考慮使用者偏好的人口統計差異可以幫助調整LLM功能，以更好地滿足不同需求，從而提高不同群體的使用率。將LLM設計與使用者偏好相符不僅會促進更廣泛的接受，還會增強LLM在日常應用中的整合，使大型語言模型使用率和接受度更高。

    Large Language Models (LLMs) have become essential to modern technology, offering advanced capabilities in text generation, summarization, and conversational assistance. Despite their widespread adoption, there is a significant gap in understanding what drives general users to adopt these AI tools. This thesis addresses this gap by exploring the preferences and priorities of everyday users concerning LLMs. Conducting data from a two-stage survey with 472 valid respondents and employing conjoint analysis, the study reveals that traditional technology acceptance models (TAM and UTAUT) aren’t able to fully capture the preferences of users. Instead, users place higher value on features such as respond accuracy, conversational retention and personalization capabilities, indicating that a seamless and engaging interaction experience is crucial for user adoption.
    The study's findings have important implications for developers of LLMs. To meet user expectations and enhance adoption, developers should focus on improving the naturalness and coherence of LLM responses, ensuring that interactions are intuitive and human-like. Maintaining high accuracy and providing contextually relevant information is essential for building user engagement, which can be supported by continuous updates and real-time data integration. Personalization also plays a key role in user engagement; combining adaptive learning mechanisms to tailor interactions based on user preferences and history can significantly boost satisfaction. Furthermore, optimizing response time is vital, particularly for time-sensitive applications. Finally, considering demographic variations in user preferences can help tailor LLM features to better address diverse needs, thereby improving adoption rates across different user segments. Matching LLM design with these insights will not only facilitate broader acceptance but also enhance the integration of LLMs into everyday applications, making advanced AI more accessible and beneficial to a wider audience.

    ABSTRACT III 摘要 IV ACKNOWLEDGEMENT V LIST OF TABLES VIII LIST OF FIGURES IX CHAPTER ONE 11 INTRODUCTION 11 Research Background 11 Research Motivation and Objectives 13 Research Gap 15 CHAPTER TWO 17 LITERATURE REVIEW 17 History and Evolution of Large Language Model 17 Different Architectures behind Large Language Model 23 Model Architecture and Training 28 Personalization and User Experience 35 CHAPTER THREE 39 RESEARCH METHODOLOGY 39 Conjoint Analysis 39 Questionnaire Design and Attributes Selecting 41 Data Analysis 42 CHAPTER FOUR 47 RESEARCH RESULTS 47 Overall Attribute Preference Analysis of Respondents 47 Overall Respondents’ Attribute Preference Analysis and Relative Importance 52 Conjoint Analysis by Demographic Variables 57 Conjoint Analysis for Respondents with Different Purpose of Use 76 Establishing a Standard Evaluation for Preference Combination of LLM 82 Summary and Conclusion 87 CHAPTER FIVE 88 CONCLUSIONS AND LIMITATIONS 88 Research Conclusion 88 Research Contributions and Managerial Implications 91 Research Limits and Future Directions 92 References 93 Appendix 1 102 Appendix 2 105

    1. Al Naqbi, H., Bahroun, Z., & Ahmed, V. (2024). Enhancing Work Productivity through Generative Artificial Intelligence: A Comprehensive Literature Review. Sustainability, 16(3), 1166.
    2. Amodei, D., Ananthanarayanan, S., Anubhai, R., Bai, J., Battenberg, E., Case, C., ... & Zhu, Z. (2016, June). Deep speech 2: End-to-end speech recognition in English and mandarin. In International conference on machine learning (pp. 173-182). PMLR.
    3. Anantrasirichai, N., & Bull, D. (2022). Artificial intelligence in the creative industries: a review. Artificial intelligence review, 55(1), 589-656.
    4. Anthropic. (2023). Introducing claude. https://www.anthropic.com/news/introducing-claude
    5. Baeza-Yates, R. (2022, February). Ethical challenges in AI. In Proceedings of the fifteenth ACM international conference on web search and data mining (pp. 1-2).
    6. Bahri, Y., Dyer, E., Kaplan, J., Lee, J., & Sharma, U. (2024). Explaining neural scaling laws. Proceedings of the National Academy of Sciences, 121(27), e2311878121.
    7. Bansal, G., Chamola, V., Hussain, A., Guizani, M., & Niyato, D. (2024). Transforming Conversations with AI—A Comprehensive Study of ChatGPT. Cognitive Computation, 1-24.
    8. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021, March). On the dangers of stochastic parrots: Can language models be too big??. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency (pp. 610-623).
    9. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., ... & Amodei, D. (2020). Language models are few-shot learners. Advances in neural information processing systems, 33, 1877-1901.
    10. Charu C, A. (2018). Neural networks and deep learning: a textbook. Springer.
    11. Chen, Y. C., Li, L., Yu, L., El Kholy, A., Ahmed, F., Gan, Z., ... & Liu, J. (2019). Uniter: Learning universal image-text representations.
    12. Chomsky, N. (1961). On the notion" rule of grammar" (pp. 155-210). American Mathematical Society.
    13. Colin, R. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR, 21(140), 1.
    14. Collobert, R., & Weston, J. (2008, July). A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th international conference on Machine learning (pp. 160-167).
    15. Davis, F. D., Bagozzi, R. P., & Warshaw, P. R. (1989). User acceptance of computer technology: A comparison of two theoretical models. Management science, 35(8), 982-1003.
    16. Dostert, L. E. (1955). The georgetown-ibm experiment. Machine translation of languages, 8, 129ff.
    17. Ferrucci, D., Levas, A., Bagchi, S., Gondek, D., & Mueller, E. T. (2013). Watson: beyond jeopardy!. Artificial Intelligence, 199, 93-105.
    18. Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., ... & Vayena, E. (2018). AI4People—an ethical framework for a good AI society: opportunities, risks, principles, and recommendations. Minds and machines, 28, 689-707.https://doi.org/10.1007/s11023-018-9482-5
    19. Gao, J., Galley, M., & Li, L. (2018, June). Neural approaches to conversational AI. In The 41st international ACM SIGIR conference on research & development in information retrieval (pp. 1371-1374).
    20. Gligorea, I., Cioca, M., Oancea, R., Gorski, A. T., Gorski, H., & Tudorache, P. (2023). Adaptive Learning Using Artificial Intelligence in e-Learning: A Literature Review. Education Sciences, 13(12), 1216.
    21. Goldberg, Y. (2016). A primer on neural network models for natural language processing. Journal of Artificial Intelligence Research, 57, 345-420.
    22. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT press.
    23. Green, P. E., & Srinivasan, V. (1990). Conjoint analysis in marketing: new developments with implications for research and practice. Journal of marketing, 54(4), 3-19.
    24. Grossberg, S. (2013). Recurrent neural networks. Scholarpedia, 8(2), 1888.
    25. Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., & Yang, G. Z. (2019). XAI—Explainable artificial intelligence. Science robotics, 4(37), eaay7120.
    26. Hauser, J. R., & Rao, V. R. (2004). Conjoint analysis, related modeling, and applications. In Marketing research and modeling: progress and prospects: a tribute to Paul E. Green (pp. 141-168). Boston, MA: Springer US.
    27. Hayes-Roth, F. (1985). Rule-based systems. Communications of the ACM, 28(9), 921-932.
    28. Hearst, M. A., Dumais, S. T., Osuna, E., Platt, J., & Scholkopf, B. (1998). Support vector machines. IEEE Intelligent Systems and their applications, 13(4), 18-28.
    29. Julian, et al "RETRO: Reinforcement Learning Enhanced Transformer for Sparse Reward Tasks" Advances in Neural Information Processing Systems (NeurIPS), 2019.
    30. Jurafsky, D., & Martin, J. H. (2019). Speech and language processing. Stanford University.
    31. Kamath, U., Liu, J., & Whitaker, J. (2019). Deep learning for NLP and speech recognition (Vol. 84). Cham, Switzerland: Springer.
    32. Kang, Y., Cai, Z., Tan, C. W., Huang, Q., & Liu, H. (2020). Natural language processing (NLP) in management research: A literature review. Journal of Management Analytics, 7(2), 139-172.
    33. Karttunen, P. (2023). LARGE LANGUAGE MODELS IN HEALTHCARE DECISION SUPPORT. Tampere University.
    34. Kepuska, V., & Bohouta, G. (2018, January). Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home). In 2018 IEEE 8th annual computing and communication workshop and conference (CCWC) (pp. 99-103). IEEE.
    35. Kim, H. C. (2015). Acceptability engineering: the study of user acceptance of innovative technologies. Journal of applied research and technology, 13(2), 230-237.
    36. Koehn, P. (2019). Statistical machine translation Philipp Koehn. Cambridge University Press.
    37. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90.
    38. Laumer, S., Maier, C., & Gubler, F. T. (2019). Chatbot acceptance in healthcare: Explaining user adoption of conversational agents for disease diagnosis.
    39. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
    40. Li, S., Puig, X., Paxton, C., Du, Y., Wang, C., Fan, L., ... & Zhu, Y. (2022). Pre-trained language models for interactive decision-making. Advances in Neural Information Processing Systems, 35, 31199-31212.
    41. Lin, T., Wang, Y., Liu, X., & Qiu, X. (2022). A survey of transformers. AI open, 3, 111-132.
    42. Louviere, J. J., Hensher, D. A., & Swait, J. D. (2000). Stated Choice Methods: Analysis and Applications. Cambridge University Press.
    43. Luce, R. D., & Tukey, J. W. (1964). Simultaneous conjoint measurement: A new type of fundamental measurement. Journal of mathematical psychology, 1(1), 1-27.
    44. Luckin, R., & Holmes, W. (2016). Intelligence unleashed: An argument for AI in education.
    45. Manning, C., & Schutze, H. (1999). Foundations of statistical natural language processing. MIT press.
    46. Manyika, J., & Hsiao, S. (2023). An overview of Bard: an early experiment with generative AI. AI. Google Static Documents, 2.
    47. Marcus, G., & Davis, E. (2020). GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about. Technology Review, 294.
    48. Menon, D., & Shilpa, K. (2023). “Chatting with ChatGPT”: Analyzing the factors influencing users' intention to Use the Open AI's ChatGPT using the UTAUT model. Heliyon, 9(11).
    49. Microsoft (2023, Feb 7). Reinventing search with a new AI-powered Microsoft Bing and Edge, your copilot for the web
    50. Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems, 26.
    51. Miller, T. (2019). Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267, 1-38.
    52. Minsky, M. (1961). Steps toward artificial intelligence. Proceedings of the IRE, 49(1), 8-30.
    53. Mitchell, M., Wolf, T., & Craden, A. (2019). Artificial Intelligence: A guide for thinking humans. Macmillan Audio.
    54. Mitchell, T. M. (1997). Does machine learning really work?. AI magazine, 18(3), 11-11.
    55. Najafabadi, M. M., Villanustre, F., Khoshgoftaar, T. M., Seliya, N., Wald, R., & Muharemagic, E. (2015). Deep learning applications and challenges in big data analytics. Journal of big data, 2, 1-21.
    56. Nguyen, Q. N., & Sidorova, A. (2018). Understanding user interactions with a chatbot: A self-determination theory approach.
    57. Orme, B. (2010). Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research. Research Publishers LLC.
    58. Patel, V. L., Shortliffe, E. H., Stefanelli, M., Szolovits, P., Berthold, M. R., Bellazzi, R., & Abu-Hanna, A. (2009). The coming of age of artificial intelligence in medicine. Artificial intelligence in medicine, 46(1), 5-17.
    59. Pichai, S. (2023, February 14). An important next step on our AI journey. Google.
    60. Pisner, D. A., & Schnyer, D. M. (2020). Support vector machine. In Machine learning (pp. 101-121). Academic Press.
    61. Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257-286.
    62. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training.
    63. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI blog, 1(8), 9.
    64. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., ... & Liu, P. J. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140), 1-67.
    65. Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J. F., Breazeal, C., ... & Wellman, M. (2022). Machine Behaviour (Originally Published 2019 by Springer Nature). Machine Learning and the City: Applications in Architecture and Urban Design, 143-166.
    66. Ravichandiran, S. (2021). Getting Started with Google BERT: Build and train state-of-the-art natural language processing models using BERT. Packt Publishing Ltd.
    67. Reddy, M. D. M., Basha, M. S. M., Hari, M. M. C., & Penchalaiah, M. N. (2021). Dall-e: Creating images from text. UGC Care Group I Journal, 8(14), 71-75.
    68. Rezwana, J., & Maher, M. L. (2022, June). Understanding user perceptions, collaborative experience and user engagement in different human-AI interaction designs for co-creative systems. In Proceedings of the 14th Conference on Creativity and Cognition (pp. 38-48).
    69. Riedl, M. O., & Young, R. M. (2010). Narrative planning: Balancing plot and character. Journal of Artificial Intelligence Research, 39, 217-268.
    70. Ruder, S. (2019). Neural transfer learning for natural language processing (Doctoral dissertation, NUI Galway).
    71. Ruiz-Real, J. L., Uribe-Toril, J., Torres, J. A., & De Pablo, J. (2021). Artificial intelligence in business and economics research: Trends and future. Journal of Business Economics and Management, 22(1), 98-117.
    72. Santhanam, S., Karduni, A., & Shaikh, S. (2020, April). Studying the effects of cognitive biases in evaluation of conversational agents. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1-13).
    73. Serban, I., Sordoni, A., Bengio, Y., Courville, A., & Pineau, J. (2016, March). Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of the AAAI conference on artificial intelligence (Vol. 30, No. 1).
    74. Tiwari, C. K., Bhat, M. A., Khan, S. T., Subramaniam, R., & Khan, M. A. I. (2023). What drives students toward ChatGPT? An investigation of the factors influencing adoption and usage of ChatGPT. Interactive Technology and Smart Education.
    75. Venkatesh, Viswanath, et al. "User acceptance of information technology: Toward a unified view." MIS quarterly (2003): 425-478.
    76. Waisberg, E., Ong, J., Masalkhi, M., Kamran, S. A., Zaman, N., Sarker, P., ... & Tavakkoli, A. (2023). GPT-4: a new era of artificial intelligence in medicine. Irish Journal of Medical Science (1971-), 192(6), 3197-3200.
    77. Wang, D. Q., Feng, L. Y., Ye, J. G., Zou, J. G., & Zheng, Y. F. (2023). Accelerating the integration of ChatGPT and other large‐scale AI models into biomedical research and healthcare. MedComm–Future Medicine, 2(2), e43.
    78. Wang, Y., Wang, L., Rastegar-Mojarad, M., Moon, S., Shen, F., Afzal, N., ... & Liu, H. (2018). Clinical information extraction applications: a literature review. Journal of biomedical informatics, 77, 34-49.
    79. Weizenbaum, J. (1966). ELIZA—a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36-45.
    80. Whittlestone, J., Nyrup, R., Alexandrova, A., & Cave, S. (2019, January). The role and limits of principles in AI ethics: Towards a focus on tensions. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (pp. 195-200).4o
    81. Wittink, D. R., & Cattin, P. (1989). Commercial Use of Conjoint Analysis: An Update. Journal of Marketing, 53(3), 91-96.
    82. Wu, Y., Zhang, Z., Kou, G., Zhang, H., Chao, X., Li, C. C., ... & Herrera, F. (2021). Distributed linguistic representations in decision making: Taxonomy, key elements and applications, and challenges in data science and explainable artificial intelligence. Information Fusion, 65, 165-178.
    83. Xu, A., Liu, Z., Guo, Y., Sinha, V., & Akkiraju, R. (2017, May). A new chatbot for customer service on social media. In Proceedings of the 2017 CHI conference on human factors in computing systems (pp. 3506-3510).
    84. Yaser, et al. Statistical machine translation. In: Final Report, JHU Summer Workshop. 1999. p. 1-42.

    下載圖示 校內：立即公開
    校外：立即公開
    QR CODE