簡易檢索 / 詳目顯示

研究生: 方品媛
Fang, Ping-Yuan
論文名稱: 具持續優化機制之形成性學習評量分析模型設計與技術開發
Design and Technological Development of a Formative Assessment and Analysis Model with Sustainable Optimization Mechanism
指導教授: 陳裕民
Chen, Yuh-Min
共同指導: 朱慧娟
Chu, Hui-Chuan
學位類別: 碩士
Master
系所名稱: 電機資訊學院 - 製造資訊與系統研究所
Institute of Manufacturing Information and Systems
論文出版年: 2024
畢業學年度: 112
語文別: 中文
論文頁數: 101
中文關鍵詞: 學習分析注意力機制學習成效預測重要影響因素分析
外文關鍵詞: learning analytics, attention mechanism, student performance prediction, factor analysis
相關次數: 點閱:139下載:26
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  • 數位學習已成為當前數位時代的主流學習方法,隨著人工智慧和深度學習技術的快速發展,學習分析也為數位學習提供了包括學習評量、學習成效預測、個人化學習等支持和應用。然而,當前的學習模式多為一次性學習與總結性評量,無法逐次推動學習的改善;其次,由於學習成效的影響因素複雜,也隨學習情境的異動而改變,傳統結果導向的總結性評量,無法針對學習過程的影響因素,提供相應的學習改善策略。
    本研究參考系統化OPDCA運作模式,提出一具持續優化機制的學習模式,同時參考資料科學分析框架,設計學習評量架構,再運用深度學習技術開發學習成效預測與原因分析模型。依據學習模式,學生進行多項學習任務與每項學習任務後的評量,包含成效評比與預測,以及透過注意力機制的可解釋性,識別學習的關鍵影響因素,以掌握學習成效與潛在的學習失敗以及原因,據此提供下一階段學習改善建議,從而實現學習策略的階段性動態調整與持續優化。
    本研究針對分析模型進行準確性評量。結果顯示,所開發的CNN結合注意力機制模型於學習成效預測的表現優傳統機器學習方法,準確率達93%,F1-Score為92.46,且隨著資料量增加,準確度不斷提升。同時,本研究依所提學習模式,開發一數位閱讀理解素養培養平台,進行持續學習評量機制之應用有效性評量。實驗顯示接受學習評量之實驗組學生的學習成效,優於未提供學習評量之對照組學生的表現,其期末考平均T分數提高8.8分,且兩者具顯著差異(p=0.000)。
    本研究所開發之學習成效評量方法具創新性與準確性,且於所提之學習模式的應用上,亦透過持續的階段性評量掌握變動的學習影響因素,能有效推動學習的持續改善。

    Digital learning has become the mainstream method in the current digital era. With rapid advancements in AI and deep learning, learning analytics now supports digital learning through assessment, performance prediction, and personalized learning. However, current models often focus on one-time learning and summative assessments, failing to promote continuous improvement. Moreover, due to the complexity of factors influencing learning outcomes, which change with context, traditional result-oriented assessments cannot provide strategies to address these factors.
    This study proposes a learning model with continuous optimization based on the OPDCA cycle and data science frameworks, using deep learning for performance prediction and causal analysis. Students undergo multi-stage learning with assessments after each stage, including performance comparisons and predictions. Through an attention mechanism, key factors influencing learning are identified to understand effectiveness, potential failures, and their causes, providing improvement suggestions for the next stage.
    The study first evaluates the model's accuracy. Results show the developed CNN model with attention mechanism outperforms traditional methods in predicting learning effectiveness, achieving 93% accuracy and 92.46 F1-score, improving with increased data. A digital reading comprehension platform was developed to evaluate the model's effectiveness. Experiments show significantly improved performance in the test group, with a 8.8 increase in final exam scores (p=0.000).
    This innovative assessment method offers accuracy and, through continuous phased evaluations, effectively promotes ongoing learning improvement by capturing changing influential factors.

    摘要 3 致謝 9 目錄 10 表目錄 13 圖目錄 14 第一章、 緒論 16 1.1 研究背景 16 1.2 研究動機 18 1.3 研究目的 19 1.4 研究問題 19 1.5 研究項目與方法 20 1.6 研究步驟 22 第二章、 文獻探討 24 2.1 領域文獻探討 24 2.1.1 學習分析 24 2.1.2 品質管理 25 2.1.3 形成性評量 26 2.1.4 學習非智力因素 27 2.2 相關技術探討 27 2.2.1 注意力機制(Attention Mechanism) 27 2.2.2 深度學習與神經網路模型 29 2.2.3 遷移學習(Transfer Learning) 30 2.3 相關研究探討 31 2.4 文獻探討總結 33 第三章、 方法設計 35 3.1 具持續優化機制之學習模式設計 35 3.2 持續優化機制 36 3.3 學習評量分析架構設計 39 3.4 數位讀寫素養培養模式設計 41 3.5 影響因素模型設計 42 3.5.1 學生個人模型 43 3.5.2 外在影響因素 45 3.6 學生學習足跡模型設計 45 第四章、 資料分析 47 4.1 資料集 47 4.2 隱性特徵轉換 49 4.3 資料前處理 51 4.3.1 特徵編碼 51 4.3.2 資料正規化 53 4.3.3 資料重平衡 53 第五章、 模型設計與技術開發 55 5.1 模型開發流程 55 5.2 重要影響因素分析模型架構設計 56 5.2.1 需求分析 56 5.2.2 模型架構設計 57 5.3 模型技術開發 60 5.3.1 遷移學習(Transfer Learning) 60 5.3.2 持續優化機制 61 5.3.3 神經網絡模型 62 5.4 實驗 64 5.4.1 驗證指標 64 5.4.2 實驗流程 66 5.4.3 實驗環境 69 5.5 實驗結果分析 70 5.5.1 資料前處理層面 70 5.5.2 模型優化層面 72 第六章、 應用有效性驗證 79 6.1 基於形成性學習分析之數位讀寫素養培養平台 79 6.1.1 學生學習前台 79 6.1.2 老師學習管理後台 80 6.2 方法應用驗證與分析 82 6.2.1 實驗設計 82 6.2.2 指標設計 83 6.2.3 實驗對象 85 6.2.4 實驗執行方式 85 6.2.5 實驗資料準備 85 6.3 實驗結果分析 86 第七章、 結論與未來展望 92 7.1 結論 92 7.2 未來展望 93 參考文獻 94

    中文參考文獻
    何子岳(2022)。基於深度學習之文章摘要提取技術研發:以階層式文章摘要能力培養之應用為例。成功大學製造資訊與系統研究所學位論文。
    巫清賢(2023)。基於 BERT 之克漏字問題生成方法與應用研究。成功大學製造資訊與系統研究所學位論文。
    品學堂(2023)。閱讀未來國際教育論壇。
    陳木金(1998)談美感教育與情意教育。北縣教育,22,24-28。
    教育部(2019)。十二年國民基本教育特殊教育課程實施規範。臺北市: 教育部。
    普皓群(2021)。基於深度學習之心智圖自動產生方法與技術研發:以數位閱 讀與寫作能力培養之應用為例。成功大學製造資訊與系統研究所學位論文。
    蘇冠瑜(2023)。基於 BERT 與 GPT 之文章概念圖 生成方法與應用研究。成功大學製造資訊與系統研究所學位論文。

    英文參考文獻
    Amrieh, E. A., Hamtini, T., & Aljarah, I. (2015). Preprocessing and analyzing educational data set using X-API for improving student's performance. In Applied Electrical Engineering and Computing Technologies (AEECT), 2015 IEEE Jordan Conference on, 1-5, IEEE.
    Amrieh, E. A., Hamtini, T., & Aljarah, I. (2016). Mining Educational Data to Predict Student’s academic Performance using Ensemble Methods. International Journal of Database Theory and Application, 9(8), 119-136.
    Andersson, C., & Palm, T. (2017). The impact of formative assessment on student achievement: A study of the effects of changes to classroom practice after a comprehensive professional development programme. Learning and instruction, 49, 92-102.
    Adarkwah, M. A. (2021). The power of assessment feedback in teaching and learning: a narrative review and synthesis of the literature. SN Social Sciences, 1(3), 75.
    Awan, U., Shamim, S., Khan, Z., Zia, N. U., Shariq, S. M., & Khan, M. N. (2021). Big data analytics capability and decision-making: The role of data-driven insight on circular economy performance. Technological Forecasting and Social Change, 168, 120766.
    Adedoyin, O. B., & Soykan, E. (2023). Covid-19 pandemic and online learning: the challenges and opportunities. Interactive learning environments, 31(2), 863-875.
    Baker, R. S., & Yacef, K. (2009). The state of educational data mining in 2009: A review and future visions. Journal of educational data mining, 1(1), 3-17.
    Black, P., & Wiliam, D. (2018). Classroom assessment and pedagogy. Assessment in education: Principles, policy & practice, 25(6), 551-575.
    Bales, K. (2019). 13 creative examples of informal assessments for the classroom: Simple and stress-free observation-based assessments.
    Balali, F., Nouri, J., Nasiri, A., Zhao, T., Balali, F., Nouri, J., ... & Zhao, T. (2020). Data analytics. Data Intensive Industrial Asset Management: IoT-based Algorithms and Implementation, 105-113.
    Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research, 16, 321-357.
    Chatti, M. A., Agustiawan, M. R., Jarke, M., & Specht, M. (2012). Toward a personal learning environment framework. In Design, implementation, and evaluation of virtual learning environments, 20-40.
    Carless, D. (2017). Scaling up assessment for learning: Progress and prospects. Scaling up assessment for learning in higher education, 3-17.
    Cerda, P., Varoquaux, G., & Kégl, B. (2018). Similarity encoding for learning with dirty categorical variables. Machine Learning, 107(8), 1477-1494.
    Chen, F., & Cui, Y. (2020). Utilizing Student Time Series Behaviour in Learning Management Systems for Early Prediction of Course Performance. Journal of Learning Analytics, 7(2), 1-17.
    Chen, C. M., Chen, L. C., & Horng, W. J. (2021). A collaborative reading annotation system with formative assessment and feedback mechanisms to promote digital reading performance. Interactive Learning Environments, 29(5), 848-865.
    Dixson, D. D., & Worrell, F. C. (2016). Formative and summative assessment in the classroom. Theory into practice, 55(2), 153-159.
    Díez-Palomar, J., García-Carrión, R., Hargreaves, L., & Vieites, M. (2020). Transforming students’ attitudes towards learning through the use of successful educational actions. PloS one, 15(10), e0240292.
    Desiani, A., Yahdin, S., Kartikasari, A., & Irmeilyana, I. (2021). Handling the imbalanced data with missing value elimination SMOTE in the classification of the relevance education background with graduates employment. IAES International Journal of Artificial Intelligence, 10(2), 346.
    Dobashi, K., Ho, C. P., Fulford, C. P., Lin, M. F. G., & Higa, C. (2022). Learning pattern classification using moodle logs and the visualization of browsing processes by time-series cross-section. Computers and Education: Artificial Intelligence, 3, 100105.
    Enarsson, L. (1998). Evaluation of suppliers: how to consider the environment. International Journal of Physical Distribution & Logistics Management, 28(1), 5-17.
    Fleming, N., & Mills, C. (2001). VARK: A guide to learning styles. Retrieved November, 30, 2004.
    Freitas, A. T. (2023). Data-Driven Approaches in Healthcare: Challenges and Emerging Trends. Multidisciplinary Perspectives on Artificial Intelligence and the Law, 65-80.
    Ge, X. (2021). Emotion matters for academic success. Educational Technology Research and Development, 69(1), 67-70.
    Huang, X., Jing, J., Chen, X. B. & Li, Y. F. (2001). Study on Children with Learning Disability and Family Influential Factors. Chinese Journal of Child Health Care, 9(5), 300-302.
    He, H. l., Yang, H. L., Yang, Y. F., Liu, L. & Lei, X. M. (2005). Associations of Influential Factors and Study Record of Primary Students. Chinese Journal of Natural Medicines, 7(4), 287-290.
    Hrastinski, S. (2008). What is online learner participation? A literature review. Computers & education, 51(4), 1755-1765.
    Hollander, M., Wolfe, D. A., & Chicken, E. (2013). Nonparametric statistical methods. John Wiley & Sons.
    Holliday, J. D., Sani, N., & Willett, P. (2015). Calculation of substructural analysis weights using a genetic algorithm. Journal of chemical information and modeling, 55(2), 214-221.
    Jensen, L. X., Bearman, M., & Boud, D. (2021). Understanding feedback in online learning–A critical review and metaphor analysis. Computers & Education, 173, 104271.
    Jurāne-Brēmane, A. (2023). Digital assessment in technology-enriched education: Thematic review. Education Sciences, 13(5), 522.
    Hwang, G. J., Chu, H. C., & Yin, C. (2017). Objectives, methodologies and research issues of learning analytics. Interactive Learning Environments, 25(2), 143-146.
    Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education promises and implications for teaching and learning. Center for Curriculum Redesign.
    Kemsley, D. H., Martinez, T. R., & Campbell, D. (1992). A survey of neural network research and fielded applications. International journal of neural networks, 2, 123-123.
    Knight, J. E., & Allen, S. (2012). Applying the PDCA cycle to the complex task of teaching and assessing public relations writing. International Journal of Higher Education, 1(2), 67-83.
    Khalil, M., & Ebner, M. (2015). Learning analytics: principles and constraints. In Edmedia+ innovate learning, 1789-1799.
    Khan, I., Ahmad, A. R., Jabeur, N., & Mahdi, M. N. (2021). A Conceptual Framework to Aid Attribute Selection in Machine Learning Student Performance Prediction Models. International Journal of Interactive Mobile Technologies, 15(15).
    LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278-2324.
    Lin, C. Y. (2004). Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, 74-81.
    LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436-444.
    Liao, C. H., & Wu, J. Y. (2022). Deploying multimodal learning analytics models to explore the impact of digital distraction and peer learning on student performance. Computers & Education, 190, 104599.
    Lumacad, G. S., & Namoco, R. A. (2023). Multilayer Perceptron Neural Network Approach to Classifying Learning Modalities Under the New Normal. IEEE Transactions on Computational Social Systems.
    Ismail, S. M., Rahul, D. R., Patra, I., & Rezvani, E. (2022). Formative vs. summative assessment: impacts on academic motivation, attitude toward learning, test anxiety, and self-regulation skill. Language Testing in Asia, 12(1), 40.
    Kitto, K., Buckingham Shum, S., & Gibson, A. (2018). Embracing imperfection in learning analytics. In Proceedings of the 8th international conference on learning analytics and knowledge, 451-460.
    Means, B., Toyama, Y., Murphy, R., Bakia, M., & Jones, K. (2009). Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies.
    Means, B., Bakia, M., & Murphy, R. (2014). Learning online: What research tells us about whether, when and how. Routledge.
    Metsärinne, M., & Kallio, M. (2016). How are students’ attitudes related to learning outcomes? International Journal of Technology and Design Education, 26, 353-371.
    Menéndez, I. Y. C., Napa, M. A. C., Moreira, M. L. M., & Zambrano, G. G. V. (2019). The importance of formative assessment in the learning teaching process. International journal of social sciences and humanities, 3(2), 238-249.
    Morris, R., Perry, T., & Wardle, L. (2021). Formative assessment and feedback for learning in higher education: A systematic review. Review of Education, 9(3), e3292.
    Mohamed Nafuri, A. F., Sani, N. S., Zainudin, N. F. A., Rahman, A. H. A., & Aliff, M. (2022). Clustering analysis for classifying student academic performance in higher education. Applied Sciences, 12(19), 9467.
    Nguyen, T. (2015). The effectiveness of online learning: Beyond no significant difference and future horizons. MERLOT Journal of online learning and teaching, 11(2), 309-319.
    Nguyen, P. H. G., & Vo, C. T. N. (2018). A CNN model with data imbalance handling for course-level student prediction based on forum texts. Springer International Publishing, 11055, 479-490.
    Nasif, A., Othman, Z. A., & Sani, N. S. (2021). The deep learning solutions on lossless compression methods for alleviating data load on IoT nodes in smart cities. Sensors, 21(12), 4223.
    Nugraha, D. Y., & Nugraha, D. (2021). The Correlation between Learning Motivation and Learning Outcomes on Mathematics Subjects in XII Science Class Senior High School 4 Bone. Anatolian Journal of Education, 6(1), 157-166.
    Neelakandan, N. (2023). The Importance Of Learning Analytics To Improve The Quality Of eLearning.
    Organisation for Economic Co-operation and Development (2017). PISA 2015 assessment and analytical framework: Science, reading, mathematic, financial literacy and collaborative problem solving (Rev. ed.)
    OECD. (2019). Trends Shaping Education 2019, OECD Publishing, Paris, https://doi.org/10.1787/trends_edu-2019-en.
    Ouahi, M., Khoulji, S., & Kerkeb, M. L. (2024). Original Research Article Predictive assessment of learners through initial interactions with encoding techniques in deep learning. Journal of Autonomous Intelligence, 7(4).
    Popham, W. J. (2008). Transformative assessment. ASCD.
    Prakasa, Y. F. (2023). Implementation of PDCA Metacognitive Learning Strategies to Metacognitive Abilities, Science Process Skills, and Student Learning Outcomes. Jurnal Pendidikan Profesi Guru, 1(1), 12-26.
    Romero, C., & Ventura, S. (2007). Educational data mining: A survey from 1995 to 2005. Expert systems with applications, 33(1), 135-146.
    Raid, J., Gehan, B., Mohammed, A., Nashwa, I., Eman, I. & Ahmed, G. (2017). Six sigma: An approach to improve student satisfaction. Conference: 4th FUE International Conference of Pharmaceutical Sciences.
    RAHİM, M. A., UTSHA, A. M., Bhuiyan, I. A., & Miah, M. R. (2019). Academic performance improvement procedure of university students applying total quality management: a case of bangladesh. International e-Journal of Educational Studies, 3(5), 29-38.
    Renò, V., Stella, E., Patruno, C., Capurso, A., Dimauro, G., & Maglietta, R. (2022). Learning analytics: analysis of methods for online assessment. Applied Sciences, 12(18), 9296.
    Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52(3-4), 591-611.
    Škrlj, B., Džeroski, S., Lavrač, N., & Petković, M. (2020). Feature importance estimation with self-attention networks. In ECAI 2020, 1491-1498.
    Sanpanich, N. (2021). Investigating Factors Affecting Students' Attitudes toward Hybrid Learning. REFLections, 28(2), 208-227.
    Sheikholeslami, S., Meister, M., Wang, T., Payberah, A. H., Vlassov, V., & Dowling, J. (2021). Autoablation: Automated parallel ablation studies for deep learning. In Proceedings of the 1st Workshop on Machine Learning and Systems, 55-61.
    Schneider, S., Beege, M., Nebel, S., Schnaubert, L., & Rey, G. D. (2022). The cognitive-affective-social theory of learning in digital environments (CASTLE). Educational Psychology Review, 34(1), 1-38.
    Swamy, V., Marras, M., & Käser, T. (2022). Meta transfer learning for early success prediction in MOOCs. In Proceedings of the ninth ACM conference on learning@ scale, 121-132.
    Subhranildey. (2024). Student performance for Multiple Linear Regression [Dataset]. https://www.kaggle.com/datasets/subhranildey85/student-performance-for-multiple-linear-regression
    Tomek, I. (2010). Two modifications of CNN. IEEE Transactions on Systems, Man, and Cybernetics, 6(11), 769–772.
    Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157-167.
    UNESCO. (2020). COVID-19 Educational Disruption and Response. https://en.unesco.org/themes/education-emergencies/coronavirus-school-closures
    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.
    Verger, M., & Escalante, H. J. (2021). Predicting students' performance in online courses using multiple data sources.
    Vu, T., Magis-Weinberg, L., Jansen, B. R., van Atteveldt, N., Janssen, T. W., Lee, N. C., ... & Meeter, M. (2022). Motivation-achievement cycles in learning: A literature review and research agenda. Educational Psychology Review, 34(1), 39-71.
    Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016). A survey of transfer learning. Journal of Big data, 3, 1-40.
    Wise, A. F., & Vytasek, J. (2017). Learning analytics implementation design. Handbook of learning analytics, 1, 151-160.
    Wise, A. F., & Jung, Y. (2019). Teaching with analytics: Towards a situated model of instructional decision-making. Journal of Learning Analytics, 6(2), 53-69.
    Winstone, N. E., & Boud, D. (2022). The need to disentangle assessment and feedback in higher education. Studies in higher education, 47(3), 656-667.
    Wei, X., Saab, N., & Admiraal, W. (2024). What rationale would work? Unfolding the role of learners’ attitudes and motivation in predicting learning engagement and perceived learning outcomes in MOOCs. International Journal of Educational Technology in Higher Education, 21(1), 5.
    Weigand, H. G., Trgalova, J., & Tabach, M. (2024). Mathematics teaching, learning, and assessment in the digital age. ZDM–Mathematics Education, 1-17.
    Xu, B. (2021). Artificial Intelligence-based Online Customized Learning Direction Guidance Model. International Journal of Science, 8(12), 121-133.
    Yang, A. C., Flanagan, B., & Ogata, H. (2022). Adaptive formative assessment system based on computerized adaptive testing and the learning memory cycle for personalized learning. Computers and Education: Artificial Intelligence, 3, 100104.
    Zhao, S. Y., Tang, M. X., Liu, J., Wu, X. M. & Liu, X. (2007). An Analysis of the Current Status of Learning Ability and Influencing Factors Related of Pupils in Changsha. Journal of Changsha Medical College, 6(20), 11-16.
    Zhang, L., Zhang, C., Gao, R., Yang, R., & Song, Q. (2016). Using the SMOTE technique and hybrid features to predict the types of ion channel-targeted conotoxins. Journal of Theoretical Biology, 403(17923), 75-84.
    Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education–where are the educators?. International Journal of Educational Technology in Higher Education, 16(1), 1-27.
    Zilvinskis, J., & Willis III, J. E. (2019). Learning Analytics in Higher Education: A Reflection. InSight: A Journal of Scholarly Teaching, 14, 43-54.

    下載圖示
    校外:立即公開
    QR CODE