本研究旨在設計與加強一套融合臉部情緒識別與眼球追蹤技術的智慧型電影推薦系統，藉此解決傳統推薦系統在即時情感反饋與使用者注意力分布考量上的不足。相較於僅依賴使用者的觀看歷史與偏好，本系統引入基於臉部表情分析的情緒辨識，並結合眼動及眼距的資料來捕捉使用者在觀影過程中的注意力焦點與感受深度，進一步提升推薦的準確性與個人化程度。
系統透過攝影機或預錄之觀影反應影片，運用深度學習模型即時分析使用者的情緒變化（如喜悅、悲傷、驚訝、憤怒等），並同步擷取眼球移動軌跡、注視點與注視時間等資訊。這些視覺與情緒資料將被量化為情感評分與注意力指標，作為推薦演算法的關鍵輸入，進而生成兼顧使用者歷史偏好、當下情緒狀態與視覺參與度的個性化電影推薦。
本系統將動態情感反應與視覺注意力同步納入推薦模型中，希望能強化推薦內容與使用者心理狀態的匹配程度，亦進一步改善整體觀影體驗與滿意度。

This study aims to design an intelligent movie recommendation system that integrates facial emotion recognition and eye-tracking technologies, addressing the limitations of traditional recommendation systems in capturing real-time emotional feedback and user attention. Unlike conventional methods that rely solely on viewing history and preferences, our approach incorporates facial expression-based emotion analysis and gaze data to better understand users’ emotional responses and visual engagement during movie watching, thereby enhancing recommendation accuracy and personalization.
The system utilizes a camera or pre-recorded viewer footage to capture users’ emotional reactions in real-time using deep learning models, identifying emotions such as joy, sadness, surprise, and anger. Simultaneously, it tracks eye movements, fixation points, and gaze duration to evaluate the user’s attention focus and engagement level. These emotional and visual data are then quantified into emotion scores and attention indicators, serving as key inputs to the recommendation algorithm. As a result, the system generates personalized movie recommendations that reflect not only user's historical preferences but also their real-time emotional states and attention patterns.
The innovation of this system lies in its ability to incorporate dynamic emotional reactions and visual attention into the recommendation process, thereby improving the alignment between recommended content and user's psychological states, while also enhancing the overall viewing experience and satisfaction.

[1] Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. IEEE Computer, 42(8), 30–37.
[2] Covington, P., Adams, J., & Sargin, E. (2016). Deep neural networks for YouTube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems (pp. 191–198).
[3] Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2017). AffectNet: A database for facial expression, valence, and arousal computing in the wild. IEEE Transactions on Affective Computing, 10(1), 18–31.
[4] Ng, H. W., & Winkler, S. (2014). A data-driven approach to cleaning large face datasets. In Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP) (pp. 343–347).
[5] Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016). Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10), 1499–1503.
[6] Pu, Q., & Hu, B. (2017). A hybrid recommendation system for movies based on collaborative filtering. Journal of Computer Applications, 37(2), 47–52.
[7] MovieLens Dataset. (n.d.). GroupLens. https://grouplens.org/datasets/movielens/
[8] Harper, F. M., & Konstan, J. A. (2015). The MovieLens datasets: History and context. ACM Transactions on Interactive Intelligent Systems (TiiS), 5(4), 19:1–19:19.
[9] Jain, P. P., Patil, Y. D., & Pawade, Y. (2023). Music and movie recommendation through emotion detection. International Journal of Novel Research and Development (IJNRD), 8.
[10] Joel, J. S., Thompson, B. E., Thomas, S. R., Kumar, T. R., Prince, S., & Bini, D. (2023). Emotion based music recommendation system using deep learning model. In 2023 International Conference on Inventive Computation Technologies (ICICT).
[11] Patel, A. A., & Dharwa, J. N. (2016). An integrated hybrid recommendation model using graph database. In 2016 International Conference on ICT in Business Industry & Government (ICTBIG).
[12] Manju, M. T., Jayashakthi, R., Nivethitha, R., & Lavanya, P. (2023). User-centric emotion modeling for next-generation recommender system. In 2023 International Conference on Research Methodologies in Knowledge Management, Artificial Intelligence and Telecommunication Engineering.
[13] OXXO.STUDIO. (2021–2024). Python 情緒分析入門. https://steam.oxxostudio.tw/category/python/ai/aiemotion.html#a1
[14] OpenCV. (2022, November 17). opencv/data/haarcascades. https://github.com/kipr/opencv/tree/master/data/haarcascades
[15] Al-Omair, O. M. (2023). Incorporating eye-tracking and facial expression recognition for preference prediction in recommender systems. In 2023 3rd International Conference on Computing and Information Technology (ICCIT).
[16] OXXO.STUDIO. (2021–2024). AI 臉部辨識教學. https://steam.oxxostudio.tw/category/python/ai/ai-face-dectection.html
[17] GroupLens. (2018, September). MovieLens Latest Datasets (ml-latest-small.zip). https://grouplens.org/datasets/movielens/latest/
[18] Pinterest. (n.d.). A general representation of training a Haar classifier. https://nl.pinterest.com/pin/39969515412911329/
[19] HackMD. (n.d.). AdaBoost classifier. https://hackmd.io/@yillkid/H1eMbtjZD
[20] SpencerPao (YouTube). (2021, May 24). Building a Recommendation System in Python [Video]. https://reurl.cc/qG406y
[21] Ho, A. T., Menezes, I. L. L., & Tagmouti, Y. (2006). E-MRS: Emotion-based movie recommender system. In Proceedings of the IADIS e-Commerce Conference, University of Washington Bothell, USA (pp. 1–8).
[22] Pixabay. (n.d.). Sleep, messy bed at night, wake up on pillow [Video]. Pixabay. https://pixabay.com/videos/sleep-messy-bed-night-wake-pillow-42966/
[23] Liao, T. L., Hou, Y. Y., & Tseng, Y. H. (2024). Movie Recommendation System Based on Facial Emotion Recognition. In Proceedings of the 2024 IEEE International Conference on Future Machine Learning and Data Science (FMLDS 2024) (pp. 141–146). IEEE.