網路的視訊會議系統已經成為很重要的訊息溝通工具，對於分散在各地的親朋好友，可藉由網路視訊系統互相的見上一面，但是在這個繁忙的時代，不可能每當開啟網路視訊時，對方一定在電腦前與您對談；因此本研究提出基於臉部情緒辨識的視訊代理系統，讓忙碌的人也能夠在無法親自做視訊對談的情況下，讓本系統藉由對對方之情緒辨識結果來回應視訊，以讓希望利用視訊聯繫的對象能夠得到一些回應。本系統實現一個基本的網路視訊系統，讓使用者可以在此系統上利用基本視訊軟體的功能建立互相溝通，提供的功能包含即時文字訊息、視訊影像及聲音；另外，在此基本架構之下更進一步的建立視訊代理系統，當使用者要求視訊的對象不在線上的時候，本系統會切換為視訊代理系統，由代理系統與使用者進行連線，利用視訊接收到的使用者影像進行臉部情緒辨識，將辨識得到的五種情緒-開心、生氣、難過、驚訝及中性表情，以視訊對象預先錄製好的影片內容回應，回應內容針對使用者情緒說一段安撫的話，讓使用者透過視訊代理系統感覺到視訊對象安慰自己，而讓情緒獲得相應的安撫。本研究提出的情緒辨識系統需要先建立一張使用者的中性表情影像，擷取此中性影像的特徵值，再針對由網路攝影機獲得的即時影像計算得到當前表情的特徵值，藉由當前影像與中性表情的特徵值相互比較，進而計算出目前的情緒結果；本系統在實際狀況中測試得到93.5的辨識率，並且能夠以代理系統給予相對應的安撫影像。

Internet video conferencing system has become a very important message communication tool so far. People located at different places can be easily connected via Internet video conferencing system. In this thesis, a live video conferencing agent system has been proposed. The proposed system provides the basic functions, text messages, video images and audio of a video conferencing system. It also designed as an agent system. While the contact person is not on line, the video conferencing system is switched to act as an agent system. The agent system uses facial emotion recognition system to decide the user’s emotion from the images captured by webcam and response the proper information extracted from the data base according to the decided user’s emotion. Five emotions: happy, angry, sad, surprised, and neutral expressions are provided and some corresponding information for each emotion has been created in the data base. The user’s emotion is determined via the comparing the features of captured on line user image with that of user’s neutral expression. This system is tested in real conditions to get recognition rate of 93.5, and the agent system could response the proper information in the result.

[1] R. L. Hsu, M. Abdel-Motteleb, A. K. Jain, “Face detection in color images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 696-706, May 2002.
[2] K. Sobottka, I. Pitas, “Extraction of Facial Regions and Features using Color and Shape Information,” Pattern Recognition,1996 ., Proceedings of the 13th International Conference on, vol. 3, pp. 421-425, August 1996.
[3] C. Garcia, G. Tziritas, “Face Detection Using Quantized Skin Color Regions Merging and Wavelet Packet Analysis,” IEEE Transaction on Multimedia, vol 1,no. 3, 1999.
[4] W. S. Yambor, B. A. Draper, J. R. Beveridge, “Analyzing PCA-based Face Recognition Algorithms: Eigenvector Selection and Distance Measures,” In Empirical Methods in Computer Vision. World Scientific Press, 2002.
[5] I. Kotsia, I. Pitas, “Facial Expression Recognition in Image Sequences Using Geometric Deformation Features and Support Vector Machines,” IEEE Transaction on Image Processing, vol. 16,pp.172-187,January 2007.
[6] S. Bashyal, G. K. Venayagamoorthy, “Recognition of facial expressions using Gabor wavelets and learning vector quantization,” Engineering Applications of Artificial Intelligence, vol. 21, pp. 1056-1064, October 2008.
[7] P. Ekman, W. V. Friesen, ” Facial Action Coding System: A Technique for the Measurement of Facial Movement,” San Francisco: Consulting Psychologists Press, 1978.
[8] 胡冠宇, "基於膚色之裸體影像偵測之研究", 碩士論文, 國立成功大學工程科學系碩士班,2004。
[9] A. D. Bimbo, ” Visual information retrieval,” Morgan Kauffman Publishers, Inc., San Francisco, California, 1999.
[10] CCIR, “Encoding parameters of digital television for studios,” CCIR Recommendation 601-2, Int. Radio Consult. Committee, Geneva, Switzerland, 1990.
[11] D. Chai, A. Bouzerdoum, “A Bayesian approach to skin color classification in YCbCr color space,” Machine Learning and Cybernetics, 2002. Proceedings. 2002 International Conference on, IEEE, vol. 2, pp. 1133-1138, Nov. 2002.
[12] P. Viola, M. Jones, “Robust Real-Time Object Detection,” International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, May 2004.

[13] P. Viola, M. Jones, “Detecting Pedestrians Using Patterns of Motion and Appearance,” International Journal of Computer Vision, vol. 63, no. 2, pp. 153-161, 2005.
[14] C. Papageorgiou, M. Oren, T. Poggio, “A general framework for Object Detection,” In International Conference on Computer Vision, vol. 1, pp. 555-562, January 1998.
[15] Y. Freund, R.E. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” Journal of Computer and System Sciences, Vol. 55, pp. 119-139, 1997.
[16] G. Woods 原著, 繆紹綱譯, 數位影像處理 Digital image processing, 3rd Ed. 普林斯頓國際有限公司, May 2009.
[17] M. Ilbeygi, H. Shah-Hosseini,” A novel fuzzy facial expression recognition system based on facial feature extraction from color face images,” Engineering Applications of Artificial Intelligence, vol. 25, pp. 130-146, 2012.
[18] Y. Li, S. Wang, Y. Zhao, Q. Ji, “Simultaneous Facial Feature Tracking and Facial Expression Recognition,” IEEE Trans. Image Processing, vol. 22, no. 7, pp. 2559 - 2573, July 2013.
[19] O. Langner, R. G. Dotsch, G. Bijlstra , D. H. J. Wigboldus, “ Presentation and validation of the Rad boud Faces Database,” Cognition and Emotion, vol. 24, no. 8, pp. 1377–1388, 2010.