情緒辨識在人機介面上扮演著重要的角色，它的應用範圍相當廣泛，像是遠距教學，孩童教學，健康照護，或是普及運算領域的各種應用等。本論文旨在提出用於情緒辨識的改良式臉部特徵擷取方式與三角形臉部特徵，並定義適用於臉部情緒的適應函數，最後透過遺傳演算法決定最佳的臉部特徵。在前處理部分，以改良過的動態輪廓模型抓取臉部的特徵點，藉以克服一般影像處理遇到環境光的影響，以及頭轉動導致特徵擷取不準確的影響，並擷取具有情緒代表性的三角形臉部特徵。實驗中採用多種機器學習模式以評估我們所提的方法，用於JAFFE靜態臉部情緒資料庫與eNTERFACE影像資料庫，並分別評估七種與六種情緒辨識結果，實驗結果顯示透過統計資訊所決定的特徵點，其JAFFE資料庫情緒辨識結果為65.1%，eNTERFACE影像資料庫辨識結果為50%，而透過我們提出新的適應性函數所挑選最佳的臉部特徵點，其JAFFE資料庫辨識結果為70.2%，eNTERFACE影像資料庫辨識結果為56.7%，新的適應函數用於在此兩資料庫之情緒辨識上，大約可提昇5%的辨識結果。

Emotions play major roles in human computer interaction, and emotion recognition system is often encountered in distance education, children education, health care, even some pervasive computing applications. In this paper, we propose a novel triangular facial feature extraction method to recognize emotions, also define a fitness function suitable for facial emotion recognition, and at the end we use Genetic Algorithm to determine the optimal facial features. In the preprocessing part, we use modified Active Shape Model (ASM) to extract facial feature points to avoid environmental conditions, and also extract representative triangular facial features. We adopt various machine learning methods to evaluate the proposed method experimenting on the JAFFE and eNTERFACE data sets for recognizing seven and six emotions respectively. The experimental results show that based on the statistical features 65.1% recognition rate was achieved in the JAFFE data set and 50% in the eNTERFACE data set, and based on the defined fitness function 70.2% recognition rate was achieved in the JAFFE data set and 56.7% in the eNTERFACE data set. It can increase about 5% recognition rate based on the fitness function we proposed in these two data sets.

[1] Q. Luo and H. Tan, ”Facial and Speech Recognition Emotion in Distance Education System,” International Conference on Intelligent Pervasive Computing, pp. 483-486, 2007.
[2] Z. Zeng, J. Tu, M. Liu, T. S. Huang, B. Pianfetti, D. Roth, and S. Levinson, “Audio-Visual Affect Recognition,” IEEE Transaction on Multimedia, vol. 9, no. 2, 2007.
[3] R. W. Picard, Affective Computing. Cambridge, MA: MIT Press, 1997.
[4] Y. Tian, T. Kanade and J. F. Cohn, “Recognizing Action Units for Facial Expression Analysis,” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 23, no. 2, pp. 97-115, 2001.
[5] M. Pantic and L. J. M. Rothkrantz, “Automatic Analysis of Facial Expressions: The State of the Art,” IEEE Transaction on Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1424-1445, 2000
[6] D. Ververidis, C. Kotropoulos and I. Pitas, “Automatic Emotional Speech Classification,” IEEE International Conference on Acoustics, Speech, and Signal Processing, Montreal, Quebec, Canada, pp 593-596, 2004.
[7] B. Schuller, G. Rigoll and M. Lang, “Speech Emotion Recognition Combining Acoustic Features and Linguistic Information in a Hybrid Support Vector Machine-Belief Network Architecture,” IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 1, pp. 577-580, 2004.
[8] P. S. Aleksic and A. K. Katsaggelos, “Audio-Visual Biometrics,” Proceeding of the IEEE, Vol. 94, No. 11, pp. 2025-2044, 2006.
[9] K. Takahashi and I. Sugimoto, “Feasibility of Emotion Recognition from Breath Gas Information,” IEEE International Conference on Advanced Intelligent Mechatronics, 2008.
[10] G. A. Tsihrintzis, M. Virvou, E. Alepis and I. O. Stathopoulou “Towards Improving Visual-Facial Emotion Recognition through Use of Complementary Keyboard-Stroke Pattern Information,” International Conference on Information Technology, 2008.
[11] H. R. LV, Z. L. Lin, W. J. Yin, and J. Dong, “Emotion Recognition on Pressure Sensor Keyboard,” IBM China Research Lab, 2008.
[12] L. C. De Silva, T. Miyasato and R. Nakatsu, “Facial Emotion Recognition Using Multi-modal Information,” IEEE International Conference on Information, Communications and Signal Processing, pp. 397-401, 1997.
[13] L. C. De Silva, “Audiovisual Emotion Recognition,” IEEE International Conference on Systems, Man and Cybernetics, pp. 649-654, 2004.
[14] H. J. Go, K. C. Kwak, D. J. Lee and M. G. Chun, “Emotion Recognition from the Facial Image and Speech Signal,” International Conference on Instrumentation, Control and Information, vol. 3, pp. 2890-2895, 2003.
[15] Y. Wang and L. Guan, “Recognizing Human Emotion from Audiovisual Information,” IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 1125-1128, 2005.
[16] M. J. Han, J. H. Hsu, K. T. Song and F. Y. Chang, “A New Information Fusion Method for SVM-Based Robotic Audio-Visual Emotion Recognition,” IEEE International Conference on Systems, Man and Cybernetics, pp. 2656–2661, 2007.
[17] M. Fujita, ”On Activating Human Communications with Pet-type Robot AIBO,” Proceeding of the IEEE, vol. 92, no. 11, pp. 1804-1813, 2004.
[18] A. Mehrabian, “Communication without Words,” Psychol.Today, vol. 2, no. 4, pp. 53-56, 1968.
[19] K. Anderson and P. W. McOwan, “A Real-Time Automated System for the Recognition of Human Facial Expressions,” IEEE Transaction on Systems, Man, and Cybernetics-part B: vol.36, no. 1, 2006.
[20] M. H. Bindu, P. Gupta, U. S. Tiwary, “Cognitive Model–Based Emotion Recognition From Facial Expressions for Live Human Computer Interaction,” IEEE Symposium on Computational Intelligence in Image and Signal Processing, pp. 351–356, 2007.
[21] I. Buciu, C. Kotrropoulus and I. Pitas: “ICA and Gabor Representation for Facial Expression Recognition,” International Conference on Image Processing, pp.855-8, 2003.
[22] B. Scassellati, “Eye Finding via Face Detection for a Foveated Active Vision System,” National Conference on Artificial Interlligence, pp. 969-976, 1998.
[23] T. F. Cootes, G. J. Taylor, D. Cooper, and J. Graham, “Active Shape Models-Their Training and Application,” Computer Vision and Image Understanding, vol. 61, no. 1, pp. 38-59, 1995.
[24] Y. Yoshitomi, S. Kim, T. Kawano, and T. Kitazoe, “Effect of Sensor Fusion for Recognition of Emotional States Using Voice, Face Image and Thermal Image of Face,” IEEE International Workshop on Robot and Human Interactive Communication, pp. 178–183, 2000.
[25] N. Esau, E. Wetzel, L. Kleinjohann and B. Kleinjohann, “Real-Time Facial Expression Recognition Using a Fuzzy Emotion Model,” IEEE International Conference on Fuzzy Systems, pp. 351-356, 2007.
[26] P. Viola and M. J. Jones, “Rapid Object Detection using a Boosted Cascade of Simple Features,” IEEE Conference on Computer Vision and Pattern Recognition, 2001.
[27] J. Holland, “Adaption in natural and artificial systems,” MIT Press, 1975.
[28] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning Internal Representations by Error Propagation,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart, J. L. McClelland et al. eds, MIT Press, Cambridge, MA, Vol. 1, pp. 318~362, 1986.
[29] C. M. Bishop, “Pattern Recognition and Machine Learning,” Springer, 2006.
[30] M. J. D. Powell, “Radial Basis Function Approximations to Polynomials,” Proceeding of 12th Biennial Numerical Analysis Conference, pp. 223–241, 1987.
[31] B. E. Boser, I. M. Guyon, and V. N. Vapnik, “A Training Algorithm for Optimal Margin Classifiers,” Proceedings of the Fifth Annual Workshop on Computational Learning Theory 5, pp. 144-152, 1992.
[32] I. H. witten and E. Frank, “Data Mining: Practical Machine Learning Tools and Techniques, 2nd Edition,” Morgan Kaufmann, San Francisco, 2005.
[33] M. J. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, “Coding Facial Expressions with Gabor Wavelets,” IEEE International Conference on Automatic Face and Gesture Recognition, pp. 200-205, 1998.
[34] O. Martin, I. Kotsia, B. Macq and I. Pitas, “The eNTERFACE'05 Audio-Visual Emotion Database,” IEEE International Conference on Data Engineering Workshops, 2006.
[35] M. Lyons, J. Budynek, and S. Akamastu, “Automatic Classification of Single Facial Images,” IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol.21, pp. 1357-1362, 1999.
[36] Y. Shinohara and N. Otsu ”Facial Expression Recognition Using Fisher Weight Maps,” IEEE Conference. on Automatic Face and Gesture Recognition, pp. 499-504, 2004.
[37] X. Feng, “Facial Expression Recognition Based on Local Binary Patterns and Coarse-to Fine Classification,” The 4th International Conference on Computer and Information Technology, pp. 178-183, 2004.
[38] Y. Horikawa, “Facial Expression Recognition using KCCA with Combining Correlation Kernels and Kansei Information,” International Conference on Computation Science and its Applications, pp. 489-498, 2007.
[39] H. B. Deng, L. W. Jin, L. X. Zhen, and J. C. Huang, “A New Facial Expression Recognition Method Based on Local Gabor Filter Bank and PCA plus LDA,” International Journal of Information Technology, 2005.
[40] X. Feng, “Automatic Facial Expression Recognition Using both Local and Global Information,” Proceedings of the 25th Chinese Control Conference, 2006.
[41] S. Liao, W. Fan, C. S. Chung, and D. Y. Yeung, “Facial Expression Recognition Using Advanced Local Binary Patterns, Tsallis Entropies and Global Appearance Features,” IEEE International Conference on Image Processing (ICIP), pp. 665-668, 2006.
[42] C. Shan, S. Gong, and P. W. McOwan, “Facial expression recognition based on Local Binary Patterns: A comprehensive study,” Image and Vision Computing, pp. 803-816, 2009.
[43] M. Turk and A. Pentland, “Eigenfaces for Recognition,” Journal of Cognitive Neuroscience, vol. 3, pp. 71–86, 1991.
[44] Simon S. Haykin, “Neural Networks: A Comprehensive Foundation,” Macmillan College Publishing, 1994.