就整個表情辨識系統來說，目前的研究大部分都假設人臉大小是固定的，或者背景需要是單純的顏色，不然就是假設人臉已找到，或者臉部的特徵是用手動來擷取，很少有完整的一套是從影像進來，就開始偵測人臉，然後擷取特徵來辨識表情的方法。本論文則展現一套自動化的臉部表情辨識系統，提供了一套完整系統性的方法；在不限定背景顏色是否單純，且不限定影像中人臉大小的情況下，從影像進來就可自動化的正確找出人臉位置，然後擷取臉部特徵進行表情辨識。故研究內容主要包括人臉偵測(face detection)、特徵擷取(feature extraction)以及表情辨識(facial expression recognition)三個部分。
在人臉偵測的部分，本研究採用進化演繹法(Evolutionary Computation)，利用皮膚顏色(skin color)及橢圓形狀(ellipse shape)來正確的偵測出人臉的位置，並準確的圈出人臉。在特徵的擷取的部分，本研究根據人臉偵測的結果，利用相對的位置來擷取特徵，包含嘴巴、眉毛以及眼睛，然後再進一步自動的取出這些部分的特徵點，作為判斷表情的依據。在表情辨識的部分，本研究採用倒傳遞類神經網路(back- propagation neural network)，將前步驟找出的特徵點分析處理後，判斷是屬於何種表情；而主要辨識的表情是高興、生氣、驚訝及面無表情。
本研究使用著名的JAFFE (Japanese Female Facial Expression) 表情影像資料庫及自拍影像資料庫，分別就單人、多人影像來測試表情辨識的正確率。最後的實驗結果證實本論文所提出的方法可得不錯的辨識率。

Most facial expression recognition systems restrict that the color of background must be simple, the face in an image have been identified and localized, or the facial feature must be manually extracted. Differing from previous systems, we propose a fully automatic facial expression analysis system that automatically detects faces with different sizes in complex background, extracts fourteen feature points, and recognizes four kinds of facial expressions – happy, angry, surprised and neutral.
Three parts in the system are developed to extract facial expression information for the fully automatic recognition. The first part is face detection with evolutionary computation, which can find the best-fit ellipse to cover the face contour and include the greater area of skin color. The second part is facial feature extraction using the preceding ellipse information to get fourteen feature points, two points in each eyebrow, three points in each eye, and four points in the mouth. The third part is facial expression recognition with the back-propagation neural network to recognize four motions (i.e. happiness, anger, surprise and neutral).
Our proposed system has been tested on the famous JAFFE (Japanese Female Facial Expression) database and the self-shoot database. The experimental results with the overall recognition accuracy of 86% for the JAFFE database and 74% for the self -shoot database demonstrate the effectiveness of the proposed system.

[1] 連國珍著， ” 數位影像處理 “，儒林圖書出版社，台北市，6 -1~6 – 9，2001年8月二版三刷。

[2] C.Garcia,G.tzirita, “Face detection using quantized skin color region merging and wavelet packet analysis “, IEEE trans . Multimedia , 1(3):264-277 ,1999

[3] C.Garcia,G.simandiris,G.tzirita ,“A feature-based face detector using wavelet frames”, Proc of Intern , workshop on very low bit coding” , pp . 71-76 ,Athens, October 2001

[4] A.Pentland,B.Moghaddam,T.Starner , ” View-based and modular eigenspaces for face recognition ” , Proc. IEEE Conf. Computer Vision and Pattern Recognition , pp. 84-91,1994

[5] E.Osuna,R.Freund,F.Girosi, “ Training support vector machines : an application to face detection “ , Proc. IEEE Conf. Computer Vision and Pattern Recognition , pp. 130-136,1997

[6] R.Brunelli and T.Poggio , “ Face recognition : features vs.template” , IEEE Trans. Pattern Analysis and Machine Intelligence, vol.15 ,no. 10,pp. 1042-1052,Oct.1993

[7] Quan YUAN, Wen GAO, Hongxun YAO,” Robust frontal face detection in complex environment “, Pattern Recognition, 2002. Proceedings. 16th International Conference on , Volume: 1 , 11-15 , pp. 25-28 , Aug. 2002

[8] Christophe Garcia,Manolis Delakis “A neural architecture for fast and robust face detection “，Pattern Recognition, 2002. Proceedings. 16th International Conference on , Volume: 2 , 11-15 , pp. 44-47 ,Aug. 2002

[9] H.A. Rowley, S. Baluja, T.Kanade, “ Neural network-based face detection “, IEEE Trans. Pattern Analysis and Machine Intelligence, vol.20 ,no.1 ,pp. 23-38,Jan.1998

[10] H.A Rowley, S.Baluja, T.Kanade, “ Rotation Invariant neural network-based face detection “, Proc. IEEE Conf. Computer Vision and Pattern Recognition , pp. 38-44,1998

[11] Y. Mitsukura, M. Fukumi, N. Akamatsu , “ A design of face detection system using evolutionary computation “, TENCON 2000. Proceedings , Volume: 2 , 24-27 Sept. 2000

[12] H.Yokoo , M.Hagiwara ,” Human faces detection method using genetic algorithm “ ,The Journal of The Institute of Electrical Engineers of Japan, Vol.117 ,no.9 ,pp.1245-1552, 1997

[13] Y. Mitsukura, M. Fukumi, N. Akamatsu, ” Fast face detection system using the GA-based threshold method “ , Seventh Australian and New Zealand Intelligent Information System Conference ,18-21 ,Nov ,2001

[14] R.Feraud, O.J.Bernier ,J.E.Viallet, M.collobert, “ A fast and Accurate face detection based on neural network” , IEEE Trans. Pattern Analysis and Machine Intelligence, vol.23 , no.1 , pp.42-53, Jan, 2001

[15] H.Wu , Q.Chen, M.Yachida ,” Face detection from color using a fuzzy pattern match method “ , IEEE Trans. Pattern Analysis and Machine Intelligence, vol.21 , no.6 , pp.557-563, June, 1999

[16] S,H,Jeng,H.Y.M.Yao,C.C.Han,M.Y.Chern and Y.T.Liu “Facial feature detection using geometrical face model : An efficient approach” Pattern Recognition,31(3):273-282,1993

[17] M.Pantic, M.Tomc ,Leon J.M. Rothkrantz, “A Hybrid Approach To mouth features detection” , Systems, Man, and Cybernetics, 2001 IEEE International Conference on , Volume: 2 , 7-10 Oct. 2001
[18] M. Bartlett, J. Hager, P.Ekman, and T. Sejnowski. “ Measuring facial expressions by computer image analysis” . Psychophysiology , 36:253–264, 1999.

[19] M. J. Black and Y. Yacoob.” Recognizing facial expressions in image sequences using local parameterized models of image motion”. International Journal of Computer Vision, 25(1):23–48, October 1997.

[20] G. Donato, M. S. Bartlett, J. C. Hager, P. Ekman, and T. J. Sejnowski. “ Classifying facial actions”. IEEE Transaction on Pattern Analysis and Machine Intelligence, 21(10) : 974–989 ,October 1999.

[21] I. A. Essa and A. P. Pentland. ” Coding, analysis, interpretation, and recognition of facial expressions” . IEEE Transc. On Pattern Analysis and Machine Intelligence, 19(7):757–763, JULY 1997.

[22] K. Fukui and O. Yamaguchi. “ Facial feature point extraction method based on combination of shape extraction and pattern matching” . Systems and Computers in Japan, 29(6):49–58, 1998.

[23] M. Lyons, S. Akamasku, M. Kamachi, and J. Gyoba. “ Coding facial expressions with gabor wavelets” . In Proceedings of International Conference on Face and Gesture Recognition,1998

[24] Y. Tian, T. Kanade, and J. Cohn. “ Eye-state action unit detection by gabor wavelets” . In Proceedings of International Conference on Multi-modal Interfaces (ICMI 2000), pages 143–150, Sept, 2000.

[25] Z. Zhang, M. Lyons, M. Schuster, and S. Akamatsu. “ Comparison between geometry-based and gabor-wavelets-based facial expression recognition using multi-layer perceptron” . In International Workshop on Automatic Face and Gesture Recognition, pages 454–459, 1998.

[26] Ying-li Tian Takeo Kanade and Jeffrey F. Cohn,” Evaluation of Gabor-Wavelet-Based Facial Action Unit Recognition in Image Sequences of Increasing Complexity” , Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition,2002

[27] P. Ekman and W.V. Friesen, “ The facial action coding system: a technique for the measurement of facial movement ”. San Francisco: Consulting Psychologists Press, 1978.

[28] Y. Tian, T. Kanade, and J. Cohn. “Recognizing action units for facial expression analysis”. IEEE Transaction on Pattern Analysis and Machine Intelligence, 23(2):1–19, February 2001.

[29] P. Ekman, “ Facial expression and emotion,”Am. Psychologist, vol. 48, pp. 384-392, 1993.

[30] T. Kanade, J. Cohn, and Y. Tian, “Comprehensive database for facial expression analysis”, Proc. Int'l Conf. Face and Gesture Recognition, pp. 46-53, Mar. 2000.

[31] J.-J.J. Lien, T. Kanade, J.F. Cohn, and C.C. Li, “Detection, tracking, and classification of action units in facial expression”, J. Robotics and Autonomous System, vol. 31, pp. 131-146, 2000.

[32] K. Mase, “Recognition of facial expression from optical flow”,
IEICE Trans., vol. E74, no. 10, pp. 3474-3483, Oct. 1991.

[33] Z. Zhang,“Feature-based facial expression recognition: sensitivity analysis and experiments with a multilayer perceptron”, Int'l J. Pattern Recognition and Artificial Intelligence, vol. 13, no. 6, pp. 893-911, 1999.

[34] M. Rosenblum, Y. Yacoob, and L.S. Davis, “Human expression recognition from motion using a radial basis function network archtecture”, IEEE Trans. Neural Network, vol. 7, no. 5, pp. 1121-1138, 1996.