藉由計算機來分析人類的行為一直是電腦科學致力的方向之一，近年來隨著計算設備與攝影設備價格的下降，以攝影機實作的非接觸式分析系統漸漸熱門起來。而計算機效能提升與電腦視覺技術的蓬勃發展，也使得原本需要大量時間的分析系統，達到可以即時分析了解人類行為的階段。當中人類臉部的特徵點的位置，能提供了解人類臉部行為的資訊，像是關注的方向，與心情等等。本論文提出一套即時系統，能在找到初始臉部資訊後，快速準確的追蹤特徵點。系統的輸入為攝影機拍攝的連續人臉影像，首先會利用人臉偵測系統找到人臉大致的位置，接著用代表特徵點的樣板比對找到的臉，為各點都建立一張位置的機率表，最後再用最佳化演算法，求得特徵點在合理形狀下各點在機率表上的機率總和最佳者，作為初始特徵點偵測結果。有了初始位置後，接著計算正面的人臉影像，藉由分析正面的人臉影像所產生的誤差，更新特徵點位置，來實現特徵點追蹤演算法。

Understanding human action is one of the most important issues of computer vision. In recent years, non-contact human action analysis system using camera has become more and more popular because the price of camera devices are getting lower. In human action analysis systems, facial feature points can tell a lot about the motion of human head such as gazing direction, drowsy and facial expression. In this thesis, a real-time system is proposed to detect the facial feature points and track these points with high accuracy and low computational cost. In the feature point detection algorithm of our system, approximately face position will be detected and compared with each feature template to create the location probability table for each point. After that, we can extract the feature points from the face by maximizing the sum of each feature point's location probability with suitable shape constraint. The feature point tracking algorithm of our system can be deliberated into three major steps. The first step is warping the face to frontal view by estimating current feature point positions. The second step is to estimate the current warping error by comparing the warped face with the trained frontal face template. The third step is updating the feature point positions by analyzing current warping error. System can track the feature points with high accuracy by iteratively updating the current feature point positions.

[1] S. Baker, R. Gross, I. Matthews, and T. Ishikawa, "Lucas-Kanade 20 Years on: A Unifying Framework: Part 2, Technical Report CMU-RI-TR-03-01," Carnegie Mellon University Robotics Institute, 2003.
[2] S. Baker and I. Matthews, "Lucas-Kanade 20 Years on: A Unifying Framework: Part 1: The Quantity Approximated, the Warp Update Rule, and the Gradient Descent Approximation," International Journal of Computer Vision, (accepted to appear in) 2004.
[3] S. Baker, I. Matthews, J. Xiao, R. Gross, T. Kanade, and T. Ishikawa, "Real-Time Non-rigid Driver Head Tracking for Driver Mental State Estimation," Proc. 11th World Congress Intelligent Transportation Systems, 2004.
[4] C.M. Bishop and N.M. Nasrabadi, Pattern Recognition and Machine Learning, Springer, 2006.
[5] D.S. Bolme, B.A. Draper, and J.R. Beveridge, "Average of Synthetic Exact Filters," Conf. Computer Vision and Pattern Recognition, 2009.
[6] R. Brunelli and T. Poggio, "Face Recognition: Features vs. Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 15, no. 10, pp. 1042-1053, 1993.
[7] P.J. Burt, "Smart Sensing within a Pyramid Vision Machine," Proceedings of the IEEE Vol. 76, pp. 1006-1015, 1988.
[8] T.F. Cootes and C.J. Taylor, "Active Shape Models," In 3rd British Machine Vision Conference, pp. 266-275, 1992.
[9] T.F. Cootes, G.J. Edwards, and C.J. Taylor, "Active Appearance Models," In 5th European Conference on Computer Vision, Vol. 2, pp. 484-498, 1998.
[10] T.F. Cootes and C.J. Taylor, "Statistical Models of Appearance for Computer Vision," University of Manchester, 2004.
[11] C. Cortes and V. Vapnik, "Support-Vector Networks," Machine Learning, 1995.
[12] D. Cristinacce and T. Cootes, "A Comparison of Shape Constrained Facial Feature Detectors," Conf. Automatic Face and Gesture Recognition, pp. 375-380, 2004.
[13] D. Cristinacce and T. Cootes, "Facial Feature Detection and Tracking with Automatic Template Selection," Conf. Automatic Face and Gesture Recognition, pp. 429-434, 2006.
[14] A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," Journal of the Royal Statistical Society, Series B (Methodological), pp. 1-38, 1977.
[15] Y. Gong and W. Xu, Machine Learning for Multimedia Content Analysis, Springer, 2007.
[16] R.C. Gonzalez and R.E. Woods, Digital Image Processing, 3rd Edition, Pearson, 2007.
[17] K. Grauman, M. Betke, J. Gips, and G.R. Bradski, "Communication via Eye Blinks - Detection and Duration Analysis in Real Time," Conf. Computer Vision and Pattern Recognition, pp. 1010-1017, 2001.
[18] R. Gross, I. Matthews, and S. Baker, "Constructing and Fitting Active Appearance Models with Occlusion," Computer Vision and Pattern Recognition Workshop, 2004.
[19] A.K. Jain, Y. Zhong, and S. Lakshmanan, "Object Matching using Deformable Templates," IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 267-278, 1996.
[20] F. Jurie and M. Dhome, "A Simple and Efficient Template Matching Algorithm," Eighth IEEE International Conference on Computer Vision. Vol. 2, 2001.
[21] S.Z. Li, Markov Random Field Modeling in Image Analysis, Springer, 2009.
[22] Z. Liu and H. Ai, "Automatic Eye State Recognition and Closed-Eye Photo Correction," International Conference on Pattern Recognition, pp. 1-4, 2008.
[23] I. Matthews and S. Baker, "Active Appearance Models Revisited," International Journal of Computer Vision, pp. 135-164, 2004.
[24] J.A. Nelder and R. Mead, "A Simplex Method for Function Minimization," Computer Journal, pp. 308-313, 1965.
[25] S.T. Roweis and L.K. Saul, "Non-linear Dimensionality Reduction by Locally Linear Embedding," Science, Vol. 290, pp. 2323-2326, 2000.
[26] C. Tu and J.J. Lien, Direct Combined Model for Facial Feature Point Detection, Face Sketch Synthesis and Occluded Face Recovery, Ph.D. Thesis, National Cheng Kung University, 2010.
[27] M. Turk and A. Pentland, "Eigenfaces for Recognition," Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp. 73-86, 1991.
[28] M. Valstar, B. Martinez, X. Binefa, and M. Pantic, "Facial Point Detection using Boosted Regression and Graph Models," Computer Vision and Pattern Recognition, pp. 2729-2736, 2010.
[29] P. Viola and M. Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features," Conf. Computer Vision and Pattern Recognition, 2001.
[30] Q. Wang, J. Yang, M. Ren, and Y. Zheng, "Driver Fatigue Detection: A Survey," Intelligent Control and Automation, pp. 8587-8591, 2010.
[31] J. Xiao, S. Baker, I. Matthews, and T. Kanade, "Real-Time Combined 2D+3D Active Appearance Models," Conf. on Computer Vision and Pattern Recognition, 2004.
[32] J. Xiao, J. Chai, and T. Kanade, "A Closed-Form Solution to Non-rigid Shape and Motion Recovery," International Journal of Computer Vision, pp. 233-246, 2006.