簡易檢索 / 詳目顯示
| 研究生： |
王德勳 Wang, Te-Hsun |
|---|---|
| 論文名稱： |
基於三維角度估計方法或漸進式透視動作模型分離固定與非固定的表情動作 Rigid and Non-rigid Expression Motion Separation Based on 3D Pose Estimation Approach or Incremental Perspective Motion Model |
| 指導教授： |
連震杰
Lien, Jenn-Jier |
| 學位類別： |
博士 Doctor |
| 系所名稱： |
電機資訊學院 - 資訊工程學系 Department of Computer Science and Information Engineering |
| 論文出版年： | 2011 |
| 畢業學年度： | 99 |
| 語文別： | 英文 |
| 論文頁數： | 80 |
| 中文關鍵詞： | 分離固定與非固定的表情動作 、三維角度估計 、透視動作模型 |
| 外文關鍵詞： | Rigid and non-rigid motion separation, 3D pose estimation, Perspective projection, Facial expression recognition, FACS action units |
| 相關次數： | 點閱：94 下載：0 |
| 分享至: |
| 查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報 |
由於人臉表情通常包含了混合的動作，即固定的頭部轉動與非固定的表情同時發生。因此，正面的人臉分析應用必需要先作固定與非固定動作的分離。在這篇論文中將介紹兩個動作分離系統，分別是：1. 基於三維角度估計方法分離固定與非固定的表情動作，以及 2. 基於漸進式透視動作模型分離固定與非固定的表情動作。
第一個系統除了將非固定的表情動作分離出來外，也即時的估計了三維的頭部轉動角度。系統採用一個虛擬的三維人臉模型來估計盡可能正確的頭部轉動角度，如此一來將有助於正確的擷取出純粹非固定動作的特徵點軌跡來改善表情辨識的效能。藉由在搖頭轉動下消失的特徵點恢復機制的使用，該系統效能獲得了更進一步的改善。
在第二個系統中，不僅使用了漸進式透視動作模型來估計畫面之間的主要固定頭部轉動變化，也將之用來從輸入的大搖頭轉動影像中變形出正面的表情影像。為了降低在較大的搖頭轉動中所出現的三維深度變化所造成的透視投影外貌變形，該系統將人臉分成三個子區域並且使用連續畫面作正面的表情影像變形計算。此系統還提出一個稱作適合區域的方法來解決三個分離的子區域結合時所出現的邊線問題。另外，此系統使用以材質區塊為基礎的區域線性回歸方法來合成一張虛擬的正面表情影像，並用來恢復正面表情變形影像中缺失的區域使其成為一張完全的正面表情影像。
Since the facial expression usually contains mixture motions, i.e. rigid head rotation and non-rigid facial expression simultaneously. Therefore, it is necessary to separate the rigid and non-rigid motions for frontal-view facial analysis. This thesis presents two motion separation systems: 1. Rigid and non-rigid motion separation using 3D pose estimation approach and 2. Warped frontal expression creation for rigid and non-rigid motion separation using incremental perspective motion model (IPMM), respectively.
The first system separates the non-rigid facial expression from the rigid head rotation and estimates the 3D rigid head rotation angle in real time. A 3D virtual face model is used to obtain accurate estimation of the head rotation angle such that the trajectories of the feature points with pure non-rigid motion components can be precisely extracted to enhance the facial expression recognition performance. The separation performance of the first system is further improved through the use of a restoration mechanism designed to recover feature points lost during large pan rotations.
The second system uses the IPMM to not only estimate the major rigid head motion transformation between frames, but also warp the frontal expression image from large pan-rotated head inputs. To reduce the perspective appearance distortion due to 3D depth variation influence of large pan-rotated head, the facial region is divided into three sub-regions and the warping process is applied in successive frames. An adaptive region method is developed to solve the boundary problem caused by separated sub-regions combination. In addition, the second system synthesizes one virtual frontal expression image using patch-based local linear regression (LLR) and takes it as reference to recover the missing region in the warped frontal expression image.
[1] B. Abboud and F. Davoine, “Bilinear Factorization for Facial Expression Analysis and Synthesis,” IEE Proceedings Vision, Image and Signal Processing, Vol. 152, No. 3, pp. 327-333, 2005.
[2] Y. Aloimonos, “Perspective Approximations,” Image and Vision Computings, pp. 177-192, 1990.
[3] K. Anderson and P.W. McOwan, “A Real-Time Automated System for the Recognition of Human Facial Expressions,” IEEE Trans. on Systems, Man, and Cybernetics: Part B, Vol. 36, No. 1, pp. 96-105, 2006.
[4] G. Antonini, M. Sorci, M. Bierlaire and J.P. Thiran, “Discrete Choice Models for Static Facial Expression Recognition,” Advanced Concepts for Intelligent Vision Systems, pp. 710-721, 2006.
[5] M.S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel and J. Movellan, “Fully Automatic Facial Action Recognition in Spontaneous Behavior,” International Conf. on Automatic Face and Gesture Recognition, pp. 223-230, 2006.
[6] J.J. Bazzo and M.V. Lamar, “Recognizing Facial Actions Using Gabor Wavelets with Neutral Face Average Difference,” International Conf. on Automatic Face and Gesture Recognition, pp. 505-510, 2004.
[7] J. Bergen, P. Anandan, K. Hanna and R. Hingorani, “Hierarchical Model-based Motion Estimation,” European Conference on Computer Vision, pp. 237-252, 1992.
[8] M. Black and Y. Yacoob, “Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion,” International Journal on Computer Vision, pp. 23-48, 1997.
[9] B. Braathen, M.S. Bartlett, G. Littlewort, E. Smith and J.R. Movellan, “An Approach to Automatic Recognition of Spontaneous Facial Actions,” International Conf. on Automatic Face and Gesture Recognition, pp. 345-350, 2002.
[10] X. Chai, S. Shan, X. Chen and W. Gao, “Locally Linear Regression for Pose-Invariant Face Recognition,” IEEE Transactions on Image Processing, Vol. 16, No. 7, pp. 1716-1725, 2007.
[11] H.Y. Chen, C.L. Huang and C.M. Fu, “Hybrid-boost Learning for Multi-pose Face Detection and Facial Expression Recognition,” Pattern Recognition, Vol. 41, No. 3, pp. 1173-1185, 2008.
[12] C.S. Chen, C.K. Yu and Y.P. Hung, “New Calibration-Free Approach for Augmented Reality Based on Parameterized Cuboid Structure,” IEEE International Conf. on Computer Vision, pp. 30-37, 1999.
[13] K. Choi, M. Carcassoni and E. Hancock, “Recovering Facial Pose with the EM Algorithm,” Pattern Recognition, Vol. 35, No. 10, pp. 2073-2093, 2002.
[14] C.F. Chuang and F.Y. Shih, “Recognizing Facial Action Units Using Independent Component Analysis and Support Vector Machine,” Pattern Recognition, Vol. 39, No. 9, pp. 1795-1798, 2006.
[15] I. Cohen, N. Sebe, F.G. Gozman, M.C. Cirelo and T.S. Huang, “Learning Bayesian Network Classifiers for Facial Expression Recognition Using both Labeled and Unlabeled Data,” IEEE Conf. on Computer Vision and Pattern Recognition, Vol. 1, pp. 595-601, 2003.
[16] D. Datcu and L.J.M. Rothkrantz, “Facial Expression Recognition with Relevance Vector Machines,” IEEE International Conf. on Multimedia and Expo, pp. 193-196, 2005.
[17] P. Ekman and W.V. Friesen, “Facial Action Coding System,” Consulting Psychologist Press Inc., San Francisco, CA, 1978.
[18] I. Essa, T. Darrell and A. Pentland, “Tracking Facial Motion,” IEEE Workshop on Non-Rigid and Articulate Motion, pp. 36-42, 1994.
[19] I. Essa, “Analysis, Interpretation and Synthesis of Facial Expressions,” MIT Media Lab., Technical Report #303, 1995.
[20] T. Ezzat and T. Poggio, “Facial Analysis and Synthesis Using Image-Based Models,” International Conference on Automatic Face and Gesture Recognition, pp. 116-121, 1996.
[21] Y. Gao, M.K.H. Leung, S.C. Hui and M.W. Tananda, “Facial Expression Recognition From Line-Based Caricatures,” IEEE Trans. on Systems, Man, and Cybernetics: Part A, Vol. 33, No. 3, pp. 407-412, 2003.
[22] Y. Gizatdinova and V. Surakka, “Feature-Based Detection of Facial Landmarks from Neutral and Expressive Facial Images,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 28, No. 1, pp. 135-139, 2006.
[23] S.B. Gokturk, J.Y. Bouguet, C. Tomasi and B. Girod, “Model-Based Face Tracking for View-Independent Facial Expression Recognition,” International Conf. on Automatic Face and Gesture Recognition, pp. 272-278, 2002.
[24] S. Goldenstein, C. Vogler and D. Metaxas, “3D Facial Tracking from Corrupted Movie Sequences,” IEEE Conf. on Computer Vision and Pattern Recognition, Vol. 1, pp. 880-885, 2004.
[25] B. Guenter, C. Grimm, D. Wood, H. Malvar and F. Pighin, “Making Faces,” ACM SIGGRAPH, pp. 55-66, 1998.
[26] G. Guo and C.R. Dyer, “Learning from Examples in the Small Sample Case: Face Expression Recognition,” IEEE Trans. on Systems, Man, and Cybernetics: Part B, Vol. 35, No. 3, pp. 477-488, 2005.
[27] T. Horprasert, Y. Yacoob and L.S. Davis, “Computing 3-D Head Orientation from a Monocular Image Sequence,” IEEE Conf. on Automatic Face and Gesture Recognition, pp. 242-247, 1996.
[28] W. Hua, “Building Facial Expression Analysis System,” Carnegie Mellon University Robotics Institute Technical Reports, 1998.
[29] R. Klette, K. Schluns and A. Koschan, “Computer Vision: Three-Dimensional Data from Images,” Springer-Verlag, 1998.
[30] I. Kotsia, S. Zafeiriou and I. Pitas, “Texture and Shape Information Fusion for Facial Expression and Facial Action Unit Recognition,” Pattern Recognition, Vol. 41, No. 3, pp. 833-851, 2008.
[31] M.W. Lee and S. Ranganath, “Pose-Invariant Face Recognition Using a 3D Deformable Model,” Pattern Recognition, Vol. 36, No. 8, pp. 1835-1846, 2003.
[32] J.J. Lien, T. Kanade, C.C. Li and J.F. Cohn, “Detection, Tracking, and Classification of Action Units in Facial Expression,” Robotics and Autonomous Systems, Special Issue: Face Expression in Human-Robot Interaction, Vol. 31, pp. 131-146, 2000.
[33] Z. Liu, Z. Zhang, C. Jacobs and M. Cohen, “Rapid Modeling of Animated Faces from Video,” Microsoft Research, Technical Report, MSR-TR-2000-11, 2000.
[34] B. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application in Stereo Vision,” International Joint Conf. on Artificial Intelligence, pp. 674-679, 1981.
[35] S. Lucey, I. Matthews, C. Hu, Z. Ambadar, F. de la Torre and J. Cohn, “AAM Derived Face Representations for Robust Facial Action Recognition,” International Conf. on Automatic Face and Gesture Recognition, pp.155-160, 2006.
[36] L. Ma and K. Khorasani, “Facial Expression Recognition Using Constructive Feedforward Neural Networks,” IEEE Trans. on Systems, Man, and Cybernetics: Part B, Vol. 34, No. 3, pp. 1588-1595, 2004.
[37] S. Malassiotos and M.G. Strintzis, “Facial Expression Recognition Based on Shape and Texture,” Pattern Recognition, Vol. 42, No. 5, pp. 1003 -1011, 2009.
[38] S. Malassiotos and M.G. Strintzis, “Robust Real-time 3D Head Pose Estimation from Range Data,” Pattern Recognition, Vol. 38, No. 8, pp. 1153-1165, 2005.
[39] T.K. Marks, J. Hershey, J.C. Roddey and J.R. Movellan, “Joint Tracking of Pose, Expression, and Texture using Conditionally Gaussian Filters,” Advances in Neural Information Processing Systems, Vol. 17, pp. 889-896, 2005.
[40] A.M. Martinez, “Recognizing Expression Variant Faces from A Single Sample Image per Class,” IEEE Conf. on Computer Vision and Pattern Recognition, Vol. 1, pp. 353-358, 2003.
[41] M. Pantic and I. Patras, “Dynamics of Facial Expression: Recognition of Facial Actions and their Temporal Segments From Face Profile Image Sequences,” IEEE Trans. on Systems, Man, and Cybernetics: Part B, Vol. 36, No. 2, pp. 433-449, 2006.
[42] N. Rose, “Facial Expression Classification Using Gabor and Log-Gabor Filters,” International Conf. on Automatic Face and Gesture Recognition, pp. 346-350, 2006.
[43] M. Rosenblum, Y. Yacoob and L.S. Davis, “Human Emotion Recognition from Motion Using a Radial Basis Function Network Architecture,” Uni. of Maryland, CS-TR-3304, 1994.
[44] Y. Saatci and C. Town, “Cascaded Classification of Gender and Facial Expression Using Active Appearance Models,” International Conf. on Automatic Face and Gesture Recognition, pp. 393-398, 2006.
[45] N. Sebe, M.S. Lew, I. Cohen, Y. Sun, T. Gevers and T.S. Huang, “Authentic Facial Expression Analysis,” International Conf. on Automatic Face and Gesture Recognition, pp. 517-522, 2004.
[46] Y. Shinohara and N. Otsu, “Facial Expression Recognition Using Fisher Weight Maps,” International Conf. on Automatic Face and Gesture Recognition, pp. 499-504, 2004.
[47] R. Szeliski and H.Y. Shum, “Creating Full View Panoramic Image Mosaics and Environment Maps,” ACM SIGGRAPH, pp. 251-258, 1997.
[48] D. Terzopoulos and K. Waters, “Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 15, No. 6, pp. 569-579, 1993.
[49] Y. Tian, T. Kanade and J.F. Cohn, “Recognizing Action Units for Facial Expression Analysis,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No. 2, pp. 97-115, 2002.
[50] Y. Tian, T. Kanade and J.F. Cohn, “Evaluation of Gabor-Wavelet-Based Facial Action Unit Recognition in Image Sequences of Increasing Complexity,” International Conf. on Automatic Face and Gesture Recognition, pp. 218-223, 2002.
[51] F. De la Torre, Y. Yacoob and L.S. Davis, “A Probabilistic Framework for Rigid and Non-rigid Appearance Based Tracking and Recognition,” International Conf. on Automatic Face and Gesture Recognition, pp. 491-498, 2000.
[52] C.T. Tu and J.J. Lien, “Automatic Location of Facial Feature Points and Synthesis of Facial Sketches Using Direct Combined Model,” IEEE Trans. on Systems, Man, and Cybernetics: Part B, Vol. 40, No. 4, pp.1158-1169, 2010.
[53] L. Vacchetti, V. Lepetit and P. Fua, “Fusing Online and Offline Information for Stable 3D Tracking in Real-Time,” IEEE Conf. on Computer Vision and Pattern Recognition, Vol. 2, pp. 241-248, 2003.
[54] L.Y. Wei, “Texture Synthesis by Fixed Neighborhood Searching,” PhD thesis, Stanford University, 2002.
[55] L.Y. Wei, “Texture Synthesis from Multiple Sources,” ACM SIGGRAPH, pp. 1-1, 2003.
[56] J. Whitehill and C.W. Omlin, “Local Versus Global Segmentation for Facial Expression Recognition,” International Conf. on Automatic Face and Gesture Recognition, pp. 357-362, 2006.
[57] C.S. Wiles, A. Maki and N. Matsuda, “Hyperpatches for 3D Model Acquisition and Tracking,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No. 12, pp. 1391-1403, 2001.
[58] J.Y. Wu and J.J. Lien, “3D Face Reconstruction under Missing Correspondence: A Factorization Approach,” IPPR Conf. on Computer Vision, Graphics and Image Processing, 2006.
[59] Z. Xue, S.Z. Li and E.K. Teoh, “Bayesian Shape Model for Facial Feature Extraction and Recognition,” Pattern Recognition, Vol. 36, No. 12, pp. 2819-2833, 2003.
[60] Y. Yacoob and L.S. Davis, “Recognizing Human Facial Expressions from Long Image Sequence Using Optical Flow,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 18, No. 6, pp. 636-642, 1996.
[61] J. Yao and W.K. Cham, “Efficient Model-Based Linear Head Motion Recovery from Movies,” IEEE Conf. on Computer Vision and Pattern Recognition, Vol. 2, pp. 414-421, 2004.
[62] L. Yin, A. Basu, S. Bernögger and A. Pinz, “Synthesizing Realistic Facial Animations Using Energy Minimization for Model-Based Coding,” Pattern Recognition, Vol. 34, No. 11, pp. 2201-2213, 2001.
[63] L. Yin and X. Wei, “Multi-Scale Primal Feature Based Facial Expression Modeling and Identification,” International Conf. on Automatic Face and Gesture Recognition, pp. 603-608, 2006.
[64] L. Zalewski and S. Gong, “Synthesis and Recognition of Facial Expressions in Virtual 3D Views,” International Conf. on Automatic Face and Gesture Recognition, pp. 493-498, 2004.
[65] Y. Zhang and Q. Ji, “Active and Dynamic Information Fusion for Facial Expression Understanding from Image Sequences,” IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 27, No. 5, pp. 699-714, 2005.
[66] Z. Zhu and Q. Ji, “Robust Real-Time Pose and Facial Expression Recognition,” IEEE Conf. on Computer Vision and Pattern Recognition, Vol. 2, pp. 681-688, 2006.
校內：2021-12-31公開校外：不公開