在本系統中提出了一個利用因式分解法，分解分段式正投影模型來逼近透視投影模型的三維人臉重建方法。本系統包含了五大模組：第一個跟第二個模組利用因式分解法分解正投影模型來重建一個初步的三維人臉模型。但是因為當人臉影像在被擷取的時候是使用透視投影模型而不是正投影模型，所以這個初步的三維人臉模型跟實際的人臉會有一定的誤差。因此，第三個模組就將初步的三維人臉模型分割成很多小群組，每個小群組都利用因式分解法搭配正投影模型的方式去重建出較為精確的部份人臉，再將這些不同群組的部份人臉合併在一起成為一個完整的三維人臉模型，而此三維人臉模型就可以很接近使用透視投影模型底下重建出來的三維人臉模型。而因為本系統使用方法的一些限制，造成無法重建出一個由很密集的點形成的三維人臉模型，所以第四個模組就提出了一個方法，利用第三個模組重建出來的三維人臉模型來產生一個由較密集點形成的三維人臉模型，使外觀看起來較為平順。最後，第五個模組利用前幾個模組的特性，來克服一個在三維人臉重建中一個常見的問題：遮蔽問題。這個問題會造成一些部位的特徵點看不見，導致傳統的重建方式失敗。在實驗結果的部份，我們可以發先我們的方法可以在相對較短的時間內得到一個可靠的三維人臉模型。

This study develops a 3D facial reconstruction system in which the perspective projection model is approximated by applying a factorization method to the piecewise orthographic projection model. The proposed system comprises five modules. The first and second modules reconstruct the 3D facial surface using a factorization method based on an orthographic projection model. However, the facial video is taken based on the perspective projection model rather than an orthographic projection model. Thus, to compensate for the difference between the two models, the third module is developed to approximate the perspective projection model by dividing the 3D facial surface into small groups and then reconstructs each group in orthographic projection module. These reconstructed results are then integrated to form a complete 3D facial surface, which is almost as accurate as the reconstruction result using perspective projection model. The fourth module implements a novel smoothing process for the 3D facial surface by interpolating additional vertices from the vectors of the existing 3D vertices. Finally, the fifth module utilizes a new solution to overcome the missing point problem, which is caused by occlusion at high pan rotation angles, commonly arising in 3D reconstruction applications. The experimental results show that the proposed system achieves a promising result within a relatively short time.

[1] A. Ahmed, A. Farag, T. Starr, A new symmetric shape from shading algorithm with an application to 3-D face reconstruction, International Conf. on Image Processing, pp.201-204, 2008.
[2] G.A. Atkinson, E.R. Hancock, Shape Estimation Using Polarization and Shading from Two Views, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.29, no.11, pp.2001-2017, 2007.
[3] V. Blanz and T. Vetter, A Morphable Model for the Synthesis of 3D Faces, Proc. of the 26th annual conference on Computer graphics and interactive techniques, pp.187–194, 1999.
[4] M. Brand, Morphable 3D models from video, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.2, pp.456-463, 2001.
[5] M. Brand, A Direct Method for 3D Factorization of Nonrigid Motion Observed in 2D, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.2. pp.122-128, 2005.
[6] A. Braquelaire and B. Kerautret, Reconstruction of Lambertian Surfaces by Discrete Equal Height Contours and Regions Propagation, Image and Vision Computing, vol.23, no.2 pp.177-189, 2004.
[7] C. Bregler, A. Hertzmann, and H. Biermann, Recovering Non-Rigid 3D Shape from Iimage Streams, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.2, pp.690–696, 2000
[8] E. Coiras, Y. Petillot and D. M. Lane, Multiresolution 3-D Reconstruction From Side-Scan Sonar Images, IEEE Transactions on Image Processing, vol.16, no.2, pp.382-390, 2007.
[9] T. F. Cootes, G. J. Edwards, and C. J. Taylor, Active appearance models, European Conference on Computer Vision, vol.2, pp.484–498, 1998.
[10] T.F.Cootes, A.Hill, C.J.Taylor, and J.Haslam, Use of active shape models for locating structures in medical images, Image and Vision Computing, vol.12, no.6, pp.355-365, 1994.
[11] T. F. Cootes and C. J. Taylor, Constrained active appearance models, International Conference on Computer Vision, pp.748-754, 2001.
[12] T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham, Active shape models-their training and application, Computer vision and image understanding, vol.61, no.1, pp.38-59, 1995.
[13] J. Costeira and T. Kanade, A Multi-body Factorization Method for Independently Moving Objects, International Journal of Computer Vision, vol.29, no.3, pp.159-179, 1998.
[14] F. Courteille, A. Crouzila, J. D. Duroua and P. Gurdjos, 3D-Spline Reconstruction Using Shape from Shading: Spline from Shading, Image and Vision Computing, vol.26, no.4 pp.466-479, 2008.
[15] D. Cristinacce and T. F. Cootes, A comparison of shape constrained facial feature detectors, International Conference on Automatic Face and Gesture Recognition, pp.375-380, 2004
[16] J. W. Demmel, Applied Numerical Linear Algebra, Society of Industrial and Applied Mathematics, Philadelphia, PA, USA, pp.114, 1997.
[17] R.A. Dwyer, A Faster Divide-and-Conquer Algorithm for Constructing Delaunay Triangulations, Algorithmica, pp.2:137-151, 1987.
[18] W. H. Equitz, A new vector quantization clustering algorithm, IEEE Transactions on Acoustics, Speech and Signal Processing, vol.37, no.10 pp.1568–1575, 1989.
[19] A. Fitzgibbon and A. Zisserman, Multibody structure and motion: 3-D reconstruction of independently moving objects, European Conference on Computer Vision, pp.891-906, 2000.
[20] P. Fua, Regularized Bundle-Adjustment to Model Heads from Image Sequences without Calibration Data, International Journal of Computer Vision, vol.38, no.2, pp.153-171, 2000.
[21] P. Gargallo and P. Sturm, Bayesian 3D Modeling from Images using Multiple Depth Maps, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.2, pp.885-891, 2005.
[22] A.Ghosh, S. Achutha, W. Heidrich and M. O'Toole, BRDF Acquisition with Basis Illumination, Proc. of IEEE International Conf. on Computer Vision, pp.1-8, 2007.
[23] A. Gruber and Y. Weiss, Multibody Factorization with Uncertainty and Missing Data Using the EM Algorithm, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.1, pp.707–714, 2004.
[24] M. Han, and T. Kanade, Reconstruction of a Scene with Multiple Linearly Moving Objects, International Journal of Computer Vision, vol.59, no.3, pp.285-300, 2004.
[25] R. Hartley and A. Zisserman, Multiple View Geometry in computer vision, Cambridge University Press, pp.144-158, 2000.
[26] R. Hassanpour and V. Atalay, Delaunay Triangulation Based 3D Human Face Modeling from Uncalibrated Images, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition Workshop on Face Processing in Video, vol.5, pp.75, 2004.
[27] E. Hecht, Optics, 4th ed., Addison Wesley, pp.150-171, 2003.
[28] A. Hertzmann and S. M. Seitz, Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.27, no.8 pp.1254–1264, 2005.
[29] T. Horprasert, Y. Yacoob and L. S. Davis, Computing 3-D Head Orientation from a Monocular Image Sequence, IEEE International Conf. on Automatic Face and Gesture Recognition, pp.242-247, 1996.
[30] X. Huang, J. Gao, L. Wang, R. Yang, Examplar-based Shape from Shading, International Conf. on 3-D Digital Imaging and Modeling, pp.349-356, 2007.
[31] N. Ichimura, Motion Segmentation Based on Factorization Method and Discriminant Criterion, Proc. of IEEE International Conf. on Computer Vision, vol.1, pp.600-605, 1999.
[32] H. Jin, S. Soatto, and A. Yezzi Multi-view Stereo Beyond Lambert, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.1, pp.171-178, 2003.
[33] Y. C. Lee, D. Terzopoulos and K. Waters, Constructing Physics-Based Facial Models of Individuals, Graphics Interface, pp.1-8, 1993.
[34] Z. Lei, Q. Bai, R. He, S.Z. Li, Face Shape Recovery from a Single Image Using CCA Mapping between Tensor Spaces, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp.1-7, 2008.
[35] Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen, Rapid Modeling of Animated Faces From Video, Microsoft Research, Technical Report, MSR-TR-2000-11, 2000.
[36] B. D. Lucas and T. Kanade, An Iterative Image Registration Technique with an Application to Stereo Vision, Proc. of DARPA Image Understanding, pp.121-130, 1981.
[37] J. Maciel and J. Costeria, Maximizing Rigidity: Optimal Matching under Scaled-Orthography, European Conference on Computer Vision, pp. 232-246, 2002.
[38] M. T. Mahmood, A. Khan and T. S. Choi, Approximating 3D Shape through Bezier Curve and Moments in Discrete Cosine Transform, International Journal of Innovative Computing, Information and Control, vol.5, no.10(A), pp.2947-2958, 2009.
[39] M. T. Mahmood, S. Shim and T. S. Choi, Shape from Focus Using Principal Component Analysis in Discrete Wavelet Transform, Optical Engineering, vol.48, no.5, pp.057203:1-057203:9, 2009
[40] I. Matthews and S. Baker, Active Appearance Models Revisited, International Journal of Computer Vision, vol. 60, no. 2, pp. 135 – 164, 2004.
[41] E. Mouragnon, F. Dekeyser, P. Sayd, M. Lhuillier and M. Dhome, Real Time Localization and 3D Reconstruction, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp.363- 370, 2006.
[42] E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, 3D Reconstruction of Complex Structures with Bundle Adjustment: an Incremental Approach, Proc. of IEEE Conf. on Robotics and Automation, pp.3055-3061, 2006.
[43] M. S. Muhammad and T. S. Choi, A Novel method for Shape from Focus in Microscopy using Bezier Surface approximation, Microscopy Research and Technique, vol.73, no.2, pp.140-151, 2010.
[44] J. L. Mundy and A. Zisserman, Geometric Invariance in Computer Vision, MIT Press, pp.512, 1992.
[45] S. Nayar and N. Yasuo, Shape From Focus, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.16, no.8, pp.824-831, 1994.
[46] J. A. Nelder and R. Mead. A simplex method for function minimization. Computer Journal, vol.7, pp.308–313, 1965.
[47] F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. H. Salesin, “Synthesizing Realistic Facial Expressions from Photographs,” ACM Transactions on Graphics (ACM SIGGRAPH), pp. 75–84, 1998.
[48] C. Poelman and T. Kanade, A Paraperspective Factorization Method for Shape and Motion Recovery, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.19, no.3, pp.206-218, 1997.
[49] Y. Shan, Z. Liu and Z. Zhang, Model-Based Bundle Adjustment with Application to Face Modeling, Proc. of IEEE International Conf. on Computer Vision, pp.644-751, 2001.
[50] H. Shum, Q. Ke and Z. Zhang, Efficient Bundle Adjustment with Virtual Key Frames: A Hierarchical Approach to Multi-Frame Structure from Motion, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.2, pp.538-543, 1999.
[51] T. Sim, S. Baker and M. Bsat, The cmu pose, illumination, and expression (PIE) database of human faces. Technical Report CMU-RI-TR-01-02, The Robotics Institute, Carnegie Mellon University, 2001.
[52] N. Snavely, S. Seitz, and R. Szeliski, Modeling the World from Internet Photo Collections, International Journal of Computer Vision, vol.80, no.2, pp.180-210, 2007.
[53] S. Soatto, A. Yezzi, H. Jin, Tales of Shape and Radiance in Multiview Stereo, Proc. of IEEE International Conf. on Computer Vision, pp.974-981, 2003.
[54] W. K. Tang and Y. S. Hung, A subspace method for projective reconstruction from multiple images with missing data, Image and Vision Computing, vol.24, no.4, pp.515-524, 2006.
[55] A. Tankus, N. Sochen and Y. Yeshurun, A New Perspective [on] Shape-from-Shading, Proc. of IEEE International Conf. on Computer Vision, vol.II, pp.862–869, 2003.
[56] A. Thelen, S. Frey, S. Hirsch and P. Hering, Improvements in Shape-From-Focus for Holographic Reconstructions With Regard to Focus Operators, Neighborhood-Size, and Height Value Interpolation, IEEE Transactions on Image Processing, vol.18, no.1 pp.151-157, 2009.
[57] C. Tomasi and T. Kanade, Shape and Motion from Image Streams under Orthography: a Factorization Method, International Journal of Computer Vision, vol.9, no.2, pp.137-154, 1992.
[58] P.H.S. Torr, Bayesian model estimation and selection for epipolar geometry and generic manifold fitting, International Journal of Computer Vision, vol.50, no.1, pp.35-61, 2002.
[59] L. Torresani, D. Yan, J. Alexander, and C. Bregler: Tracking and Modeling Non-Rigid Objects with Rank Constraints, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.1, pp.493–500, 2001.
[60] A. Treuille, A. Hertzmann, and S. Seitz, Example-Based Stereo with General BRDFs. European Conference on Computer Vision, vol.2, pp.457-469, 2004.
[61] B. Triggs, Factorization Methods for Projective Structure and Motion, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp.845-851, 1996.
[62] B. Triggs, P.F. McLauchlan, R.I. Hartley, and A.W. Fitzibbon, Bundle Adjustment - A Modern Synthesis, Proc. of the International Workshop on Vision Algorithms: Theory and Practice, pp.298-372, 1999.
[63] C. T. Twu and J. J. Lien, Automatic Location of Facial Feature Points and Synthesis of Facial Sketches Using Direct Combined Model, IEEE Transactions on Systems, Man and Cybernetics—Part B: Cybernetics, vol.40, no.4, pp.1158-1169, 2010.
[64] R. Vidal, and R. Hartley, Motion Segmentation with Missing Data using PowerFactorization and GPCA, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.2, pp.310-316, 2004.
[65] P. Viola and M. Jones, Rapid object detection using a boosted cascade of simple features. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol.1, pp.511–518, 2001.
[66] G. Vogiatzis, P. H. S. Torr and R. Cipolla, Multi-view Stereo via Volumetric Graph-cuts, Proc. of EEE Conf.on Computer Vision and Pattern Recognition, vol.1, pp.391-398, 2005.
[67] S. Wang, Depth from Shading Based on 2D Maximum Entropy, International Conf. on Intelligent Computation Technology and Automation, pp. 119-121, 2008.
[68] X. Wang, X. Huang, J. Gao, and R. Yang. Illumination and personinsensitive head pose estimation using distance metric learning, European Conference on Computer Vision, pp.624–637, 2008.
[69] T. H. Wang and J. J. Lien, Rigid and Non-Rigid Motion Separation Using 3D Model, Computer Vision, Graphics, and Image Processing, pp.A2-5, 2004.
[70] G. Wang, G Sun, X. Li, and S. Wang, Modelling Nonrigid Object from Video Sequence Based on Power Factorization, International Conference on Artificial Reality and Telexistence – Workshops, pp.98-103, 2006.
[71] G. Wang and Q.M. Wu, Quasi-perspective Projection Model: Theory and Application to Structure and Motion Factorization from Uncalibrated Image Sequences, International Journal of Computer Vision, vol.87, no.3, pp.213-234, 2010.
[72] J. Xiao, J. X. Chai and T. Kanade, A Closed-Form Solution to Non-Rigid Shape and Motion Recovery, European Conference on Computer Vision, pp.573-587, 2004.
[73] J. Xiao and T. Kanade, Uncalibrated Perspective Reconstruction of Deformable Structures, Proc. of IEEE International Conf. on Computer Vision, vol.2, pp.1075-1082, 2005.
[74] J. Yan and M. Pollefeys, A Factorization-Based Approach for Articulated Nonrigid Shape, Motion, and Kinematic Chain Recovery from Video, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.30, no.5, pp.865-877, 2008.
[75] L. You and J. J. Zhang, Fast Generation of 3-D Deformable Moving Surfaces, IEEE Trans. on IEEE Trans. on Systems, Man and Cybernetics—Part B: Cybernetics, vol.33, no.4, pp.616-625, 2003.
[76] Y.K. Yu, K.H. Wong, and M.M.Y. Chang, Recursive Three-Dimensional Model Reconstruction Based on Kalman Filter, IEEE Trans. on Systems, Man and Cybernetics—Part B: Cybernetics, vol.35, no.3, pp.587-592, 2005.
[77] Z. Zhang, and Y. Shan, Incremental Motion Estimation through Modified Bundle Adjustment, International Conf. on Image Processing, vol. 3, pp.343-346, 2003
[78] R. Zhang, P. S. Tsai, J. E. Cryer and M. Shah, Shape from Shading: A Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.21, no.8 pp.690-705, 1999.