半監督式學習是近年來機器學習領域研究的熱點問題，在人臉辨識上的重要性也不容忽視。它主要考慮如何利用少量的標記資料和大量的未標記資料進行訓練和分類的問題。半監督學習對於減少標記代價，提高學習機器的性能具有非常重大的實際意義。本論文提出了一種半監督式學習模型，對於未標記的資料給予模型正規化（model regularization）以達到最佳辨識率。此外我們闡述了監督式支持向量機（support vector machine, SVM）和轉換式支持向量機（transductive SVM, TSVM）可以作為此模型中的特例。針對未標記資料部份，可以被看作是半監督式學習模型中，根據其可信度高低對決策邊界添加額外的正規化懲罰。本論文以GT及FERET人臉資料庫的實驗，再利用貝氏（Bayes）理論，將模型參數的事後機率更新，用事後機率的平均值向量計算出模型中兩個類別的機率分佈，對未標記資料屬那類別機率分別求出來以作為可信度，從初步實驗結果可看出本論文方法的有效性。

Semi-supervised learning is a popular issue in the areas of pattern classification and machine learning. This issue is especially crucial in the application of face recognition. Traditional classifiers are trained by only using the labeled data. However, the labeled samples are often difficult, expensive, or time consuming to be collected since a lot of efforts should be involved from experienced human annotators. Semi-supervised learning addresses this problem by using large amount of unlabeled data, together with a limited amount of labeled data, to build a good classifier. We discuss the framework of Transductive Support Vector Machine (TSVM) from the perspective of the regularization strength induced by the unlabeled data. In this framework, SVM and TSVM are regarded as a learning machine without regularization and one with full regularization from the unlabeled data, respectively. Therefore, in order to supplement this framework of the regularization strength, it is necessary to introduce data-dependant partial regularization. To this end, we reformulate TSVM into a form with controllable regularization, which includes SVM and TSVM as special cases. Furthermore, we introduce a method of adaptive regularization that is based on Bayes theory and is updated according to its posterior distribution of parameters. The experiments on GT and FERET facical data sets indicate the promising results of the proposed work.

[1] Y. Altun, and M. Belkin, "Maximum margin semi-supervised learning for structured variables.," Advances in Neural Information Processing Systems, 2005.
[2] J. L. An, Z. O. Wang, and Z. P. Ma, "An incremental learning algorithm for support vector machine," Proceedings of the Second International Conference on Machine Learning and Cybernetics, vol. 2, pp. 1153-1156, 2003.
[3] M.-F. Balcan, A. Blum, and K. Yang, "Co-training and expansion: Towards bridging theory and practice.," Advances in Neural Information Processing Systems, Cambridge, MA: MIT Press., 2005.
[4] P.N. Bellhumeur, J.P. Hespanha, and D.J. Kriegman, “Eigenfaces vs. fisherfaces: recognition using class specific linear projection”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[5] M. Belkin, I. Matveeva, and P. Niyogi, "Regularization and semi-supervised learning on large graphs," COLT: Proceedings of the Workshop on Computational Learning Theory, 2004.
[6] M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: a geometric framework for learning from examples (Technical Report TR-2004-06)." University of Chicago, 2004.
[7] K. Bennett, and A. Demiriz, "Semi-supervised support vector machines," Advances in Neural Information Processing Systems, vol. 12, 1998.
[8] C. M. Bishop, Pattern Recognition and Machine Learning, Springer, 2006.
[9] A. Blum, and T. Mitchell, "Combining labeled and unlabeled data with co-training," COLT: Proceedings of the Workshop on Computational Learning Theory, 1998.
[10] A. Blum, and S. Chawla, "Learning from labeled and unlabeled data using graph mincuts," Proceedings of the 18th International Conference on Machine Learning, 2001.
[11] O. Bousquet, O. Chapelle, and M. Hein, "Measure based regularization," Advances in Neural Information Processing Systems, vol. 16, 2004.
[12] O. Bousquet, D. Zhou, T. N. Lal, J. Weston, and B. Schlkopf, "Learning with local and global consistency," Advances in Neural Information Processing Systems, 2003.
[13] C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Mining Knowledge Discovery, vol. 2, pp. 121-167, 1998.
[14] O. Chapelle, "Training a Support Vector Machine in the Primal," Neural Computation, vol. 19, pp. 1155-1178, 2007.
[15] O. Chapelle, V. Sindhwani, and S. Sathiya Keerthi., "Branch and bound for semi-supervised support vector machines.," Advances in Neural Information Processing Systems. MIT Press, Cambridge, MA, 2007.
[16] O. Chapelle, B. Schölkopf, and A. Zien, editors., "Semi-Supervised Learning," MIT Press, Cambridge, MA, 2006.
[17] O. Chappelle, J. Weston, and B. Scholkopf, "Cluster kernels for semi-supervised learning," Neural Information Processing Systems, 2002.
[18] O. Chapelle, and A. Zien, "Semi-supervised classification by low density separation," In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, pp. 57–64, 2005.
[19] K. Chen, and S. Wang, "Regularized boost for semi-supervised learning.," Advances in Neural Information Processing Systems, vol. 20, pp. 281-288, 2008.
[20] J.T Chien, and C.C Wu, "Discriminant waveletfaces and nearest feature decisions for face recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1644–1649, 2002.
[21] C. Cortes, and V. Vapnik, "Support-vector Networks," Machine Learning, vol. 20, pp. 273-297, 1995.
[22] N. Christiani, and J. Shawe-Taylor, "Kernel Methods for Pattern Analysis," Cambridge University Press, 2004.
[23] N. Cristianini, and J. Shawe-Taylor, An Introduction to Support Vector Machines: Cambridge Univ. Press, 2000.
[24] T. Joachims, "Transductive inference for text classification using support vector machines.," In Proceedings of the 16th International Conference on Machine Learning, pp. 200-209, 1999.
[25] T. Joachims., "Transductive learning via spectral graph partitioning.," In Proceedings of the 20th International Conference on Machine Learning, pp. 290-297, 2003.
[26] S. S. Keerthi, and E. G. Gilbert, "Convergence of a generalized SMO algorithm for SVM classifier design," Machine Learning, vol. 46, pp. 351-360, 2002.
[27] K. I. Kim, K. Jung, S. H. Park, and H. J. Kim, "Support vector machines for texture classification," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 24, pp. 1542-1550, Nov. 2002.
[28] J. Lafferty, Y. Liu., and X. Zhu, "Kernel conditional random fields: Representation, clique selection, and semi-supervised learning," In Proceedings of the 22nd International Conference on Machine Learning, 2004.
[29] W.S. Lee, H.J. Lee, and J.H. Chung, "Wavelet-Based FLD for face recognition," Proc. IEEE Midwest Symp. on Circuits and Systems, Lansing MI, pp. 734-737, Aug. 2000.
[30] A.M. Martinez, and A.C. Kak, “PCA versus LDA”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 23, pp. 228-233, 2001.
[31] H. Narayanan, M. Belkin, and P. Niyogi, "On the relation between low density separation, spectral clustering and graph cuts," Advances in Neural Information Processing Systems, vol. 19, 2006.
[32] B. Schölkopf, K. Sung, C. Burges, F. Girosi, P. Niyogi, T. Pogio, and V. Vapnik, "Comparing support vector machines with Gaussian kernels to radial basis function classifiers," IEEE Trans. on Signal Processing, vol. 45, pp. 2758-2765.
[33] M. Seeger, "Learning with labeled and unlabeled data," Technical Report. University of Edinburgh, 2001.
[34] V. Sindhwani, and S. S. Keerthi, "Large Scale Semi-supervised Linear SVMs," ACM Special Interest Group on Information Retrieval, pp. 477–484, 2006.
[35] V. Sindhwani, P. Niyogi, M. Belkin, and S. Keerthi, "Linear manifold regularization for large scale semi-supervised learning," In Proceedings of the 22nd ICML Workshop on Learning with Partially Classified Training Data, 2005.
[36] V. Sindhwani, P. Niyogi, and M. Belkin, "Beyond the point cloud: from transductive to semi-supervised learning," In Proceedings of the 22nd International Conference on Machine Learning, 2005.
[37] A. J. Smola, and B. Scholkopf, Learning with Kernels, MIT Press, 2002.
[38] B. Taskar,C. Guestrin, and D. Koller, "Max-margin markov networks," Advances in Neural Information Processing Systems, 2004.
[39] N. Takahashi, and T. Nishi, "Rigorous proof of termination of SMO algorithm for support vector machines," IEEE Trans. on Neural Networks, vol. 16, pp. 774-776, May 2005.
[40] I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun, "Support vector machine learning for interdependent and structured output spaces.," In Proceedings of the 22nd International Conference on Machine Learning, pp. 823-830, 2005.
[41] V. Vapnik, "The nature of statistical learning theory," New-York: Springer-Verlag, 1995.
[42] V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
[43] L. Xu, and D. Schuurmans, "Unsupervised and semi-supervised multi-class support vector machines," In Proceedings of the Twentieth National Conference on Artificial Intelligence, 2005.
[44] Z. Xu, R. Jin, J. Zhu, I. King, and M. R. Lyu., "Efficient convex relaxation for transductive support vector machine.," Advances in Neural Information Processing Systems, vol. 20, 2008.
[45] Z. Xu, R. Jin, J. Zhu, I. King, M. R. Lyu, and Z. Yang, "Adaptive regularization for transductive support vector machine," Neural Information Processing Systems, 2009.
[46] M. H. Yang, N. Ahuja, D. Kraegman, "Face recognition using kernel eigenfaces," In Proceedings of the International Conference on Image Processing, 2000.
[47] D. Yarowsky, "Unsupervised word sense disambiguation rivaling supervised methods.," Proceedings of the 33rd Annual Meeting of the Association for Computational Linguistics, pp. 189-196, 1995.
[48] X. Zhu, Z. Ghahramani, and J. Lafferty, "Semi-supervised learning using Gaussian fields and harmonic functions," In Proceedings of the 20th International Conference on Machine Learning (ICML)., pp. 912-919, 2003
[49] X. Zhu., "Semi-supervised learning literature survey," Technical Report, Computer Sciences, University of Wisconsin-Madison, 2005.