無

In the field of machine learning, occasions of insufficient data are often encountered. Especially when time and cost are limited. Without exception, the emerging learning method SVM (support vector machine) also faces this situation. Since a small data set usually leads learning systems to a low learning accuracy, find a way to cope with the problem becoming meaningful in academics consequently.
Theoretically, so called insufficient data don’t mean the absolute number of data is small, but mainly points on the inappropriate ratios between the number of data and the associated dimension. It is clean viewing this concept through the Generalization theory. The theory fully illustrates the relationship among the learning error, the data size, and the dimensions. Therefore, to reduce the learning error by determining a proper relationship between size and dimensions is the basic approach proposed in this study. Technically, two ways are the method to increase learning accuracies including (1) Virtual Samples generation, and (2) dimension reducing. The study will derivate an algorithm for generating a set of training samples for leaning and refer the reason of dimension reducing.

Evgeniou, T., Poggio, T. Pontil, M., and Verri, A, Regularization and statistical learning theory for data analysis, Computational Statistics & Data Analysis, 38, pp. 421-431, 2002.
Neumann, J, L earning the systematic transformation of holographic reduced representations, Cognitive Systems Research, pp. 227–235, 2002.
Niyogi, P., Girosi, N., and Poggio, T., Incorporating Prior Information in Machine Learning by Creating Virtual Examples, Proceedings of the IEEE, Vol. 86,no. 11, 1998.
Platt, J., Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods – Support Vector Learning, pp. 185-208. MIT Press, 1999.
Rosenblatt, F., The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Reviews, 65, pp. 386-408, 1959.
Smola, A.J., Bcholkopf, B., and Muller, K., The connection between regularization operators and support vector kernels, Neural Networks, 11, pp. 637-649, 1998.
Tikhonov, A.N., Arsenin, V.Y., Solutions of Ill-posed Problems. Winston, W.H, Washington,D.C. , 1977.
Vapnik, V. and Chervonenkis, A.Ja., On the uniform convergence of relative frequencies of events to their probabilities, Theory Probab. Apl.16, pp. 264-280, 1971.
Vapnik, V., The Nature of Statistical Learning Theory, Springer-Verlag, NY, 1999.
Vukovich,G., Pedrycz,w., Feature analysis through information granulation and fuzzy sets. Pattern Recognition, Vol. 35, pp.825-834, 2002.
Wahba, G.., Splines Models for Observational Data. Series in Applied Mathematics, Philadelphia, Vol. 59, SIAM, 1990.