在Big Data的議題下，由於數據資料不斷增加或是原始資料保存不易等問題的限制，因此便無法直接從原始資料之中新增特徵或屬性的相關資訊到現有的資料集之中，進而需要透過重新收集的方式來取得一組同時包含新舊特徵的資料集，再透過此資料集來訓練一個新的預測模型來取代現有的預測模型。然而隨著特徵或屬性不斷的增加，便不斷地浪費過去所學習到的資訊；但實際上先前所學習到的模型參數也代表著那些資料所提供的資訊，因此若能夠將過去所學習到的資訊，與現階段所收到的新資訊結合再一起，便可以在一個資訊較完整的情況下進行預測。

為了能夠達到上述的目的，本論文提出了一套遞增學習演算法，透過結合各個分類器的方式來將新增的資訊與過去所學習到的資訊做結合。針對每次所收集到的資料集，為其每一個特徵空間訓練各自的預測模型，再將這些預測模型與先前所訓練的預測模型進整合。此外為了能夠達到一個較佳的預測能力，進一步透過篩選機制將新舊資訊中較差的部分剔除。

In Big Data, people could not add new features or attributes into the original dataset since the raw data ongoing increment or maybe lost due to the difficulty of preserving whole data. In order to solve this problem, the traditional method is to discard the existing classifier, and retrain the new classifier using the new collected dataset which includes the existing features and new features. However, the information it learnt before will be wasted while features and attributes increase continuously, which ignores the fact that the model learnt also represents the characteristic of the past data; therefore, if the prediction model could combine all the past information and new information together, it could make better a prediction in a more information complete situation. To achieve this goal, this thesis proposes the new incremental learning algorithm which applies the ensemble of classifiers to integrate those information. It trains a prediction model for each feature subspace when the new collected dataset is added, which then combine with the existing classifiers into a new model. Furthermore, we also use the selection mechanism to remove the worst classifiers from ensemble of classifiers to improve the efficacy of this method.

[1] R. Polikar, "Ensemble based systems in decision making," IEEE Circuits and Systems Magazine, vol. 6, pp. 21-45, 2006.
[2] G. Cauwenberghs and T. Poggio, "Incremental and decremental support vector machine learning," Advances in neural information processing systems, pp. 409-415, 2001.
[3] S. Pang, S. Ozawa and N. Kasabov, "Incremental linear discriminant analysis for classification of data streams," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 35, no. 5, pp. 905-914, 2005.
[4] F. L. Minku, H. Inoue and X. Yao, "Negative correlation in incremental learning," Natural Computing, vol. 8, pp. 289-320, 2009.
[5] R. Polikar, L. Udpa, S. S. Udpa and V. Honavar, "Learn++: an incremental learning algorithm for supervised neural networks," IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews, vol. 31, no. 4, pp. 497-508, 2001.
[6] Y. Freund and R. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," Journal of Computer and System Sciences, vol. 55, pp. 119-139, 1997.
[7] Q.-L. Zhao, Y.-H. Jiang and M. Xu, "Incremental Learning by Heterogeneous Bagging Ensemble," Advanced Data Mining and Applications, pp. 1-12, 2010.
[8] L. Breiman, "Bagging Predictors," Machine Learning , vol. 24, pp. 123-140, 1996.
[9] X.-W. Liu, G.-M. Zhang, Y.-B. Zhan and E. Zhu, "An Incremental Feature Learning Algorithm Based on Least Square Support Vector Machine," Frontiers in Algorithmics, pp. 330-338, 2008.
[10] J. Suykens and J. Vandewalle, "Least Squares Support Vector Machine Classifiers," Neural Processing Letters, vol. 9, no. 3, pp. 293-300, 1999.
[11] D. Parikh and R. Polikar, "An Ensemble-Based Incremental Learning Approach to Data Fusion," IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 37, pp. 437-450, 2007.
[12] G. Tsoumakas, I. Partalas and I. Vlahavas, "An Ensemble Pruning Primer," in Applications of Supervised and Unsupervised Ensemble Methods, 2009.
[13] Q. FU, S.-X. HU and S.-Y. Zhao, "Clustering-based selective neural network ensemble," Journal of Zhejiang University SCIENCE A, vol. 6, no. 5, pp. 387-392, 2005.
[14] Z.-H. Zhou, J.-X. Wu and W. Tang, "Ensembling neural networks: Many could be better than all," Artificial Intelligence, vol. 137, no. 1-2, pp. 239-263, 2002.
[15] Q.-L. Zhao, Y.-H. Jiang and M. Xu, "Incremental Learning Based on Ensemble Pruning," 2011 Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), vol. 1, pp. 377 - 381.
[16] G. Martinez-Munoz and A. Suarez, "Pruning in ordered bagging ensembles," pp. 1266-1273, 2006.
[17] P. Pudil, F. Ferri, J. Novovicova and J. Kittler, "Floating search methods for feature selection with nonmonotonic criterion functions," Pattern Recognition, vol. 2, pp. 279-283, 1994.
[18] L. Xu, A. Krzyzak and C. Y. Suen, "Methods of combining multiple classifiers and their applications to handwriting recognition," IEEE Transactions on Systems, Man, and Cybernetics, vol. 22, no. 3, pp. 418-435, 2002.
[19] L. I. Kuncheva, J. C. Bezdek and R. P. Duin, "Decision templates for multiple classifier fusion: an experimental comparison," Pattern Recognition, vol. 34, pp. 299-314, 2001.
[20] G. Rogova, "Combining the results of several neural network classifiers," Neural Networks, vol. 7, pp. 777-781, 1994.
[21] R. Fisher, "The Use of Multiple Measurements in Taxonomic Problems," Annals of Eugenics, pp. 179-188, 1936.
[22] G.-B. Huang, Q.-Y. Zhu and C.-K. Siew, "Extreme learning machine: Theory and applications," Neurocomputing, vol. 70, no. 1-3, pp. 489-501, 2006.
[23] C. Cortes and V. Vapnik, "Support-vector networks," Machine Learning, vol. 20, no. 3, pp. 273-297, 1995.
[24] D. Asunion, "UCI machine learning repository," 2007. [Online]. Available: http://archive.ics.uci.edu/ml/datasets.html.
[25] C.-J. Lin, "LIBSVM: A Library for Support Vector Machines," [Online]. Available: http://www.csie.ntu.edu.tw/~cjlin/libsvm/. [Accessed 2013].
[26] G.-B. Huang, "Extreme Learning Machines," [Online]. Available: http://www.ntu.edu.sg/home/egbhuang/elm_codes.html. [Accessed 2013].