簡易檢索 / 詳目顯示

研究生: 鍾昀珊
Chung, Yun-Shan
論文名稱: 單調性模糊支援向量機模型之研究
Toward a Monotonic Fuzzy Support Vector Machines Model
指導教授: 李昇暾
Li, Sheng-Tun
學位類別: 碩士
Master
系所名稱: 管理學院 - 資訊管理研究所
Institute of Information Management
論文出版年: 2013
畢業學年度: 101
語文別: 英文
論文頁數: 56
中文關鍵詞: 單調性模糊支援向量機支援向量機模糊支援向量機單調性限制式信用評分資料探勘先驗知識
外文關鍵詞: MC-FSVM, SVM, FSVM, monotonicity constraint, credit scoring, data mining, prior knowledge
相關次數: 點閱:138下載:0
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  •   資料探勘技術能讓我們從大量資料中找出隱含的模式,進而萃取出有用的知識,其技術已被廣泛運用於分析與萃取知識信用貸款、破產預測,但是大部分資料探勘的研究皆為資料導向,但在實務運用上,可能會因為缺少企業智慧而降低決策品質,這造成學術與實務上巨大的鴻溝。而在實際生活中我們可以發現,屬性與類別存在著單調性的關係,利用這種性質,將先驗知識引入於分類模型中可以使分類正確率提高。
      支援向量機(support vector machine, SVM)是近年來資料探勘熱門的工具之一,出色的學習能力成為目前機器學習研究焦點,在處理分類問題上已被廣泛應用。由於SVM是根據訓練實例來建構分類模型,會對於較不具重要性的資料或噪點過於重視及敏感,導致分類正確率下降。模糊理論概念的引入使模糊支援向量機可提供不同資料之不同的重要性,對於決策問題較具貢獻的資料應給予較高的貢獻值。
      本研究提出知識導向於單調性來建構限制式,並利用各領域專家所提供的先備知識判斷資料中貢獻度,來建構知識導向具單調性限制式的模糊支援向量機模型。借助專家智慧,擷取資料集的單調性規則,接著進行資料前處理,並針對每筆資料給予不同的貢獻度,最後使用本研究提出的MC-FSVM模型來完成學習任務。經實驗證實,以具有單調性限制式及不同貢獻度的MC-FSVM模型在分類結果上,確實能有效增加分類器的成效,而且比傳統的SVM及FSVM模型好。

    Data mining techniques, a part of Knowledge Discovery, is used to extract the hidden valuable information from large amounts of data. It is widely used for consumer loan evaluation and forecasting financial distress analysis. However, most of them are lack of business intelligence since they are data-driven and it causes a big gap between academic and business goal. In many real-world problems, we can see that there are some monotonicity relationships between the class and attributes and it has been shown that a classification technique incorporated with monotonicity constraints can improve accuracy.
    Support vector machine (SVM) is a state-of-the-art artificial neural network based on statistical learning. The excellent ability is the focus of research in machine learning. However, some input points are more important to be fully assigned to one class to that SVM can separate these points more correctly. Some are corrupted by noises are less meaningful and the machine should discard them. Since importing the fuzzy theory, fuzzy SVM can provide different importance on the different information and give a higher membership to the information which is more contributions for decision-making problems.
    In this study, we propose a knowledge-oriented new fuzzy support vector machine model with monotonicity constraints. Exploiting the experts knowledge to retrieve the monotonic rules from datasets. Constructing monotonicity constraints and determining the contribution of each information to implement the proposed classification model. The results of the experiments show that the proposed method, which considers the prior domain knowledge of monotonicity and different contribution of each data, performs better than the original SVM and FSVM model on classification problem.

    摘要 I ABSTRACT II 誌謝 III CONTENTS IV LIST of FIGURE V LIST of TABLE V Chapter 1 Introduction 1 1.1 Background and motivation 1 1.2 Objectives of Research 4 1.3 Organization of Research 4 Chapter2 Literature Review 6 2.1 Support vector machine (SVM) 6 2.1.1 Introduce SVM 6 2.1.2 Architecture of SVMs 8 2.2 Fuzzy support vector machines (FSVMs) 11 2.2.1 Introduce FSVMs 11 2.2.2 Architecture of FSVMs 11 2.3 Classification with Monotonicity Constraints 14 Chapter 3 Research Methodology 18 3.1 Data preprocessing 19 3.2 Concept of monotonicity 20 3.3 Derivation of the Monotonicity Constrained Fuzzy SVM Model 21 3.4 Constructing Monotonicity Constraints 25 3.5 Generating Membership 27 3.6 MC-FSVM Algorithm 27 Chapter 4 Experiment and Result analysis 30 4.1 Environment of Experiments and Data Collection 30 4.1.1 Experiment environment 30 4.1.2 Data Collection 31 4.2 Experiment step 37 4.3 Performance measures 37 4.4 Experiment result 41 Chapter 5 Conclusions and Suggestions 50 5.1 Conclusions 50 5.2 Recommendations for future research 51 Reference 52

    Archer, N. P., & Wang, S. (1993). Learning bias in neural networks and an approach to controlling its effect in monotonic classification. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 15(9), 962-966. doi: 10.1109/34.232084
    Burges, C. J. C. (1998). A Tutorial on Support Vector Machines for Pattern Recognition. Data Min. Knowl. Discov., 2(2), 121-167. doi: 10.1023/a:1009715923555
    Chun-Fu, L., & Sheng-De, W. (2002). Fuzzy support vector machines. Neural Networks, IEEE Transactions on, 13(2), 464-471. doi: 10.1109/72.991432
    Cortes, C., & Vapnik, V. (1995). Support-Vector Networks. Mach. Learn., 20(3), 273-297. doi: 10.1023/a:1022627411411
    Courant, R., & Hilbert, D. (1970). Methods of Mathematical Physics (Vol. I, II). New York: Wiley Interscience.
    Davis, S. M., & Botkin, J. W. (1994). The Monster Under the Bed: How Business Is Mastering the Opportunity of Knowledge for Profit: Simon & Schuster.
    Decherchi, S., Ridella, S., Zunino, R., Gastaldo, P., & Anguita, D. (2010). Using Unsupervised Analysis to Constrain Generalization Bounds for Support Vector Classifiers. Neural Networks, IEEE Transactions on, 21(3), 424-438. doi: 10.1109/tnn.2009.2038695
    Dembczyński, K., Kotłowski, W., & Słowiński, R. (2008). Ensemble of Decision Rules for Ordinal Classification with Monotonicity Constraints. In G. Wang, T. Li, J. Grzymala-Busse, D. Miao, A. Skowron & Y. Yao (Eds.), Rough Sets and Knowledge Technology (Vol. 5009, pp. 260-267): Springer Berlin Heidelberg.
    Doumpos, M., & Pasiouras, F. (2005). Developing and Testing Models for Replicating Credit Ratings: A Multicriteria Approach. Computational Economics, 25(4), 327-341. doi: 10.1007/s10614-005-6412-4
    Doumpos, M., & Zopounidis, C. (2009). MONOTONIC SUPPORT VECTOR MACHINES FOR CREDIT RISK RATING. New Mathematics and Natural Computation, 05(03), 557-570. doi: doi:10.1142/S1793005709001520
    Doumpos, M., Zopounidis, C., & Golfinopoulou, V. (2007). Additive Support Vector Machines for Pattern Classification. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 37(3), 540-550. doi: 10.1109/tsmcb.2006.887427
    Duivesteijn, W., & Feelders, A. (2008). Nearest Neighbour Classification with Monotonicity Constraints. Paper presented at the Proceedings of the 2008 European Conference on Machine Learning and Knowledge Discovery in Databases - Part I, Antwerp, Belgium.
    Evgeniou, T., Boussios, C., & Zacharia, G. (2005). Generalized Robust Conjoint Estimation. Marketing Science, 24(3), 415-429. doi: 10.1287/mksc.1040.0100
    Falck, T., Suykens, J. A. K., & De Moor, B. (2009, 15-18 Dec. 2009). Robustness analysis for Least Squares kernel based regression: an optimization approach. Paper presented at the Decision and Control, 2009 held jointly with the 2009 28th Chinese Control Conference. CDC/CCC 2009. Proceedings of the 48th IEEE Conference on.
    Gamarnik, D. (1998). Efficient learning of monotone concepts via quadratic optimization. Paper presented at the Proceedings of the eleventh annual conference on Computational learning theory, Madison, Wisconsin, United States.
    Gestel, T. V., Baesens, B., Suykens, J. A. K., Van den Poel, D., Baestaens, D.-E., & Willekens, M. (2006). Bayesian kernel based classification for financial distress detection. European Journal of Operational Research, 172(3), 979-1003. doi: http://dx.doi.org/10.1016/j.ejor.2004.11.009
    Greco, S., Matarazzo, B., & Słowiński, R. (1998). A new rough set approach to evaluation of bankruptcy risk. [Book Section]. Operational tools in the management of financial risks, 121-136.
    Gruber, C., Gruber, T., Krinninger, S., & Sick, B. (2010). Online Signature Verification With Support Vector Machines Based on LCSS Kernel Functions. Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on, 40(4), 1088-1100. doi: 10.1109/tsmcb.2009.2034382
    Guyon, I., Matic, N., & Vapnik, V. (1996). Discovering informative patterns and data cleaning. In M. F. Usama, P.-S. Gregory, S. Padhraic & U. Ramasamy (Eds.), Advances in knowledge discovery and data mining (pp. 181-203): American Association for Artificial Intelligence.
    Hua, Z., Wang, Y., Xu, X., Zhang, B., & Liang, L. (2007). Predicting corporate financial distress based on integration of support vector machine and logistic regression. Expert Systems with Applications, 33(2), 434-440. doi: http://dx.doi.org/10.1016/j.eswa.2006.05.006
    Huang, W., Nakamori, Y., & Wang, S.-Y. (2005). Forecasting stock market movement direction with support vector machine. Computers & Operations Research, 32(10), 2513-2522. doi: http://dx.doi.org/10.1016/j.cor.2004.03.016
    Huang, Z., Chen, H., Hsu, C.-J., Chen, W.-H., & Wu, S. (2004). Credit rating analysis with support vector machines and neural networks: a market comparative study. Decis. Support Syst., 37(4), 543-558. doi: 10.1016/s0167-9236(03)00086-1
    Kim, H. S., & Sohn, S. Y. (2010). Support vector machines for default prediction of SMEs based on technology credit. European Journal of Operational Research, 201(3), 838-846. doi: http://dx.doi.org/10.1016/j.ejor.2009.03.036
    Kramer, K. A., Hall, L. O., Goldgof, D. B., Remsen, A., & Luo, T. (2009). Fast support vector machines for continuous data. Trans. Sys. Man Cyber. Part B, 39(4), 989-1001. doi: 10.1109/tsmcb.2008.2011645
    Kumar, M. A., & Gopal, M. (2010). A comparison study on multiple binary-class SVM methods for unilabel text categorization. Pattern Recognition Letters, 31(11), 1437-1444. doi: http://dx.doi.org/10.1016/j.patrec.2010.02.015
    Lauer, F., Suen, C. Y., & Bloch, G. (2007). A trainable feature extractor for handwritten digit recognition. Pattern Recognition, 40(6), 1816-1824. doi: http://dx.doi.org/10.1016/j.patcog.2006.10.011
    Li, S.-T., Shiue, W., & Huang, M.-H. (2006). The evaluation of consumer loans using support vector machines. Expert Systems with Applications, 30(4), 772-782. doi: http://dx.doi.org/10.1016/j.eswa.2005.07.041
    Lin, C. F., & Wang, S. D. (2002). Fuzzy Support Vector Machines. IEEE Transactions on Neural Networks, 13(2), 464 - 471.
    Müller, K. R., Smola, A. J., Rätsch, G., Schölkopf, B., Kohlmorgen, J., & Vapnik, V. (1997). Predicting time series with support vector machines. In W. Gerstner, A. Germond, M. Hasler & J.-D. Nicoud (Eds.), Artificial Neural Networks — ICANN'97 (Vol. 1327, pp. 999-1004): Springer Berlin Heidelberg.
    Man Gyun, N., Won Seo, P., & Dong Hyuk, L. (2008). Detection and Diagnostics of Loss of Coolant Accidents Using Support Vector Machines. Nuclear Science, IEEE Transactions on, 55(1), 628-636. doi: 10.1109/tns.2007.911136
    Mariéthoz, J., & Bengio, S. (2007). A Kernel Trick For Sequences Applied to Text-Independent Speaker Verification Systems. Pattern Recognition.
    Mercer, J. (1909). Functions of Positive and Negative Type, and Their Connection with the Theory of Integral Equations. Transactions of the London Philosophical Society (V), 9, 415-446.
    Mukherjee, S., Osuna, E., & Girosi, F. (1997, 24-26 Sep 1997). Nonlinear prediction of chaotic time series using support vector machines. Paper presented at the Neural Networks for Signal Processing [1997] VII. Proceedings of the 1997 IEEE Workshop.
    Pazzani, M. J., Mani, S., & Shankle, W. R. (2001). Acceptance of Rules Generated by Machine Learning among Medical Experts. Methods of Information in Medicine(2001 (Vol. 40): Issue 5 2001), 380-385.
    Pelckmans, K., Espinoza, M., Brabanter, J., Suykens, J. A., & Moor, B. (2005). Primal-Dual Monotone Kernel Regression. Neural Process. Lett., 22(2), 171-182. doi: 10.1007/s11063-005-5264-1
    Pendharkar, P. C. (2005). A data envelopment analysis-based approach for data preprocessing. Knowledge and Data Engineering, IEEE Transactions on, 17(10), 1379-1388. doi: 10.1109/tkde.2005.155
    Pendharkar, P. C., & Rodger, J. A. (2003). Technical efficiency-based selection of learning cases to improve forecasting accuracy of neural networks under monotonicity assumption. Decision Support Systems, 36(1), 117-136. doi: http://dx.doi.org/10.1016/S0167-9236(02)00138-0
    Potharst, R., & Feelders, A. J. (2002). Classification trees for problems with monotonicity constraints. SIGKDD Explor. Newsl., 4(1), 1-10. doi: 10.1145/568574.568577
    Qinghua, H., Xunjian, C., Lei, Z., Zhang, D., Maozu, G., & Yu, D. (2012). Rank Entropy-Based Decision Trees for Monotonic Classification. Knowledge and Data Engineering, IEEE Transactions on, 24(11), 2052-2064. doi: 10.1109/tkde.2011.149
    Ravikumar, B., Thukaram, D., & Khincha, H. P. (2009). An Approach Using Support Vector Machines for Distance Relay Coordination in Transmission System. Power Delivery, IEEE Transactions on, 24(1), 79-88. doi: 10.1109/tpwrd.2008.2002971
    Schölkopf, B., & Smola, A. J. (2002). Learning with Kernels --Support Vector Machines, Regularization, Optimization and Beyond. Cambridge, Massachusetts: The MIT Press.
    Shilton, A., Palaniswami, M., Ralph, D., & Ah Chung, T. (2005). Incremental training of support vector machines. Neural Networks, IEEE Transactions on, 16(1), 114-131. doi: 10.1109/tnn.2004.836201
    Shin, K.-S., Lee, T. S., & Kim, H.-j. (2005). An application of support vector machines in bankruptcy prediction model. Expert Systems with Applications, 28(1), 127-135. doi: http://dx.doi.org/10.1016/j.eswa.2004.08.009
    Vapnik, V. N. (1995). The nature of statistical learning theory: Springer-Verlag New York, Inc.
    Vapnik, V. N. (1998). Statistical learning theory: Wiley.
    Vazirigiannis, M., Halkidi, M., & Gunopulos, D. (2003). Uncertainty handling and quality assessment in data mining. London; New York: Springer.
    Wang, S. (1995). The Unpredictability of Standard Back Propagation Neural Networks in Classification Applications. Management Science, 41(3), 555-559. doi: 10.2307/2632981
    Wang, S. (2003). Adaptive non-parametric efficiency frontier analysis: a neural-network-based model. Computers & Operations Research, 30(2), 279-295. doi: http://dx.doi.org/10.1016/S0305-0548(01)00095-8
    Xu, Y., Wang, X. B., Ding, J., Wu, L. Y., & Deng, N. Y. (2010). Lysine acetylation sites prediction using an ensemble of support vector machine classifiers. Journal of theoretical biology, 264(1), 130-135.
    Xuegong, Z. (1999, Aug 1999). Using class-center vectors to build support vector machines. Paper presented at the Neural Networks for Signal Processing IX, 1999. Proceedings of the 1999 IEEE Signal Processing Society Workshop.

    無法下載圖示 校內:2023-12-31公開
    校外:不公開
    電子論文尚未授權公開,紙本請查館藏目錄
    QR CODE