比較分類演算法成效時，會透過統計檢定來判斷分類正確率間是否有顯著差異，而分類正確率間獨立或相依的情況下，採取的檢定做法會有所不同，因此本研究利用歸納偏誤的角度來探討兩分類演算法間的分類正確率為獨立或相依關係。分類演算法中的歸納偏誤可分為語言偏誤和搜尋偏誤，本研究中設計了三組實驗，第一組為兩分類演算法間具有不同的語言偏誤、相同的搜尋偏誤，第二組為具有相同的語言偏誤、不同的搜尋偏誤，第三組為具有不同的語言偏誤、不同的搜尋偏誤，接著利用相依性檢定方法針對分類正確率間的相關係數進行統計檢定，來檢驗兩分類演算法間具備哪一種歸納偏誤組合時，分類正確率會是獨立或相依。共利用30個資料檔進行實驗，實驗結果顯示，具備不同語言偏誤、不同搜尋偏誤的分類演算法間，分類正確率在大多數資料檔下為獨立關係，評估彼此間成效時較能直接以獨立假設進行檢定，而不同語言偏誤、相同搜尋偏誤的分類演算法間，若是離散化與否所導致的語言偏誤不同，分類正確率亦在大多數情況下為獨立的關係，而相同語言偏誤、不同搜尋偏誤的分類演算法間，多數資料檔下分類正確率呈現相依關係，但是仍有不少獨立的情形，因此建議仍需採用相依性檢定方法進行判定。

In comparing the performance of classification algorithms, most studies apply statistical methods to test whether their accuracy estimates are significantly different. However, the statistical methods for this purpose should depend on whether the accuracy estimates are independent or not. This study analyzes the dependency relationships between the accuracy estimates of two classification algorithms based on their inductive bias. The inductive bias of a classification algorithm is a combination of language bias and search bias. Three experiments are proposed in this study for the dependency analysis of the accuracy estimates resulting from k-fold cross validation, and they are: two algorithms with different language bias and same search bias, two algorithms with same language bias and different search bias, and two algorithms with different language bias and different search bias. Thirty data sets and three classification algorithms are chosen to perform the experiments. The experimental results suggest that the accuracy estimates for classification algorithms with different language bias and search bias can be assumed to be independent. In addition, it is also appropriate to assume that classification algorithms with different language bias and same search bias because of attribute discretization will have independent accuracy estimates.

陳育生 (2015)。多個資料檔下比較兩分類方法表現之有母數統計方法。國立成功大學資訊管理研究所碩士論文。

何儀珊 (2016)。評估兩相依分類演算法效能之方法。國立成功大學資訊管理研究所碩士論文。

Alpaydm, E. (1999). Combined 5× 2 cv F test for comparing supervised classification learning algorithms. Neural Computation, 11(8), 1885-1892.

Bruha, I. and Franek, F. (1996). Comparison of various routines for unknown attribute value processing: The covering paradigm. International Journal of Pattern Recognition and Artificial Intelligence, 10(8), 939-955.

Dougherty, J., Kohavi, R., and Sahami, M. (1995). Supervised and unsupervised discretization of continuous features. Proceedings of the 12th International Conference on Machine Learning, 194-202.

Fayyad, U. M. and Irani, K. B. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. Proceedings of the 13th International Joint Conference on Artificial Intelligence, 1022-1027.

Friedman, J. H. (1998). Data Mining and Statistics: What's the Connection? Paper presented at the 29th Symposium on the Interface: Computer Science and Statistics, Houston, U.S.A..
Grzymala-Busse, J. W. and Hu, M. (2000). A comparison of several approaches to missing attribute values in data mining. Rough Sets and Current Trends in Computing: 2nd International Conference, 378-385.

Jensen, D. and Oates, T. (1997). The effects of training set size on decision tree complexity. Proceedings of the 14th International Conference on Machine Learning, 254-262.

John, G. H., Kohavi, R., and Pfleger, K. (1994). Irrelevant features and the subset selection problem. Proceedings of the 11th International Conference on Machine Learning, 121-129.

Kerber, R. (1992). Chimerge: Discretization of numeric attributes. Proceedings of the 10th National Conference on Artificial Intelligence, 123-128.

Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. Proceedings of the 14th International Joint Conference on Artificial Intelligence, 1137-1143.

Kotsiantis, S. and Kanellopoulos, D. (2006). Discretization techniques: A recent survey. International Transactions on Computer Science and Engineering, 32(1), 47-58.

Kotsiantis, S. B., Kanellopoulos, D., and Pintelas, P. E. (2006). Data preprocessing for supervised leaning. International Journal of Computer Science, 1(2), 111-117.

Langley, P. and Sage, S. (1994). Induction of selective Bayesian classifiers. Proceedings of the 10th International Conference on Uncertainty in Artificial Intelligence, 399-406.

Liu, H., Hussain, F., Tan, C. L., & Dash, M. (2002). Discretization: An enabling technique. Data Mining and Knowledge Discovery, 6(4), 393-423.

Liu, H. and Setiono, R. (1995). Chi2: Feature selection and discretization of numeric attributes. Proceedings of the 7th International Conference on Tools with Artificial Intelligence, 388-391.

Mitchell, T. M. (1980). The Need for Biases in Learning Generalizations. Rutgers CS tech report (Report CBM-TR-117). New Brunswick, U.S.A.: Rutgers University, Department of Computer Science.

Mitchell, T. M. (1997). Machine Learning. U.S.A: McGraw Hill.

Singhi, S. K. and Liu, H. (2006). Feature subset selection bias for classification learning. Proceedings of the 23rd International Conference on Machine Learning, 849-856.

Snedecor, G. W. and Cochran, W. G. (1989). Statistical Methods(8th ed.). Ames, U.S.A: The Lowa State University Press.

Tan, P. N., Steinbach, M., and Kumar, V. (2006). Introduction to Data Mining. U.S.A: Pearson Education.

Whigham, P. A. (1996). Search bias, language bias and genetic programming. Proceedings of the 1st Annual Conference on Genetic Programming, 230-237.

Witten, I. H., Frank, E., Hall, M. A., and Pal, C. J. (2011). Data Mining: Practical Machine Learning Tools and Techniques(3rd ed.). U.S.A: Morgan Kaufmann.

Wolpert, D. H. & Macready, W. G. (1995). No Free Lunch Theorems for Search. Technical Report SFI-TR-95-02-010, Santa Fe Institute.

Wong, T. T. (2015). Performance evaluation of classification algorithms by k-fold and leave-one-out cross validation. Pattern Recognition, 48(9), 2839-2846.