在知識擷取領域中，凱利方格法是最常被用來擷取知識的方法論之一。現今，凱利方格法被不斷改良，且被廣泛運用在各種不同領域的研究上，甚至被用來幫助開發各種決策支援系統或專家系統。而在這些應用中，凱利方格主要被用來擷取研究對象的經驗與知識，以協助研究者完成研究的目的，或是支援個人或組織決策者制定決策，達成問題解決的目標。但是在這些以凱利方格法所建構完成的方格為資料來源的研究或是開發來源的系統，往往沒有考量到知識來源的正確性，或知識擷取過程中是否有不良的影響，造成擷取出來的知識品質有問題，在這樣的情況下，運用未評估過的方格作為研究或系統開發的資料來源，將可能有潛在的問題。
本研究以知識發現的方式，提出一套知識擷取效能的評估模式。試圖以模糊凱利方格作為知識擷取的工具，擷取出專家知識並轉換成具備可讀性的法則，進一步將擷取出來的專家知識與大量的案例資料庫結合，萃取出隱藏在資料庫中的法則，將兩方產生出來的規則同時針對未知的資料進行預測，並藉著計算兩方規則的預測精確度與一致性程度，來達成評估知識擷取品質之目的。最後，本研究將此模式應用於評估訪談試管嬰兒領域專家所擷取出來的知識，展示如何以預測精確度與一致性來評估該次知識擷取的知識品質，並有效地將專家知識轉換成可用的法則。

Recently, the repertory grid technique, named as RG, has been one of the most popular methods used for knowledge acquisition from experts. Nowadays, RG has also been continuously improved and widely applied to build decision support systems or knowledge-based systems. In addition, RG is also being modified to accommodate group decision problems. The main drawback of RG is the lack of quality control which may lead to serious impacts in further researches. The low quality knowledge may be resulted from poor experts or inappropriate RG procedures and thus impede the efficiency of knowledge management activities. Therefore, how to evaluate the effectiveness of knowledge becomes the most critical issue in developing knowledge-based systems.
In this research, we propose a knowledge discovery approach to evaluate the quality of knowledge which was extracted from RG. In the first step, RG is used as a tool of knowledge acquisition, to extract the knowledge from experts and converts it to readable rules. In the second step, the tacit knowledge embedded in huge case base will be extracted to formulate the other readable rule set. In the third step, a rule comparison algorithm is proposed in order to calculate the accuracy and consistency between above two readable rule sets. Finally, we build a criterion using accuracy and consistency to evaluate the performance of knowledge acquisition. In the experimentation, RG is applied to extract tacit knowledge from an In Vitro Fertilization domain expert. The results indicate that the proposed method is well performed in evaluating domain expert’s knowledge while reasoning to the huge knowledge base. Furthermore, the uncovered rules can also facilitate knowledge comprehension and achieve better knowledge management effectiveness.

Adelman, L. Measurement issues in knowledge engineering. Systems, Man and Cybernetics, IEEE Transactions on, 19(3), 483-488, 1989.
Bhatia, S. K., & Yao, Q. A new approach to knowledge acquisition by repertory grids, 1993.
Byrd, T. A. Expert systems implementation. Industrial Management & Data Systems, 95(10), 3-7, 1995.
Castro-Schez, J. J., Castro, J. L., & Zurita, J. M. Fuzzy Repertory Table: A Method for Acquiring Knowledge About Input Variables to Machine Learning Algorithm. IEEE TRANSACTIONS ON FUZZY SYSTEMS, 12(1), 123-139, 2004.
Castro-Schez, J. J., Jimenez, L., & Rodriguez, J. M. L. Using Fuzzy Repertory Talbe-based Technique For Decision Support. Decision Support System, 39, 293-307, 2005.
Chu, H. C., & Hwang, G. J. A Delphi-based approach to developing expert systems with the cooperation of multiple experts. Expert Systems with Applications, 34(4), 2826-2840, 2008.
Cullen, J., & Bryman, A. The Knowledge Acquisition Bottleneck: Time for Reassessment? Expert Systems, 5(3), 216-225, 1988.
Diederich, R. A. J., & Tickle, A. B. Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks. Knowledge-Based Systems, 8(6), 373-389, 1995.
Dubois, D., & Prade, H. Fuzzy Sets and Systems: Theory and Applications: Academic Press New York, 1980.
Dunham, M. H. Data mining: Introductory and advanced topics: Prentice Hall PTR Upper Saddle River, NJ, USA, 2002.
Easterby-Smith, M., Thorpe, R., & Holman, D. Using Repertory Grids In Management. Journal of European Industrial Training, 20(3), 3-30, 1996.
Fallman, D., & Waterworth, J. A. Dealing with user experience and affective evaluation in HCI design: a repertory grid approach, 2005.
Ford, K. M., Petry, F. E., Adams-Webber, J. R., & Chang, P. J. An approach to knowledge acquisition based on the structure ofpersonal construct systems. Knowledge and Data Engineering, IEEE Transactions on, 3(1), 78-88, 1991.
Gaines, B. R., & Shaw, M. L. G. New directions in the analysis and interactive elicitation of personal construct systems. Int. J. Man Mach. Stud., 13, 81-116, 1980.
Garbe, H., Janssen, C., Mobus, C., Seebold, H., & de Vries, H. KARaCAs: Knowledge Acquisition with Repertory Grids and Formal Concept Analysis for Dialog System Construction. LECTURE NOTES IN COMPUTER SCIENCE, 4248, 3, 2006.
Hong, T. P., & Chen, J. B. Finding relevant attributes and membership functions. Fuzzy Sets and Systems, 103(3), 389-404, 1999.
Hong, T. P., & Chen, J. B. Processing individual fuzzy attributes for fuzzy rule induction. Fuzzy sets and systems, 112(1), 127-140, 2000.
Huang, M. J., & Chen, M. Y. Integrated design of the intelligent web-based Chinese Medical Diagnostic System (CMDS)–Systematic development for digestive health. Expert Systems with Applications, 32(2), 658-673, 2007.
Hudlicka, E. Requirements elicitation with indirect knowledge elicitationtechniques: comparison of three methods, 1996.
Hudlicka, E. Knowledge Elicitation in Complex Military Environments. Paper presented at the IEEE conference and Workshop on Enineering of Computer-Based System, Nashville, TN, USA, 1999.
Hwang, G. Knowledge acquisition for fuzzy expert systems. International journal of intelligent systems, 10(6), 541-560, 1995.
Hwang, G. J., & Tseng, S. S. EMCUD: A knowledge acquisition method which captures embedded meanings under uncertainty. International Journal of Human-Computer Studies, 33, 431-451, 1990.
Jankowicz, D. The Easy Guide to Repertory Grids: Wiley, 2004.
Jankowicz, D. The easy guide to repertory grids: Wiley, 2004.
Jurisica, I., Mylopoulos, J., Glasgow, J., Shapiro, H., & Casper, R. F. Case-based reasoning in IVF: prediction and knowledge mining. Artificial Intelligence in Medicine, 12(1), 1-24, 1998.
Kelly, G. A. The Psychology of Personal Constructs, New York: Norton., 1955.
Klir, G. J., & Yuan, B. Fuzzy sets and fuzzy logic: theory and applications: Prentice Hall Upper Saddle River, NJ, 2005.
Leach, C., & Freshwater, K. Analysis of repertory grids in clinical practice. A Handbook of Research Methods for Clinical and Health Psychology, 125, 2005.
Lee, J., Lee, Y., Ryu, Y., & Kang, T. H. Information Quality Drivers of KMS, 2007.
Leitich, H., Adlassnig, K. P., & Kolarz, G. Evaluation of two different models of semi-automatic knowledge acquisition for the medical consultant system CADIAG-II/RHEUMA. Artificial Intelligence In Medicine, 25(3), 215-225, 2002.
Manning, C. D., Raghavan, P., & Schtze, H. Introduction to information retrieval: Cambridge University Press New York, NY, USA, 2008.
Ostrovsky, M. A., Weetall, H. H., Parama, J. R., Penabad, M. R., Brisaboa, N. R., & Amparo, A. B. EECB: a knowledge elicitation tool based on repertory grid and city block metric. Expert Systems with Applications, 14(3), 249-258, 1998.
Shanteau, J., Weiss, D. J., Thomas, R. P., & Pounds, J. C. Performance-based assessment of expertise: How to decide if someone is an expert or not. European Journal of Operational Research, 136(2), 253-263, 2002.
Shaw, M. L. G., & Gaines, B. R. Comparing Conceptual Structures: Consensus, Conflict, Correspondence and Contrast. Knowledge Acquisition, 1(4), 341-363, 1989.
Shaw, M. L. G., & Thomas, L. F. FOCUS On Education: An Interactive Computer System For The Development And Analysis Of Repertory Grid. International Journal of Man-Machine Studies, 10(2), 139-173, 1978.
Soo, C. W., & Devinney, T. M. The role of knowledge quality in firm performance. Organizations as Knowledge Systems: Knowledge, Learning and Dynamic Capabilities, 252, 2004.
Stumpf, S., & McDonnell, J. (2003). Using Repertory Grids to Test Data Quality and Experts' Hunches. Paper presented at the the 14th International Workshop on Database and Expert Systems Applications(DEXA'03), 2003.
Tan, F. B., & Hunter, M. G. The Repertory Grid Technique: A Method for the Study of Cognition in Information Systems. MIS Quarterly, 26(1), 39-57, 2002.
Wagner, W. P., Najdawi, M. K., & Chung, Q. B. Selection of knowledge acquisition techniques based upon the problem domain characteristics of production and operations management expert systems. Expert Systems, 18(2), 76-87, 2001.
Wilson, J. M., & Retsas, A. P. Personal Constructs of Nursing Practice: A Comparative Analysis Of Three Groups Of Australian Nurses. International Journal of Nursing Studies, 34(1), 63-71, 1997.
Zadeh, L. A. The role of fuzzy logic in the management of uncertainty in expert systems. Fuzzy sets and systems, 11(3), 199-227, 1983.