簡易檢索 / 詳目顯示

研究生: 周博文
Chou, Po-Wen
論文名稱: 基於馬可夫隨機場之非線性核函數特徵萃取
An MRF-based Kernel Methods for Nonlinear Feature Extraction
指導教授: 謝璧妃
Hsieh, Pi-Fuei
學位類別: 碩士
Master
系所名稱: 電機資訊學院 - 資訊工程學系
Department of Computer Science and Information Engineering
論文出版年: 2007
畢業學年度: 95
語文別: 英文
論文頁數: 66
中文關鍵詞: 紋理輻射基底函數核函數非線性降維馬可夫隨機場
外文關鍵詞: Markov random field, Nonlinear feature extraction, Contexture, Radial basis function, Kernel function
相關次數: 點閱:159下載:2
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報
  •   在處理資料時,特徵粹取主要應用於移除多餘的特徵,並且增進處理效果。當類別間的邊界為非線性時,線性降維方法功效不如非線性降維方法,可在低維度空間分離出類別資訊。基於核函數之非線性降維法是以一正定函數點積方式,投影資料到高維度並結合線性降維方法,來模擬實現非線性降維法。在核函數方法中,輻射基底函數是常見拿來應用的核函數,然而對於特徵粹取方法而言,基於輻射基底函數之核函數方法只考慮歐式距離,因此,此函數並不適用於分類的問題中。
      在本論文中分別應用影像上資料的紋理以及類別資訊加以改進輻射基底函數。若資料屬於同類別或者其影像上的結構相似,我們將適時增強其在核函數中的相似度,這使得應用核函數之非線性降維法時,能增加其類別間的鑑別度。
      我們以合成及實際影像來測試效能好壞。實驗證實所提出之基於馬可夫隨基場核函數比輻射基底函數和線性降維法有較好的分類正確率,而且所提出的基於馬可夫隨基場核函數特徵粹取能以單點分類器模式勝過基於馬可夫隨機場之分類器。

      Feature extraction has been intensively used in pattern recognition for removing redundant features and accelerating data processing. When classes are separated by a nonlinear boundary, the linear feature reduction methods may not as efficiently as nonlinear methods separate classes in a low dimensional feature space. By replacing the inner product with an appropriate positive definite function, the kernel-based nonlinear feature extraction methods implicitly perform a nonlinear mapping of the input data into a high-dimensional feature space and then compute a linear feature extraction method. The radial basis kernel function (RBF) is a commonly seen kernel trick. However, the RBF kernel method may not fulfill feature extraction perfectly in classification problem since its similarity measure is based merely on the Euclidean distance.
      This study incorporates contextualand class information into the RBF kernel function to improve the performance of kernel-based feature extraction. Samples are considered closer in case of the same class labels or in case that their neighbors are alike in the MRF sense. This leads to an increased discrimination between samples when using kernel function implicitly for a nonlinear mapping of data into a higher-dimensional space.   
      Experiments have been tested on three synthesized dataset and real images. In our experiments, the proposed MRF-based kernel method yielded comparatively higher classification accuracy than the traditional RBF-based kernel method and other linear feature extraction methods. Also, the MRF-based kernel feature extraction, followed by a pixel-wise classifier, out performed the MRF-based contextual classifier.

    1. Introduction............................................................................................. 1 1.1 Motivation........................................................................................................1 1.2 Related Work....................................................................................................6 1.3 Objective/Method...........................................................................................10 1.4 Organization...................................................................................................12 2. Markov Random Field........................................................................... 13 2.1 Neighbor system and Cliques..........................................................................13 2.2 Definition for Markov random fields ..............................................................14 2.2.1 Hammersley-Clifford theorem.............................................................17 2.2.2 M-Level Markov random fields...........................................................18 3. Kernel methods for nonlinear feature extraction .................................... 20 3.1 Introduction....................................................................................................20 3.2 Kernel-based Statistical Pattern Analysis ........................................................21 3.3 The Kernel Trick ............................................................................................21 3.4 Rules for Kernel matrix ..................................................................................23 3.5 Reproducing Kernel Hilbert Space (RKHS) ....................................................24 3.6 Examples of Kernels.......................................................................................25 4. Proposed Method................................................................................... 27 4.1 Architecture and Function...............................................................................27 4.2 Kernel Function..............................................................................................28 5. Experimental Results............................................................................. 32 5.1 Simulated Dataset ...........................................................................................33 5.1.1 Dataset 1.............................................................................................34 5.1.2 Dataset 2.............................................................................................39 5.1.3 Dataset 3.............................................................................................46 5.2 Real Dataset ...................................................................................................52 5.2.1 Dataset 4.............................................................................................52 5.2.2 Dataset 5.............................................................................................58 6. Conclusions ........................................................................................... 62 References.................................................................................................... 64

    [1] D. K. Panjwani and G. Healey, “Markov random field models for unsupervised segmentation of textured color images,” IEEE Trans. Pattern Analysis and Machine
    Intelligence, vol. 17, no. 10, pp. 939–954, Oct. 1995.
    [2] T. Yamasaki and D. Gingras, “Image classification using spectral and spatial information based on MRF models,” IEEE Trans. Image Processing, vol. 4, pp.
    1333–1339, Sep. 1995.
    [3] G. Rellier, X. Descombes, F. Falzon, and J. Zerubia, ”Texture feature analysis using a Gauss–Markov model in hyperspectral image classification,” IEEE Trans. Geosci. Remote Sensing, vol. 42, pp 1543-1551, July 2004.
    [4] R. A. Fisher, “The used of multiple measurements in taxonomic problems,”Ann. Eugenics, vol. 7, pp. 179–188, 1936.
    [5] K. Fukunaga, Introduction to Statistical Pattern Recognition, 2nd ed. New York: Academic, 1990.
    [6] S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K.-R. Muller, “Fisher discriminant analysis with kernels,” Proc.IEEE Int’l Workshop Neural Networks for Signal Processing IX, pp. 41–48, Aug. 1999.
    [7] V. Roth and V. Steinhage, “Nonlinear discriminant analysis using kernel functions,” Advances in Neural Information Processing Systems, S. A. Solla, T. K. Leen, and K. R. Muller, Eds. Cambridge, MA: MIT Press, vol. 12, pp. 568–574, 1999.
    [8] G. Baudat and F. Anouar, “Generalized discriminant analysis using a kernel approach,” Neural Comput., vol. 12, no. 10, pp. 2385–2404, 2000.
    [9] B. Scho"lkopf, A. Smola, and K.R. Mu"ller, “Nonlinear Component analysis as a kernel eigenvalue problem,” Neural Comput., vol. 10, no. 5, pp. 1299–1319, 1998.
    [10] S. Mika, G. Ra"tsch, J. Weston, B. Scho"lkopf, and K. R.Mu"ller, “Fisher discriminant analysis with kernels,” Proc. IEEE Int’l Workshop Neural Networks for Signal Processing IX, pp. 41–48, Aug. 1999.
    [11] G. Baudat and F. Anouar, “Generalized discriminant analysis using a kernel approach,” Neural Comput., vol. 12, no. 10, pp. 2385–2404, 2000.
    [12] G. C. Valls and L. Bruzzone, “Kernel-based methods for hyperspectral image classification,” IEEE Trans. Geosci. Remote Sensing, vol. 43, no. 6, pp. 1351–1362, Jun. 2005.
    [13] K. I. Kim, S. H. Park, and H. J. Kim, “Kernel principal component analysis for texture classification,” IEEE Signal Processing Letters, vol. 8, no. 2, pp. 39–41, Feb. 2001.
    [14] K. I. Kim, K. Jung, and H. J. Kim, “Face recognition using kernel principal component analysis,” IEEE Signal Processing Letters, vol. 9, no. 2, pp. 40–42, Feb. 2002.
    [15] M. Zhu and A. M. Martinez, “Subclass discriminant analysis,” IEEE Trans. Pattern Anal. Machine Intell., vol. 28, no. 8, pp. 1274–1286, Aug., 2006.
    [16] A. Kocsor and L. Tóth, “Kernel-based feature extraction with a speech technology application,” IEEE Trans. Signal Processing, vol. 52, no. 8, pp. 2250–2263, Aug. 2004.
    [17] D. Comaniciu and P. Meer, “Kernel-based object tracking,” IEEE Trans. Pattern Anal. Machine Intell., vol. 25, no. 5, pp. 564–577, May 2003.
    [18] M. M. Dundar and D. A. Landgrebe, “Toward an optimal supervised classifier for the analysis of hyperspectral data,” IEEE Trans. Geosci. Remote Sensing, vol. 42, no. 1, pp. 271–277, Jan. 2004.
    [19] J. Ma, J. L. S. Gómez and S. C. Ahalt, “Nonlinear muticlass discriminant analysiss,”IEEE Signal Processing Letters, vol0, no 7, pp. 196–199, Jul. 2003.
    [20] K. I. Kim, M. O. Franz, and B. Scho¨lkopf,“Iterative kernel principal component analysis for image modeling,” IEEE Trans. Pattern Anal. Machine Intell., vol. 27, no. 9, pp. 1351–1366, Sep. 2005.
    [21] H. Kwon and N. M. Nasrabadi, “Kernel orthogonal subspace projection for hyperspectral signal classification,” IEEE Trans. Geosci. Remote Sensing, vol. 43, no. 12, pp. 2952–2962, Dec. 2005.
    [22] H. Kwon and N. M. Nasrabadi, “Kernel matched subspace detectors for hyperspectral target detection,” IEEE Trans. Pattern Anal. Machine Intell., vol. 28, no. 2, pp. 178–194, Feb. 2006.
    [23] F. Camastra and A. Verri, “A novel kernel method for clustering,” IEEE Trans. Pattern Analy. Machine Intell., vol. 27, no. 5, pp. 801–805, May 2005.
    [24] J. Peng, D. R. Heisterkamp, and H. K. Dai, “Adaptive quasiconformal kernel nearest neighbor classification,” IEEE Trans. Pattern Analy. Machine Intell., vol. 26, no. 5, pp. 656–661, May 2004.
    [25] L. Wang, K. L. Chan, and P. Xue, “A criterion for optimizing kernel parameters in KBDA for image retrieval,” IEEE Trans. Systems, Man, and Cybernetics-Part B: Cybernetics, vol. 35, no. 3, pp. 556–612, June 2005.
    [26] G. C. Valls, L. G. Chova, J. M. Marí, J. V. Francés, and J. C. Maravilla, “Composite kernels for hyperspectral image classification,” IEEE Trans. Geosci. Remote Sensing, vol. 3, no. 1, pp. 93–97, Jan. 2006.
    [27] S. K. Zhou and R. Chellappa, “From sample similarity to ensemble similarity: probabilistic distance measures in reproducing kernel Hilbert space,” IEEE Trans. Pattern Anal. Machine Intell., vol. 28, no. 6, pp. 917–929, Jun., 2006.
    [28] T. Jebara and R. Kondon, “Bhattarcharyya and expected likehood kernels,” Proc. Conf. Learning Theory (COLT), 2003.
    [29] J. W. Woods, “Markov image modeling,” IEEE Trans. Automatic Control, vol. AC-23, pp. 846-850, Oct. 1978.

    下載圖示 校內:2009-08-29公開
    校外:2009-08-29公開
    QR CODE