簡易檢索 / 詳目顯示

回結果列表
研究生： 許志遠
Hsu, Chih-Yuan
論文名稱： 多維度核密度函數偏導數之帶寬選擇與密度函數的眾數估計
Bandwidth selectors for kernel estimation of density partial derivatives and mode estimates of multivariate data
指導教授： 吳鐵肩
Wu, Tiee-Jian
學位類別： 博士
Doctor
系所名稱： 管理學院 - 統計學系
Department of Statistics
論文出版年： 2013
畢業學年度： 101
語文別： 英文
論文頁數： 88
中文關鍵詞： 密度函數估計 、交叉確認 、密度函數的偏導數 、加權平均 、半參數法
外文關鍵詞： Density estimation, cross-validation, density partial derivatives, weighted average, semi-parametric approach
相關次數： 點閱：108  下載：4 
分享至:
查詢本校圖書館目錄 查詢臺灣博碩士論文知識加值系統 勘誤回報

    • 假設我們觀察到一組樣本數為n的多維資料，母體分佈未知。在本論文中，有兩個主題被研究：一是多維度核密度函數偏導數之帶寬選擇，二是多維密度函數的眾數估計。在第一個主題中，我們提出的估計量具有漸近常態、根號n的收斂速度與變異數達到推測的訊息下界(information bound)等大樣本性質。而在第二個主題中，我們提出兩個眾數估計量。首先，我們推廣Bickel (2003) 的方法並且應用多維的Box-Cox 變換，得到我們的第一個估計量。我們發現它在小樣本的表現尤其傑出。然而，也發現它可能有不一致性(non-consistent)的問題產生。為了改善這個問題，我們使用了有母數(parametric)密度函數估計與無母數(nonparametric)密度函數估計加權平均的方法。結果證實，第二個估計量不但保有小樣本的優良表現並且也成功的克服在大樣本時不一致的缺點。

    Based on a random sample of size n from an unknown multivariate density f, two research topics are investigated: (i) bandwidth selection in multivariate kernel density partial derivatives, and (ii) mode estimation of a multivariate density. On the topic (i), a bandwidth selector is proposed, which extends the ones of Wu (1997) and Wu and Tsai (2004). The bandwidth selector is asymptotically normal with the optimal root n relative convergence rate and achieves the (conjectured) “lower bound” on the covariance matrix. On the topic (ii), two mode estimates are proposed. The first one is a multivariate extension to the one of Bickel (2003), which is an application of the joint Box-Cox transform. Also, we show that the estimate is quite efficient when the sample size n is relatively small. However, the first one may not be consistent due to the restriction of the transform method. To solve the non-consistent problem, the second estimate is proposed, which is based on a weighted average of a parametric density estimate and a nonparametric density estimate. It is shown that the estimate not only keeps the strengths of the first one at small n but also overcomes the non-consistent drawback at large n.

    1 Introduction 1 2 Bandwidth Selection in Multivariate Kernel Density Partial Derivatives 3 2.1 Introduction to Bandwidth Selection . . . . . . . . . . . . . . . . . . . 3 2.2 The Proposed method . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.2.1 The Source of Variation . . . . . . . . . . . . . . . . . . . . . . 6 2.2.2 The Proposed Bandwidth Selector . . . . . . . . . . . . . . . . . 8 2.2.3 The Theoretical Results . . . . . . . . . . . . . . . . . . . . . . 10 2.2.4 The Modification of the Proposed Bandwidth Selector . . . . . . 11 2.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 3 Mode Estimation of a Multivariate Density 16 3.1 Introduction to Mode Estimation . . . . . . . . . . . . . . . . . . . . . 16 3.2 An Efficient Mode Estimate . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2.1 The Proposed Estimator . . . . . . . . . . . . . . . . . . . . . . 19 3.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 3.3.1 Simulation Study for i.i.d. Data from a Unimodal Density . . . 23 3.3.2 Sensitivity to Multimodalities . . . . . . . . . . . . . . . . . . . 24 3.3.3 Simulation Study for Dependent Data . . . . . . . . . . . . . . 25 3.3.4 Simulation Study for Functional Data . . . . . . . . . . . . . . . 26 3.3.5 Real Data Applications . . . . . . . . . . . . . . . . . . . . . . . 28 3.4 Mode Estimate by a Semi-Parametric Method . . . . . . . . . . . . . . 30 3.4.1 The Proposed Estimator . . . . . . . . . . . . . . . . . . . . . . 30 3.4.2 Theoretical Results . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.4.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . 34 4 Conclusions 37 5 Proofs 38 5.1 Theorem 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 5.2 Theorem 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 5.3 Theorems 3 and 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Appendix A Tables and Figures 58 Bibliography 85 List of Tables 2.1 Table 1. Parameters for eight example normal mixture densities. . . . . . . 14 3.1 Table 2. Parameters for six example bivariate unimodal densities. . . . . . 24 3.2 Table 3. Parameters for four example bivariate multimodal densities. . . . 24 3.3 Table 4. Parameters for six example univariate densities. . . . . . . . . . . 36 3.4 Table 5. Parameters for six example bivariate densities. . . . . . . . . . . 36 A.1 Tables A1 - A8. Simulation Results for Estimating fm when the bandwidth in each coordinate direction is the same. . . . . . . . . . . . . . . . . . . . 58 A.2 Tables B1 - B8. Simulation Results for Estimating fm when the bandwidth in each coordinate direction varies freely. . . . . . . . . . . . . . . . . . . . 66 A.3 Table C1. MSE of mode estimate for the six example bivariate unimodal densities. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 A.4 Table C2. MSE of mode estimate for the four example multimodal densities. 74 A.5 Table C3. MSE of mode estimate for the four example ARMA processes with  = (8; 8). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 A.6 Table C4. Mean L2-norm error of functional mode estimate. . . . . . . . . 75 A.7 Table C5. The relative frequency (in %) of correct recognition out of approximately 350 instances for each digit from the test set. . . . . . . . . . 75 A.8 Tables D1 and D2. MSE of mode estimate for the twelve densities #1 - #12 defined in Tables 3 and 4. . . . . . . . . . . . . . . . . . . . . . . . . 76 List of Figures A.1 Figure 1. Contour plots of the bivariate densities #1 - #6 de ned in Table 1. . 78 A.2 Figure 2. Contour plots of the bivariate unimodal densities #1 - #6 de ned in Table 2. The solid circle indicates the location of the mode. . . . . . . . . . . . . 79 A.3 Figure 3. Contour plots of the bivariate multimodal densities #7 - #10 de ned in Table 3. The solid circle and solid triangle indicate the locations of the major mode and minor mode(s), respectively. . . . . . . . . . . . . . . . . . . . . . . . . 80 A.4 Figure 4. The scatter plot of climatology data (gray open circles), the four modes A, B, C, D by Corti et al. (1999) (open triangles), the two best candidates for modes by Burman and Polonik (2009) (solid stars); and the mode estimates ˆmT (solid circle), ˆmF (solid square), ˆmM (solid triangle), ˆmP (solid diamond) and ˆmS (plus sign). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 A.5 Figure 5. Results using the training set. The top row shows the (approximately) 750 poly-lines for each digit. The 2nd to the 6th rows show the mode estimates ˆmT , ˆmF , ˆmM, ˆmP and ˆmS, respectively, while the 8 points (solid dots) constituting a mode estimate are connected sequentially to visualize an estimated digit. . . . . . . 82 A.6 Figure 6. Density plots of the univariate densities #1 - #6 de ned in Table 3. . . 83 A.7 Figure 7. Contour plots of the bivariate densities #7 - #12 de ned in Table 4. The solid circle and solid triangle indicate the locations of the major mode and minor mode(s), respectively. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

    Abraham, C., Biau, G. & Cadre, B. (2003). Simple estimation of the mode of a multivariate density.
    The Canadian Journal of Statistics, 31, 23-34.
    Alimoglu, F. & Alpaydin, E. (1996). Methods of combining multiple classifiers based on different representations
    for pen-based handwritten digit recognition. In: Proceedings of the Fifth Turkish
    Arti cial Intelligence and Arti cial Neural Networks Symposium, TAINN 96.
    Andrews, D.F., Gnanadesikan, R. & Warner, J. L. (1971). Transformations of multivariate data.
    Biometrics, 27, 825-840.
    Bickel, D.R. (2003). Robust and efficient estimation of the mode of continuous data: the mode as
    a viable measure of central tendency. Journal of Statistical Computation and Simulation, 73,
    889-912.
    Bickel, D.R. & Fr¨uhwirth, R. (2006). On a fast, robust estimator of the mode: comparisons to
    other robust estimators with applications. Computational Statistics and Data Analysis, 50,
    3500-3530.
    Box, G. E. P. & Tiao, G. C. (1992). Bayesian inference in statistical analysis, John Wiley and Sons,
    New York.
    Burman, P. & Polonik, W. (2009). Multivariate mode hunting: Data analytic tools with measures of
    significance. Journal of Multivariate Analysis, 100, 1198-1218.
    Chiu, S.T. (1991). Bandwidth Selection for Kernel Density Estimation. The Annals of Statistics, 19,
    1883-1905.
    Chiu, S.T. (1992). An automatic Bandwidth Selector for Kernel Density. Biometrika, 79, 771-782.
    Chac´on J.E. and Duong, T. (2013). Data-driven density derivative estimation, with applications to
    nonparametric clustering and bump hunting. Electronic Journal of Statistics, 7, 499-532.
    Cheng, M.Y. & Hall, P. (1998). On mode testing and empirical approximations to distributions.
    Statistics and Probability Letters, 39, 245-254.
    Comaniciu, D. (2003). An algorithm for data-driven bandwidth selection. IEEE Transactions on
    Pattern Analysis and Machine Intelligence, 25, 281-288.
    Comaniciu, D. & Meer, P. (2002). Mean Shift: A robust approach toward feature space analysis.
    IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 603-619.
    Corti, S., Molteni F. & Palmer P.N. (1999). Signature of recent climate change in frequencies of
    natural atmospheric circulation regimes. Nature, 398, 799-802.
    Devroye L.P. & Wagner T.J. (1980). The strong uniform consistency of kernel density estimates.
    Multivariate Analysis V, ed. P. R. Krishnaiah, New York: North-Holland, 59-77.
    Dobrovidov, A.V. & Rud’ko, I.M. (2010). Bandwidth selection in nonparametric estimator of density
    derivative by smoothed cross-validation method. Autom. Remote Control, 71, 209-224.
    Doukhan, P. (1994). Mixing: Properties and Examples. In: Lecture Notes in Statistics, 85. Springer-
    Verlag, New York.
    Duong, T., Cowling, A., Koch, I. & Wand, M.P. (2008). Feature significance for multivariate kernel
    density estimation. Computational Statistics and Data Analysis, 52, 4225-4242.
    Eddy, W.F. (1980). Optimum kernel estimators of the mode. The Annals of Statistics, 8, 870-882.
    Feller, W. (1968). An Introduction to Probability Theory and Its Application, 2nd ed. John Wiley &
    Sons, Inc.
    Ferraty, F. & Vieu, P. (2006). Nonparametric Functional Data Analysis: Theory and Practice, New
    York: Springer.
    Gasser, T., Hall, P. & Brett, P. (1998). Nonparametric estimation of the mode of a distribution of
    random curves. Journal of the Royal Statistical Society, B, 60, 681-691.
    Godtliebsen, F., Marron J.S. & Chaudhuri P. (2002). Significance in Scale Space for Bivariate Density
    Estimation. Journal of Computational and Graphical Statistics, 11, 1-21.
    Griffin, L.D & Lillholm, M. (2003). Mode estimation by pessimistic scale space tracking. In: Scale
    Space ’03. Isle of Skye. Springer, UK.
    Griffin, L.D. & Lillholm, M. (2005). A multiscale mean shift algorithm for mode estimation. IEEE
    Transactions on Pattern Analysis and Machine Intelligence (submitted for publication).
    Hall, P. (1983). Large sample optimality of least squares cross-validation in density estimation. The
    Annals of Statistics, 11, 1156-1174.
    Hall, P. (1984). Central limit theorem for integrated square error of multivariate nonparametric
    density estimators. Journal of Multivariate Analysis, 14, 1-16.
    Hall, P. & Heckman, N.E. (2002). Estimating and depicting the structure of a distribution of random
    functions. Biometrika, 89, 145-158.
    Hall, P. & Marron, J.S. (1991a). Lower bounds for bandwidth selection in density estimation. Prob-
    ability Theory and Related Fields, 90, 149-173.
    Hall, P. & Marron, J.S. (1991b). Local minima in cross-validation functions. Journal of the Royal
    Statistical Society, Ser.B, 53, 245-252.
    Hall, P., Sheather, S.J., Jones, M.C. & Marron, J.S. (1991). On Optimal Data-Based Bandwidth
    Selection in Kernel Density Estimation. Biometrika, 78, 521-530.
    H¨ardle, W., Marron, J.S. & Wand, M.P. (1990) Bandwidth Choice for Density Derivatives. Journal
    of the Royal Statistical Society, Ser.B, 52, 223-232.
    Hedges, S.B. & Shah P. (2003). Comparison of mode estimation methods and application in molecular
    clock analysis. BMC Bioinformatics, 4, 31.
    Hernandez, F. & Johnson, R.A. (1980). The large-sample behavior of transformations to normality.
    Journal of the American Statistical Association, 75, 855-861.
    Hewitt, E. & Stromberg, K. (1969). Real and Abstract Analysis. New York: Springer.
    Horov´a, I. & Vopatov´a, K. (2011). Kernel density gradient estimate. In Recent Advances in Func-
    tional Data Analysis and Related Topics (ed F. Ferraty), Physica Verlag, Heidelberg, 177-182.
    Janssen, P., Marron, J.S., Veraverbeke, N. & Sarle, W. (1995). Scale measures for bandwidth selection.
    Journal of Nonparametric Statistics, 5, 359-380.
    Jing, J., Koch, I. & Naito, K. (2012). Polynomial histograms for multivariate density and mode
    estimation. Scandinavian Journal of Statistics, 39, 75-96.
    Jones, M.C. (1991). The role of ISE and MISE in density estimation. Statistics and Probability
    Letters, 12, 51-56.
    Klemel¨a, J. (2005). Adaptive estimation of the mode of a multivariate density. Journal of Nonpara-
    metric Statistics, 17, 83-105.
    Liu, X. & Muller, H.G. (2003). Modes and clustering for time-warped gene expression profile data.
    Bioinformatics, 19, 1937-1944.
    Mardia, K.V. (1962). Multivariate Pareto distributions. Annals of Mathematical Statistics, 33, 1008-
    1015.
    Marchette, D.J. & Wegman, E.J. (1997). The filtered mode tree. Journal of Computational and
    Graphical Statistics, 6, 143-159.
    Marron, J.S. (1993). Discussion of: “Practical performance of several data driven bandwidth selectors”
    by Park and Turlach. Comput. Statist., 8, 17-19.
    Mckay, A.T. (1934). Sampling from batches. Journal of the Royal Statistical Society (Supplement 1),
    207-216.
    Minnotte, M.C., Marchette, D.J. & Wegman, E.J. (1998). The bumpy road to the mode forest.
    Journal of Computational and Graphical Statistics, 7, 239-251.
    Mokkadem, A. (1988). Mixing properties of ARMA processes. Stochastic Processes and their Appli-
    cations, 29, 309-315.
    Olkin I. & Speiegelman C.H. (1987). A semiparametric approach to density estimation. Journal of
    the American Statistical Association, 82, 858-865.
    Ooi, H. (2002). Density visualization and mode hunting using trees. Journal of Computational and
    Graphical Statistics, 11, 328-347.
    Parzen, E. (1962). On estimation of a probability density function and mode. The Annals of Math-
    ematical Statistics, 33, 1065-1076.
    Quintela-del-R´ıo, A. & Vieu, P. (1997). A nonparametric conditional mode estimate. Journal of
    Nonparametric Statistics, 8, 253-266.
    Ramsay, J.O. & Silverman, B.W. (1997). Functional Data Analysis. Springer, New York.
    Roberson, T.J. & Cryer, J.D. (1974). An iterative procedure for estimating the mode. Journal of the
    American Statistical Association, 69, 1012-1016.
    Sager, T.W. (1979). An iterative method for estimating a multivariate mode and isopleth. Journal
    of the American Statistical Association, 74, 329-339.
    Sain, S.R., Baggerly, K.A. & Scott, D.W. (1994). Cross-Validation of Multivariate Densities. Journal
    of the American Statistical Association, 89, 807-817.
    Sancetta, A. (2013). Weak conditions for shrinking multivariate nonparametric. Journal of Multi-
    variate Analysis, 115, 285-300.
    Scott, D.W. (1992). Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley,
    New York.
    Sheather, S.J. & Jones M.C. (1991). A reliable data-based bandwidth selection method for kernel
    density estimation. Journal of the Royal Statistical Society, B, 683-690.
    Silverman, B.W. (1981). Using kernel density estimates to investigate multimodality. Journal of the
    Royal Statistical Society, B, 43, 97-99.
    Silverman, B.W. (1986). Density Estimation for Statistics and Data Analysis, London: Chapman
    and Hall.
    Silverstein, J.W. & Bai, Z.D. (1995). On the empirical distribution of eigenvalues of a class of large
    dimensional random matrices. Journal of Multivariate Analysis, 54, 175-192.
    Vieu, P. (1996). A note on density mode estimation. Statistics and Probability Letters, 26, 297-307.
    Wand, M.P. (1992). Error analysis for general multivariate kernel estimators. Nonparametric Statist,
    2, 1-15.
    Wand, M.P. & Jones, M.C. (1993). Comparison of smoothing parameterizations in bivariate kernel
    density estimation. Journal of the American Statistical Association, 88, 520-528.
    Wand, M.P. & Jones, M.C. (1995). Kernel Smoothing. Chapman & Hall.
    Wu, T.-J. (1997). Root n bandwidth selectors for kernel estimation of density derivatives. Journal
    of the American Statistical Association, 92, 536-547.
    Wu, T.-J., Hsu, C.-H., Chen, H.-Y. & Yu, H.-C. (2013). Root n estimates of vectors of integrated
    density partial derivative functionals. Annals of the Institute of Statistical Mathematics, submitted.
    Wu, T.-J. & Lin, Y. (2000). Information Bound for bandwidth selection in kernel estimation of
    density derivatives. Statistica Sinica, 10, 457-473.
    Wu, T.-J. & Tsai, M.H. (2004). Root n bandwidth selectors in multivariate kernel density estimation.
    Probability Theory and Related Fields, 129, 537-558.
    Yang, L. & Tschernig, R. (1999). Multivariate bandwidth selection for local linear regression. Journal
    of the Royal Statistical Society, B, 61, 793-815.

    下載圖示 校內：2017-07-22公開
    校外：2018-07-22公開
    QR CODE