本篇嘗試將變分推論應用於混合迴歸模型上的變數選擇問題。本研究考量資料可能有多個迴歸模型組成，且每個迴歸模型重要的共變項會不一樣。變分推論亦是一種貝氏計算技巧，將後驗分布的近似問題轉換成最佳化問題，使變分分布與原本的後驗分布逼近，並以所得的近似機率分布作為後續推斷的依據。因為將其轉換成最佳化問題，在計算成本上有其優勢。為求結果的穩定性，以信息準則來選取最佳的超參數值。透過模擬與實際資料的運算結果，可以得到此方法表現優異的地方與其限制

In this thesis, the variable selection problem for finite mixture model of linear regression is considered. The mixture model contains more than two linear regression models, and the important variables for the different linear regression models are different. The Bayesian approach is proposed to identify the active variables for the mixture model. In the proposed Bayesian approach, there are two sets of indicators added into the model. The first set of indicators is the memberships of the observations and the second set of the indicators is used to denote whether the variables are important or not. Instead of the MCMC sampling procedure for indicators, a variational Bayes approach is adopted to get the approximation of the posterior density function for the future inference. In addition, based on information criteria, a tuning procedure for the hyper-parameters is also introduced. Finally, the performance of the proposed variational Bayes approach is illustrated via a series of the simulations and a real example.

[1] Christopher M Bishop. Pattern recognition and machine learning. springer, 2006.
[2] Marcin Błażejowski, Paweł Kufel, and Jacek Kwiatkowski. Model simplification and variable selection: A replication of the uk inflation model by hendry (2001). Journal of Applied Econometrics, 35(5):645–652, 2020.
[3] Peter Carbonetto, Matthew Stephens, et al. Scalable variational inference for bayesian variable selection in regression, and its accuracy in genetic association studies. Bayesian analysis, 7(1):73–108, 2012.
[4] David F Hendry. Modelling uk inflation, 1875–1991. Journal of applied econometrics, 16(3):255–275, 2001.
[5] X Huang, J Wang, and F Liang. A variational algorithm for bayesian variable selection. arxiv e­prints (2016). arXiv preprint arXiv:1602.07640.
[6] Ching­Kang Ing and Tze Leung Lai. A stepwise regression method and consistent model selection for high­dimensional sparse linear models. Statistica Sinica, pages 1473–1513, 2011.
[7] Abbas Khalili and Jiahua Chen. Variable selection in finite mixture of regression models. Journal of the american Statistical association, 102(479):1025–1038, 2007.
[8] Kuo­Jung Lee, Ray­Bing Chen, and Ying Nian Wu. Bayesian variable selection for finite mixture model of linear regressions. Computational Statistics & Data Analysis, 95:1–16, 2016.
[9] Prasad A Naik, Peide Shi, and Chih­Ling Tsai. Extending the akaike information criterion to mixture regression models. Journal of the American Statistical Association, 102(477):244–254, 2007.
[10] John T Ormerod, Chong You, Samuel Müller, et al. A variational bayes approach to variable selection. Electronic Journal of Statistics, 11(2):3549–3594, 2017.
[11] Russell J Steele and Adrian E Raftery. Performance of bayesian model selection criteria for gaussian mixture models. Frontiers of statistical decision making and bayesian analysis, 2:113–130, 2010.
[12] Michalis Titsias and Miguel Lázaro­Gredilla. Spike and slab variational inference for multi­task and multiple kernel learning. Advances in neural information processing systems, 24:2339–2347, 2011.
[13] Chun­Xia Zhang, Shuang Xu, and Jiang­She Zhang. A novel variational bayesian method for variable selection in logistic regression models. Computational Statistics & Data Analysis, 133:1–19, 2019.