積分形式的概似函數常見於一般統計模型當中。在積分沒有解析解的情況下，其最大概似估計往往不易取得。傳統作法以數值積分方法近似其概似函數，在取對數後，針對目標參數求其數值微分，最後結合最佳化演算法進行參數估計。由於數值積分精確度及運算效率受維度影響極大，且最佳化演算法只允許確切的目標函數值，故傳統作法經常受限於高維度及較複雜的模型假設。本論文透過直接建立對數概似函數數值微分，以Metropolis-Hastings 演算法取得其不偏估計，結合同步擾動隨機近似，大幅減少運算負擔且在數值微分為估計值的情況下，迭代參數得以機率收斂到真實參數，發展出基於對數概似函數之參數估計方法。本論文以線性混和模型及混和效應羅吉斯迴歸為例，模擬結果顯示在兩個模型下，此方法都有不錯的表現。

Stochastic approximation is an optimization algorithm targeting at the likelihood functions which involve with integrations and have no closed-form representation. A special requirement of the stochastic approximation algorithm is that unbiased estimates of its objective function is needed. General practices of applying stochastic approximation face the dilemma of choosing likelihood or log-likelihood as its objective function. First, when using likelihood function, the unbiasness can be obtained easily by Monte-Carlo approaches. But the integrand, products of probabilities, can be too small to evaluate accurately. Therefore, underflow issue may happen. Second, when using log-likelihood function, the log-integration is difficult to be unbiasedly estimated, though the underflow issue is prevented. In this thesis, we construct a numerical derivative of log-likelihood directly, and apply Metropolis-Hastings algorithm to have its unbiased estimation. Hence, the dilemma no longer exists. We also take advantages of simultaneous perturbation stochastic approximation to alleviate the computational burden. We take linear mixed model and mixed effect logistic regression as examples. Simulation results show that this procedure performs well under both models.

Besag, J., Green, P., Higdon, D. andMengersen, K. (1995). Bayesian Computation and Stochastic Systems. Statistical Science, vol. 10, pp. 3-66.
Booth, J. G. and Hobert, J. P. (1999). Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm. Journal of Royal Statistical Society, Series B 61, 265–285.
Breslow, N. E. and Clayton, D. G. (1993). Approximate inference in generalized linear mixed models. Journal of the American Statistical Association 88, 9-25.
Breslow, N. E. and Lin, X. (1995). Bias correction in generalized linear mixed models with a single component of dispersion. Biometrika 82, 81-91.
Chang, S.-M., (2007). A Stationary Stochastic Approximation Algorithm for Estimation in the GLMM, Ph.D. dissertaion, North Carolina State University, Raleigh, NC.
Harville, D. A. (1974). Bayesian inference for variance components using only error contrasts. Biometrika 61, 383-385.
Jang, W. and Lim, J. (2006). PQL estimation biases in generalized linear mixed models. Institute of Statistics and Decision Sciences, Duke University, Durham, NC, USA., 05–21.
Kiefer, J. and Wolfowitz, J. (1952). Stochastic estimation of a regression function. Annals of Mathematical Statistics 23, 462–466.
Liu, Q. and Pierce, D. A. (1994). A note on Gauss-Hermite quadrature. Biometrika 81, 624-629.
Monahan, J. F. and Genz, A. (1997). Spherical-radial integration rules for Bayesian computation. Journal of the American Statistical Association 92, 664–674.
Press, W.H., Teukolsky, S.A., Vetterling, W.T. and Flannery, B.P. (1996). Numerical Recipes in Fortran 77: The Art of Scientific Computing. Cambridge University Press.
Robbins, H. and Monro, S. (1951). A stochastic approximation method. Annals of Mathematical Statistics 22, 400–407.
Spall, J. C. (1992). Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Transactions on Automatic Control 37, 332–341.
Spall, J. C. (2003). Introduction to Stochastic Search and Optimization: Estimation, Simulation and Control. Wiley, New York.
Wolfinger, R., Tobias, R., and Sall, J. (1994). Computing Gaussian likelihoods and their derivatives for general linear mixed model. SIAM Journal of Scientific Computation 15, 1294–1310.