現代局部最佳化的理論自從二十世紀以來已經極為成熟，而且許多光學商業軟體也都將最佳化應用在系統上面，而使用導數的最佳化又是最常被應用，也最準確的最佳化方法，因此本文會著重於介紹導數局部最佳化的理論，及其在光學系統的有效焦距和像差上的應用。因此在第二章會先介紹評價函數，引出評價函數的導數，利用導數建構Jacobian和Hessian矩陣，並且利用評價函數的梯度的概念，來進行最佳化的理論與其應用。在第三章介紹各種局部最佳化的方法，包括共軛梯度法、信賴領域法、阻滯最小平方法等等。介紹完理論之後，第四章則使用網路的最佳化函式庫和光學軟體來模擬光學系統，對單一透鏡、佩茲瓦爾透鏡和庫克三分離式透鏡，測試各種最佳化的方法、變數起點和權重改變，會對於最佳化的過程造成哪些影響。在確認變數起點的改變會影響局部最佳化的結果之後，在第五章再透過隨機改變起點方法，和象限變動起點方法，來探索整個系統中可能存在的最小值，並且證明象限變動起點方法在評價函數的象限邊界存在極大阻礙的時候，在收斂速度上比完全隨機改變起點更快，甚至在尋找所有的全域最小值上面，象限變動起點方法也更為詳盡，證明象限變動起點方法有其應用價值。

This thesis introduces modern optimization by starting from the construction of the merit function, Jacobian and Hessian matrices, and further introduce theories and application of optimization using gradients. After introducing the theories of gradient optimization, we then use open source libraries and optical software to simulate the optimization of optical systems. We will test changes in optimization process by optimizing single lens, Petzval lens, and Cooke triplet. The difference in methods, variables’ starting points and weight will have a huge impact on the optimization process. After confirming that the change in the variable starting point will affect the results of local optimization, we then use the method of randomly changing the starting point, and the quadrant changing starting point method to explore the minimum value that may exist in the entire system, and it is proved that the quadrant changing starting point method has its value when there are great obstacles at the quadrant boundary of the merit function.

[1] Joseph Raphson, Analysis Aequationum Universalis, Nabu Press, (1697).
[2] Carl Friedrich Gauss, Theoria Motus Corporum Coelestium in Sectionibus Conicis Solem Ambientium, Cambridge University Press, (1809).
[3] A. Cauchy, “Methode generale pour la resolution des systemes d'equations simultanees,” C.R. Acad. Sci., Paris, Vol. 25, pp.536–538, (1847).
[4] Hestenes, Magnus R., and Stiefel, Eduard, “Methods of Conjugate Gradients for Solving Linear Systems,” Journal of research of the National Bureau of Standards, Vol. 49, pp. 409-435, (1952).
[5] W. C. Davidon, “Variable Metric Method for Minimization”. SIAM Journal on Optimization, Vol. 1, No.1, pp. 1-17, (1959).
[6] C. G. Broyden, “The convergence of a class of double-rank minimization algorithms”, Journal of the Institute of Mathematics and Its Applications, Vol. 6, pp. 76-90, (1970).
[7] K. Levenberg, “A method for the solution of certain non-linear problems in least squares,” Quarterly of Applied Mathematics, Vol. 2, pp. 164-168., (1944).
[8] D. W. Marquardt, “An algorithm for least-squares estimation of nonlinear parameters,” Journal of SIAM, Vol. 11, pp.431-441, (1963).
[9] R. Fletcher, Practical Methods of Optimization (second revised edition), pp. 95-109, Wiley, (2000).
[10] Sorensen D. C., “Newton's Method with a Model Trust Region Modification,” SIAM J. N. Anal., Vol. 19, No.2, pp. 409–426, (1982).
[11] M. J. D. Powell, “A new algorithm for unconstrained optimization,” Nonlinear Programming. Academic Press, pp. 31–66, (1970).
[12] Nelder, J. A., and R. Mead, “A Simplex Method for Function Minimization,” The Computer Journal, Vol. 7, pp. 308-313. (1965).
[13] M. J. D. Powell,. “An efficient method for finding the minimum of a function of several variables without calculating derivatives,” The Computer Journal, Vol. 7, pp. 155-162, (1964).
[14] Rios L.M., and Sahinidis N.V., “Derivative-free optimization: A review of algorithms and comparison of software implementations,” J. Glob. Optim., volume 56, pages 1247–1293, (2013).
[15] Storn R., and Price K., “Differential Evolution - a Simple and Efficient Heuristic for Global Optimization over Continuous Spaces,” Journal of Global Optimization, vol. 11, pp. 341-359, (1997).
[16] Constantino Tsallis, and Daniel A. Stariolo, “Generalized Simulated Annealing,” Physica A, Vol. 233, pp. 395-406, (1996).
[17] Wales D. J., and Doye J. P. K., “Global Optimization by Basin-Hopping and the Lowest Energy Structures of Lennard-Jones Clusters Containing up to 110 Atoms,” Journal of Physical Chemistry A, Vol. 101, pp. 5111, (1997).
[18] Seidel, L., “Zur Dioptrik, von Ludwig Seidel,” Astronomische Nachrichten, vol. 37, no. 8, pp. 105, (1853).
[19] 王奕鈞。軸對稱光學系統的波前像差轉換至光線像差之問題探討。國立成功大學機械工程學系碩士學位論文。取自https://hdl.handle.net/11296/37tz2d
[20] 陳廷碩。軸對稱光學系統的主要光線像差與波前像差之研究。國立成功大學機械工程學系碩士學位論文。取自https://hdl.handle.net/11296/3nbq5e
[21] Welford W.T., Aberrations of Optical Systems (1st ed.), Routledge, Chap.3, pp.23-25, (1986).
[22] Grievenkamp John E., “Field Guide to Geometrical Optics”, SPIE Press, pp. 6–9, (2004).
[23] F. Bociort, Optical System Optimization, Encyclopedia of Optical Engineering, Marcel Dekker, New York, pp.1843-1850, (2003).
[24] Juan L. Rayce, and Martha Rosete-Agui1ar, “A discontinuity in the constrained damped least squares (DLS) method of optimization,” Proceedings of the SPIE, Vol.5524, pp.250-260, (2004).
[25] Maarten van Turnhout, “A systematic analysis of the optical merit function landscape: towards improved optimization methods in optical design,” Applied Sciences, (2009).
[26] Joseph M. Geary, “Introduction to Lens Design with Practical ZEMAX examples,” Willmann-Bell Inc, Richmond, Verginia, ch.11.2, pp.117-118, (2002).
[27] Michael J. Kidger, “Use of the Levenberg-Marquardt (damped least-squares) optimization method in lens design,” Optical Engineering 32(08), pp.1731-1739, (1993).
[28] Jorge Nocedal, and Stephen J. Wright, Numerical Optimization, Springer-Verlag Inc., New York, Ch. 2.1, pp.13-17, (2000).
[29] James Stewart, Multivariable Calculus: Concepts & Contexts, Brooks/Cole, ch.11.7, p.801-803, (2005).
[30] Steven Leon, and Lisette de Pillis, Linear Algebra with Applications(10th edition), Pearson, ch.6.6, p.388-390, (2020).
[31] Yu-Hong Dai, “Alternate step gradient method,” A Journal of Mathematical Programming and Operations Research, 52:4-5, p. 395-415, (2003).
[32] Hestenes, M.R. and Stiefel, E. “Methods of Conjugate Gradients for Solving Linear Systems,” Journal of Research of the National Bureau of Standards, Vol. 49, p. 409-435, (1952).
[33] Jonathan R. Shewchuk, “An Introduction to the Conjugate Gradient Method Without the Agonizing Pain,” Carnegie Mellon University, United States, ch.7, p. 29-32, (1994).
[34] Michael J. Hayford Michael J. Hayford, "Optimization Methodology," Proc. SPIE Geometrical Optics, Vol. 0531, p. 68-81, (1985).
[35] K. Madsen, H.B. Nielsen, O. Tingleff, “Methods For Non-Linear Least Squares Problems (2nd ed.),” IMM, Denmark, ch.3, p.20-26 (2004).
[36] W. H. Swann, “A survey of non-linear optimization techniques,” FEBS Letters, Volume 2, Supplement 1, Pages S39-S55, (1969)
[37] Wolfe, P., “Convergence Conditions for Ascent Methods,” SIAM Review, vol.11 (2), pp.226–235, (1969)
[38] Wolfe, P., “Convergence Conditions for Ascent Methods. II: Some Corrections,” SIAM Review, vol.13 (2), pp.185–188, (1971)
[39] H. B. Nielsen, “Damping Parameter in Marquardt's Method,” Informatics and Mathematical Modelling, Technical University of Denmark, (1999)
[40] Maarten van Turnhout, and Florian Bociort, "Chaotic behavior in an algorithm to escape from poor local minima in lens design," Opt. Express 17, 6436-6450 (2009).
[41] Virtanen, P. et al., “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python”, Nature Methods, 17, pp.261–272. (2020).
[42] Kramer Harrison, Optiland (python source code), GitHub, (2017).
[43] More, J. J., “The Levenberg-Marquardt Algorithm: Implementation and Theory,” Numerical Analysis, ed. G. A. Watson, vol 630, (1978).
[44] Branch, M.A., T.F. Coleman, and Y. Li, "A Subspace, Interior, and Conjugate Gradient Method for Large-Scale Bound-Constrained Minimization Problems," SIAM Journal on Scientific Computing, Vol. 21, Number 1, pp 1-23, (1999).
[45] Conn, A. R., Gould, N. I., and Toint, P. L., Trust region methods, Siam, pp.169-200, (2000).
[46] Michael Bass, “Handbook of Optics: Volume II - Design, Fabrication, and Testing; Sources and Detectors; Radiometry and Photometry, 3rd Edition”, chap2.3, (2010).
[47] Psang Dain Lin, “Advanced Geometrical Optics.” Springer, (2016).