在資料科學領域中，為使模型具有良好的性能（Performance），選取適當的超參數（Hyper-parameter）至關重要，而根據超參數的類型可分做類別型的定性因子（Qualitative Factors）及連續型的定量因子（Quantitative Factors）。本論文期望在同時具有上述兩類型因子的情境下，為多個受雜訊（Noise）影響的評估指標建立替代模型（Surrogate Model），了解因子與評估指標之間的關係，並將其作為多目標（Multiple Responses）的最佳化問題，探討如何在有限的計算成本下找出模型多項評估指標的柏拉圖集合（Pareto Set）及對應因子設計的非支配解集合（Non-dominated Solution Set）。整體實驗流程主要以（1）應用多輸出高斯過程模型（Multi-output Gaussian Process Model）；（2）依據多目標填充準則（Infill Criteria）進行序列設計（Sequential Design），透過上述兩步驟的迭代來優化當前所蒐集的柏拉圖集合。在數值實驗中以目標函數之間是否具有相關性設計兩種不同情境進行序列實驗，透過迭代的過程比較不同多輸出高斯過程模型以及填充準則在兩種情境下的優劣。而應用分析以卷積神經網路模型（Convolutional Neural Network）的超參數優化為例。本論文利用多任務之定性及定量型高斯過程模型（Multi-task QQGP）結合後驗估計之超容積期望改進量（Posterior-based Expected Hypervolume Imporvement）執行最佳化程序，在迭代的過程逐步找出神經網路模型於 Macro F1 及 Micro F1 兩項指標的柏拉圖集合。

This thesis mainly focuses on surrogate-assisted tuning procedures for qualitative and quantitative factors in multiple response models with noises. Basically, a surrogate-assistant approach iterates the following two steps until a stop criterion is met. First based on the current explored points, a surrogate surface is constructed and then due to the surrogate model, an infill criterion is adopted to identify the next explored point.
In this thesis, we treat the tuning problem as a multi-objective optimization problem. In order to efficiently construct the Pareto set via a surrogate-assistant approach, first a surrogate construction approach for multiple responses is introduced to deal with the qualitative and quantitative factors scenario and then the corresponding infill criterion is also modified. To illustrate the performance of this surrogate-assistant approach, two numerical experiments are illustrated due to the different correlations among the responses.
Here a tuning problem for a CNN model is studied. The two responses are Macro F1 and Micro F1 scores and we treated them as random responses. Overall tuning results show that the proposed surrogate-assistant approach can quickly identify the proper Pareto set.

Bonilla, E. V., Chai, K., & Williams, C. (2007). Multi-task Gaussian process prediction. Advances in neural information processing systems, 20.
Ba, S., Myers, W. R., & Brenneman, W. A. (2015). Optimal sliced Latin hypercube designs. Technometrics, 57(4), 479-487.
Cao, Y., Smucker, B. J., & Robinson, T. J. (2015). On using the hypervolume indicator to compare Pareto fronts: Applications to multi-criteria optimal experimental design. Journal of Statistical Planning and Inference, 160, 60-74.
Emmerich, M., & Klinkenberg, J. W. (2008). The computation of the expected improvement in dominated hypervolume of Pareto front approximations. Rapport technique, Leiden University, 34, 7-3.
Emmerich, M. T., Deutz, A. H., & Klinkenberg, J. W. (2011, June). Hypervolume-based expected improvement: Monotonicity properties and exact computation. In 2011 IEEE Congress of Evolutionary Computation (CEC) (pp. 2147-2154). IEEE.
Goulard, M., & Voltz, M. (1992). Linear coregionalization model: tools for estimation and choice of cross-variogram matrix. Mathematical Geology, 24(3), 269-286.
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778).
Jones, D. R., Schonlau, M., & Welch, W. J. (1998). Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13(4), 455-492.
Keane, A. J. (2006). Statistical improvement criteria for use in multiobjective design optimization. AIAA journal, 44(4), 879-891.
Liu, H., Cai, J., & Ong, Y. S. (2018). Remarks on multi-output Gaussian process regression. Knowledge-Based Systems, 144, 102-121.
Mwebaze, E., Gebru, T., Frome, A., Nsumba, S., & Tusubira, J. (2019). iCassava 2019 fine-grained visual categorization challenge. arXiv preprint arXiv:1908.02900.
Nguyen, T. V., & Bonilla, E. V. (2014, July). Collaborative Multi-output Gaussian Processes. In UAI (pp. 643-652).
Qian, P. Z. G., Wu, H., & Wu, C. J. (2008). Gaussian process models for computer experiments with qualitative and quantitative factors. Technometrics, 50(3), 383-396.
Sobester, A., Forrester, A., & Keane, A. (2008). Engineering design via surrogate modelling: a practical guide. John Wiley & Sons.
Swersky, K., Snoek, J., & Adams, R. P. (2013). Multi-task bayesian optimization. Advances in neural information processing systems, 26.