在單一長回合(one long run) 的穩態模擬系統中，由於觀察值間幾乎不為獨
立同分配，若使用樣本變異數建立信賴區間會產生偏誤，於是許多學者使用不
同的方式來建立信賴區間，常見的方法為對觀察值進行無重疊分批，利用批次
的平均數及變異數搭配Student’s t 分配建立信賴區間。過去數十年中已有許多
估計穩態平均數的信賴區間程序被提出，但常用來判斷其優劣的指標卻只有實
現涵蓋率、信賴區間半長的期望值及變異數等等，然而這些都不是足夠明確的
指標，能讓使用者判斷各信賴區間程序的優劣。直到Yeh and Schmeiser (2015) 提
出VAMP1RE 指標，才能以單一數值的大小評估信賴區間程序的優劣，但其被限
制在只能應用於固定樣本數的信賴區間程序。
本研究提出通用性VAMP1RE，此指標在運行時不需事先設定信賴區間程序的
樣本數，而是能以其他隨機的停止條件做為給定之參數，同時保留VAMP1RE 指
標在固定樣本數時的基礎與優點。本研究同時提出如何使用蒙地卡羅模擬來
得到通用性VAMP1RE 估計值的步驟，並使用估計穩態平均數的信賴區間程
序ASAP3 與Skart為例計算通用性VAMP1RE的估計值。
經由AR(1) 與M/M/1 實驗情境的模擬後，發現Skart 在AR(1) 的情境下能
相對有效率的使用樣本的資訊，因此能得到比較小的通用性VAMP1RE 值；
而ASAP3 在M/M/1 的實驗情境下所計算的涵蓋值誤差相對Skart 來的小，因此能
得到比較小的通用性VAMP1RE 值，讓使用者能根據情境選擇比較好的信賴區間
程序進行估計。

A Confidence-Interval Procedure (CIP) is given with a random data set and then
an interval with a specified nominal coverage probability is determined. Up to the
present time, many CIPs have been proposed, where most were designed to estimate the steady-state mean. Traditionally, the criterion for evaluating CIPs are the expected interval half-length, variance of the interval half-length, the sample size used by the CIP, or the achieved coverage probability. However, with these criteria, users have to choose a better CIP through multiple dimensions, and obviously, this process is unclear. We propose a generalized VAMP1RE for rating and ranking CIPs that uses a random sample size based on a single criterion. The generalized VAMP1RE focuses on two factors that may influence CIP performance: departure from validity and inability to mimic. It compares those CIPs using a random sample size with an ideal CIP that is both valid and based on an agreed-upon standard. For a given CIP, the generalized VAMP1RE criterion is the expected squared difference of the Schruben's coverage values which calculate separately between the given CIP and the ideal CIP, where a smaller value means the CIP has a better performance. In the experiment, we use Tafazzoli and Wilsons' Skart,
Steiger and colleagues' ASAP3 as examples to evaluate their performance. In the AR(1) test situation, the data used by Skart is more efficiently. Therefore, it has a better performance. In the M/M/1 test situation ASAP3 can calculate the coverage value with fewer errors, and it obtains a smaller generalized VAMP1RE value. We let the user decide the better CIP under different test situations by using the generalized VAMP1RE.

[1] Alexopoulos, C., Goldsman, D., Mokashi, A. C., Tien, K. W., & Wilson, J. R.
(2019). Sequest: A sequential procedure for estimating quantiles in steady-state simulations. Operations Research, 67(4), 1162-1183.
[2] Alexopoulos, C., Goldsman, D., Mokashi, A. C., &Wilson, J. R. (2017). Automated estimation of extreme steady-state quantiles via the maximum transformation. ACM Transactions on Modeling and Computer Simulation (TOMACS), 27(4), 1-29.
[3] Box, G. E. P., Jenkins, G. M., & Reinsel, G. C. (2008). Time series analysis: forecasting and control. 4th Ed. John Wiley & Sons.
[4] Fishman, G. S., & Yarberry, L. S. (1997). An implementation of the batch means
method. INFORMS Journal on Computing, 9(3), 296-310.
[5] Kang, K., & Schmeiser, B. (1990). Graphical methods for evaluating and comparing confidence-interval procedures. Operations Research, 38(3), 546-553.
[6] Maillard, D. (2018). A user’s guide to the Cornish Fisher expansion. Available at
SSRN 1997178.
[7] Moran, P.A.P., (1975). The estimation of standard errors in Monte Carlo simulation experiments. Biometrika, 62(1), 1-4.
[8] Royston, J. P. (1982a). An extension of Shapiro and Wilk’s W test for normality to large samples. Applied Statistics 31 (2), 115-124. 37
[9] Sargent, R.G., Kang, K., & Goldsman, D. (1992). An investigation of finite-sample behavior of confidence interval estimators. Operations Research, 40(5), 898-913.
[10] Schmeiser, B. W., & Scott, M. D. (1991). SERVO: Simulation experiments with
random-vector output. In Proceedings of the 1991 Winter Simulation Conference, (pp. 927-936). Institude of Electrical and Electronics Engineers.
[11] Schmiser, B., & Tina Song, W. M. (1989). Inverse-transformation algorithms for
some common stochastic processes. Proceedings of the 1989Winter simulation Conference, (pp. 490-496). Institude of Electrical and Electronics Engineers.
[12] Schmeiser, B., & Yeh, Y. (2002). On choosing a single criterion for confidenceinterval procedures. In Proceedings of the Winter Simulation Conference, (pp.345-352). Institude of Electrical and Electronics Engineers.
[13] Schruben, L. W. (1980). A coverage function for interval estimators of simulation response. Management Science, 26(1), 18-27.
[14] Shapiro, S. S., & Wilk, M. B. (1965). An analysis of variance test for normality
(complete samples). Biometrika, 52(3/4), 591-611.
[15] Song, W. T., Chih, M., & Chuang, S. (2012). Variance reduction techniques on
generating M/M/1 processes in simulation output analysis. IEEE Transactions on
Automatic Control, 58(4), 1035-1040.
[16] Song, W. T., & Chih, M. (2013). Run length not required: Optimal-mse dynamic
batch means estimators for steady-state simulations. European Journal of Operational
Research, 229(1), 114-123. 38
[17] Steiger, N. M., Lada, E. K., Wilson, J. R., Alexopoulos, C., Goldsman, D., &
Zouaoui, F. (2002). ASAP2: An improved batch means procedure for simulation
output analysis. In Proceedings of the Winter Simulation Conference, (pp. 336-344).
Institude of Electrical and Electronics Engineers.
[18] Steiger, N. M., & Wilson, J. R. (2002a). An improved batch means procedure for
simulation output analysis. Management Science, 48(12), 1569-1586.
[19] Steiger, N. M., Lada, E. K., Wilson, J. R., Joines, J. A., Alexopoulos, C., & Goldsman, D. (2005). ASAP3: A batch means procedure for steady-state simulation analysis. ACM Transactions on Modeling and Computer Simulation (TOMACS), 15(1), 39-73.
[20] Tafazzoli, A., & Wilson, J. R. (2010). Skart: A skewness-and autoregressionadjusted batch-means procedure for simulation analysis. IIE Transactions, 43(2), 110-128.
[21] Von Neumann, J. (1941). Distribution of the ratio of the mean square successive
difference to the variance. The Annals of Mathematical Statistics, 12(4), 367-395.
[22] Yeh, Y., & Schmeiser, B. W. (2015). VAMP1RE: a single criterion for rating and ranking confidence-interval procedures. IIE Transactions, 47(11), 1203-1216.