Abstract
Selecting an appropriate number of replications to run with a simulation model is important in assuring that the desired accuracy and precision of the results are attained with minimal effort. If too few are selected then accuracy and precision are lost; if too many are selected then valuable time is wasted. Given that simulation is often used by non-specialists, it seems important to provide guidance on the number of replications required with a model. In this paper an algorithm for automatically selecting the number of replications is described. Test results show the algorithm to be effective in obtaining coverage of the expected mean at a given level of precision and in reducing the bias in the estimate of the mean of the simulation output. The algorithm is consistent in selecting the expected number of replications required for a model output.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Alexopoulos C and Goldsman D (2004). To batch or not to batch? ACM Transactions on Modeling and Computer Simulation 14(1): 76–114.
Anscombe FJ (1953). Sequential estimation. J Roy Stat Soc B Met 15(1): 1–29.
Balci O (1994). Validation, verification, and testing techniques throughout the life cycle of a simulation study. Ann Opns Res 53: 121–173.
Banks J, Carson JS II, Nelson BL and Nicol DM (2005). Discrete-Event System Simulation, 4th edn. Prentice Hall: Upper Saddle River, NJ.
Cheng RCH (2006). Validating and comparing simulation models using resampling. J Simulat 1(1): 53–63.
Chow YS and Robbins H (1965). On the asymptotic theory of fixed-width sequential confidence intervals for the mean. Ann Math Stat 36: 457–462.
Conway RW (1963). Some technical problems in digital simulation. Mngt Sci 10(1): 47–61.
Fishman GS (1971). Estimating sample size in computing simulation experiments. Mngt Sci 18(1): 21–38.
Fishman GS (1973). Concepts and Methods in Discrete Event Simulation. Wiley: New York.
Fishman GS (1978). Grouping observation in digital simulation. Mngt Sci 24(5): 510–521.
Fox BL, Goldman D and Swain JJ (1991). Spaced batch means. Opns Res Lett 10: 255–263.
Heidelberger P and Welch PD (1981). A spectral method for confidence interval generation and run length control in simulations. Commun of the ACM 24(4): 233–245.
Hlupic V (1999). Discrete-event simulation software: What the users want. Simulation 73(6): 362–370.
Hoad K, Robinson S and Davies R (2007). Classification of discrete event simulation models and output: Creating a sufficient model set, www2.warwick.ac.uk/fac/soc/wbs/projects/autosimoa/output_data_types_/ (accessed August 2007).
Hogg RV and Tanis EA (1997). Probability and Statistical Inference, 5th edn. Prentice Hall: New Jersey.
Hollocks BW (2001). Discrete-event simulation: An inquiry into user practice. Simulat Pract Theory 8: 451–471.
Law AM (2007). Simulation Modeling and Analysis, 4th edn. McGraw-Hill: New York.
Law AM and Kelton WD (1982). Confidence intervals for steady-state simulations, II: A survey of sequential procedures. Mngt Sci 28(5): 550–562.
Law AM and Kelton WD (1984). Confidence intervals for steady-state simulations: I. A survey of fixed sample size procedures. Opns Res 32(6): 1221–1239.
Law A and McComas M (1990). Secrets of successful simulation studies. Ind Eng 22(5): 47–72.
L'Ecuyer P (1999). Good parameters and implementations for combined multiple recursive random number generators. Opns Res 47(1): 159–164.
Moshman J (1958). The application of sequential estimation to computer simulation and Monte Carlo procedures. J ACM 5(4): 343–352.
Robinson S (1994). Successful Simulation: A Practical Approach to Simulation Projects. McGraw-Hill: Maidenhead, UK.
Robinson S (2004). Simulation: The Practice of Model Development and Use. John Wiley & Sons: Chichester, UK.
Schruben L (2007). Modeling for analysis. In: Chick S, Yücesan E, Chen C-H and Henderson SG (eds). Proceedings of the INFORMS Simulation Society Research Workshop. INFORMS Simulation Society, pp. 19–26.
Shao J and Tu D (2006). The Jackknife and Bootstrap. Springer Series in Statistics, Springer-Verlag: New York.
Stein C (1949). Some problems in sequential estimation. Econometrica 17(1): 77–78.
Acknowledgements
This work is part of the Automating Simulation Output Analysis (AutoSimOA: www.wbs.ac.uk/go/autosimoa) project that is funded by the UK Engineering and Physical Sciences Research Council (EP/D033640/1). The work is being carried out in collaboration with SIMUL8 Corporation, who are also providing sponsorship for the project.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Hoad, K., Robinson, S. & Davies, R. Automated selection of the number of replications for a discrete-event simulation. J Oper Res Soc 61, 1632–1644 (2010). https://doi.org/10.1057/jors.2009.121
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1057/jors.2009.121