Review on Monte Carlo Simulation Stopping Rules: How Many Samples Are Really Enough?

Abstract

Due to extensive usage of stochastic simulation models correct execution of Monte Carlo simulation has become more and more important. Hereby the unknown real mean of the simulation result is estimated by the sample mean of a large number of simulation evaluations. Unfortunately, this procedure is often done carelessly. Modellers commonly use replication counts without scientific justification and sometimes underestimate the consequences of a bad or even wrong choice: if it is chosen too small, the sample mean is not a representative approximation for the regarded mean, and not only the simulation output, but also any kind of simulation analysis will not be representative at all. If the number is chosen too high, the Monte Carlo experiment will consume unnecessary computation time, which could, exemplarily, be invested into deeper model analysis instead. In this work, we present four methods that allow calculating an optimal replication number for Monte Carlo simulation and getting an image about the error between the estimated and the real mean value. The methods are furthermore evaluated on a simple case study, a stochastic cellular automaton model for simulation of an infectious disease.