On confidence intervals from simulation of finite Markov chains - Burnetas, AN; Katehakis, M.N. (Mathematical Methods of Operations Research, 1997)
Consider a finite state irreducible Markov reward chain. It is shown that there exist simulation estimates and confidence intervals for the expected first passage times and rewards as well as the expected average reward, with 100% coverage probability. The length of the confidence intervals converges to zero with probability one as the sample size increases; it also satisfies a large deviations property.
Keywords: Computational methods; computer simulation; Convergence of numerical methods; Discrete Markov chains; Markov processes; probability