Publications by Year: 1995

1995
Burnetas, A.N. & Katehakis, M.N., 1995. Efficient estimation and control for Markov processes. In Proceedings of the IEEE Conference on Decision and Control. pp. 1402-1407. Website
Burnetas, A.N., 1995. Computing Optimal Policies for Markovian Decision Processes Using Simulation. Probability in the Engineering and Informational Sciences, 9, pp.525-537. Website
Burnetas, A.N. & Katehakis, M.N., 1995. Computing Optimal Policies for Markovian Decision Processes Using Simulation. Probability in the Engineering and Informational Sciences, 9, pp.525-537. Website Abstract
A simulation method is developed for computing average reward optimal policies, for a finite state and action Markovian decision process. It is shown that the method is consistent; i.e., it produces solutions arbitrarily close to the optimal. Various types of estimation errors and confidence bounds are examined. Finally, it is shown that the probability distribution of the number of simulation cycles required to compute an é-optimal policy satisfies a large deviations property. © 1995, Cambridge University Press. All rights reserved.
Burnetas, A.N. & Katehakis, M.N., 1995. Efficient estimation and control for Markov processes. In Proceedings of the IEEE Conference on Decision and Control. New Orleans, LA, USA: IEEE, Piscataway, NJ, United States, pp. 1402-1407. Website Abstract
We consider the problem of sequential control for a finite state and action Markovian Decision Process with incomplete information regarding the transition probabilities P ∈ Papprox. Under suitable irreducibility assumptions for Papprox.. We construct adaptive policies that maximize the rate of convergence of realized rewards to that of the optimal (non adaptive) policy under complete information. These adaptive policies are specified via an easily computable index function, of states, controls and statistics, so that one takes a control with the largest index value in the current state in every period.