EM Algorithm and Stochastic Control

We propose a Monte Carlo simulation based approach, called the dynamic EM algorithm, to solve stochastic control problems. In the special case of just searching for an optimal parameter, the algorithm simply becomes the classical Expectation-Maximization (EM) algorithm in statistics. The new algorithm extends the existing literature as follows: (1) We do not assume any particular dynamics of the stochastic processes such as diffusion or jump diffusions. (2) We show the monotonicity of performance improvement in every iteration, which leads to the convergence results. (3) We focus on finite-time horizon problems, where the optimal policy is not necessarily stationary. Various applications are given, such as real business cycle, stochastic growth, and airline network revenue management. This is a joint work with Paul Glasserman, Xianhua Peng, and Xingbo Xu.