Structural properties of a class of robust inventory and queueing control problems

Abstract

In standard stochastic dynamic programming, the transition probability distributions of the underlying Markov Chains are assumed to be known with certainty. We focus on the case where the transition probabilities or other input data are uncertain. Robust dynamic programming addresses this problem by defining a min-max game between Nature and the controller. Considering examples from inventory and queueing control, we examine the structure of the optimal policy in such robust dynamic programs when event probabilities are uncertain. We identify the cases where certain monotonicity results still hold and the form of the optimal policy is determined by a threshold. We also investigate the marginal value of time and the case of uncertain rewards.