On the Computation of Randomized Markov Equilibria

This note extends the usual method for finding totally mixed strategy equilibria in 2×2 matrix games to the computation of randomized Markov equilibria in infinite-horizon discrete-time stochastic games. It is shown that the method can be applied in the class of separable sequential games with binary action sets when continuation payoffs are monotone in each player's own-mixing probabilities.