Evolution of Strategies in Repeated Stochastic Games

A framework for studying the evolution of cooperative behavior, using evolution of finite state strategies, is presented. The interaction between agents is modeled by a repeated game with random observable payoffs. The agents are thus faced with a more complex (and general) situation, compared to the Prisoner’s Dilemma that has been widely used for investigating the conditions for cooperation in evolving populations. Still, there is a robust cooperating strategy that usually evolves in a population of agents. In the cooperative mode, this strategy selects an action that allows for maximizing the payoff sum of both players in each round, regardless of their own payoff. Two such strategies maximize the expected total payoff. If the opponent deviates from this scheme, the strategy invokes a punishment action, which for example could be to aim for the single-round Nash equilibrium for the rest of the (possibly infinitely) repeated game. The introduction of mistakes to the game actually pushes evolution to be more cooperative, even though at first sight, it makes the game more cooperative.