# Replicator dynamics¶

## Motivating example: The Hawk Dove Game¶

Consider a population of animals. These animals when they interact will always share their food. Due to a genetic mutation, some of these animals may act in an aggressive manner and not share their food. If two aggressive animals meet they both compete and end up with no food. If an aggressive animal meets a sharing one, the aggressive one will take most of the food.

These interactions can be represented using the matrix $$A$$:

$\begin{split}A = \begin{pmatrix} 2 & 1\\ 3 & 0 \end{pmatrix}\end{split}$

In this scenario: what is the likely long term effect of the genetic mutation?

Over time will:

• The population resist the mutation and all the animals continue to share their food.

• The population get taken over by the mutation and all animals become aggressive.

• A mix of animals are present in the population some act aggressively and some share.

To answer this question we will assume a vector $$x$$ represents the population. In this case:

• $$x_1$$ represents the proportion of the population that shares.

• $$x_2$$ represents the proportion of the population that acts aggressively.

Note that as the components of $$x$$ are proportions of the population this implies:

$\sum_i x_i = 1$

We will also assume that any given individual in the population is playing a strategy $$\chi$$, which:

• shares $$\chi_1$$ proportion of the time.

• acts aggressively $$\chi_2$$ proportion of the time.

The overall fitness of an individual in the population is then given by their expected utility (as given by $$A$$) as they interact with the population:

$\chi A x$

We can in fact write down the fitness corresponding to each action (sharing or being aggressive):

$f = A x$

The average fitness in the population is then given by:

$\phi = x ^ T f$

To understand how the population will evolve relative to their fitness the following differential equation will be used:

$\frac{dx_i}{dt} = x_i(f_i - \phi)\text{ for all }i$

In our case the differential equations are:

\begin{split}\begin{align*} \frac{dx_1}{dt} &= x_1(2x_1 + x_2 - \phi)\\ \frac{dx_2}{dt} &= x_2(3x_1 - \phi) \end{align*}\end{split}

where:

$\phi=x_1(2x_1 + x_2) + x_2(3x_1)$

This differential equation can then be solved numerically to show the evolution of the population over time. We can see that it looks like in our particular situation the mutation stays within the population and a mix of both sharing and aggressive animals will coexist.

## The replicator dynamics equation¶

Given a population with $$N$$ types of individuals. Where the fitness of an individual of type $$i$$ when interacting with an individual of type $$j$$ is given by $$A_{ij}$$ where $$A\in\mathbb{R}^{N \times N}$$. The replicator dynamics equation is given by:

$\frac{dx_i}{dt} = x_i(f_i - \phi)\text{ for all }i$

where:

$\phi = \sum_{i=1} ^ N x_i f_i(x)$

where $$f_i$$ is the population dependent fitness of individuals of type $$i$$:

$f_i(x) = (Ax)_i$

Note that there are equivalent linear algebraic definitions to the above:

$f = Ax \qquad \phi=x^TAx$

Question

For Rock Paper Scissors, what is the replicator dynamics equation?

## Stability of the replicator dynamics equation¶

Stability of the replicator dynamics equation is achieved when $$\frac{dx_i}{dt} = 0$$ for all $$i$$.

For a population vector $$x^*$$ for which $$\frac{dx^*_i}{dt} = 0$$ for all $$i$$ the population will not change without some other effect. This is referred to as a stable population.

Question

For the following games, what are the stable populations?

$\begin{split}A = \begin{pmatrix} 0 & -1 & 1 \\ 1 & 0 & -1\\ -1 & 1 & 0\\ \end{pmatrix}\end{split}$
$\begin{split}A = \begin{pmatrix} 2 & 1\\ 3 & 0 \end{pmatrix}\end{split}$

## Evolutionary stable strategies¶

Evolutionary stable strategies are strategies that when adopted by an entire population are resistant to an alternative strategy that is initially rare.

By definition an evolutionary stable strategy corresponds to a stable population.

For the hawk dove game there are 3 stable populations:

• $$x=(1, 0)$$

• $$x=(0, 1)$$

• $$x=(1 / 2, 1 / 2)$$

However, if a small deviation is made from the first two populations then the population does not “resist”. For example, we consider the initial population $$x=(1, 0)$$ and introduce a small population aggressive behaviours to have: $$x = (1 - \epsilon, \epsilon)$$ where $$\epsilon>0$$. The plot below shows this with $$\epsilon=10 ^ -5$$:

This is also what happens if we start with a population of aggressive animals: We consider the initial population $$x=(0, 1)$$ and introduce a small population aggressive behaviours to have: $$x = (\epsilon, 1 - \epsilon)$$ where $$\epsilon>0$$. The plot below shows this with $$\epsilon=10 ^ {-5}$$:

However, this is not the case with the third stable population: $$x=(1 / 2, 1 / 2)$$. The plot below shows $$x=(1 / 2 - \epsilon, 1 / 2 + \epsilon)$$ with $$\epsilon=10^{-2}$$:

These observations can be confirmed analytically. Information on this can be found in [Fudenberg1998], [Webb2007] and [Nowak2006].

The replicator equations were first presented in [Maynard1974].

## The replicator-mutation dynamics equation¶

An extension of the replicator equation is to allow for mutation [Komarova2004]. In this case reproduction is imperfect and individuals of a given type can give individuals of another.

This is expressed using a matrix $$Q$$ where $$Q_{ij}$$ denotes the probability of an individual of type $$j$$ is produced by an individual of type $$i$$.

In this case the replicator equation can be modified to give the replicator-mutation equation:

$\frac{dx_i}{dt} = \sum_{j=1}^Nx_j f_j Q_{ji}- x_i\phi\text{ for all }i$

where, as before:

$f = Ax \qquad \phi=x^TAx$

This can modify emergent behaviour. For the Hawk Dove game if there is a 10% change that aggressive individuals will produce sharing ones the matrix $$Q$$ is given by:

$\begin{split}Q = \begin{pmatrix} 1 & 0\\ 1 / 10 & 9 / 10 \end{pmatrix}\end{split}$

The plot below shows the evolution of the system:

Question

Show that for $$Q=I_N$$ (the identity matrix of size $$N$$) the replicator-mutation equation corresponds to the replicator equation.

## Exercises¶

1. Consider a population with two types of individuals: $$x=(x_1, x_2)$$ such that $$x_1 + x_2 = 1$$. Obtain all the stable distribution for the system defined by the following fitness functions:

1. $$f_1(x)=x_1 - x_2\qquad f_2(x)=x_2 - 2 x_1$$

2. $$f_1(x)=x_1x_2 - x_2\qquad f_2(x)=x_2 - x_1 + 1/2$$

3. $$f_1(x)=x_1 ^ 2 \qquad f_2(x)=x_2^2$$

2. For the following games, obtain all the stable distributions for the evolutionary game:

1. $$A = \begin{pmatrix}2 & 4 \\ 5 & 3\end{pmatrix}$$

2. $$A = \begin{pmatrix}1 & 0 \\ 0 & 1\end{pmatrix}$$

## Using Nashpy¶

See Use replicator dynamics for guidance of how to use Nashpy to obtain numerical solutions of the replicator dynamics equation. See Use replicator dynamics with mutation for guidance of how to use Nashpy to obtain numerical solutions of the replicator-mutation dynamics equation.This is what is used to obtain all the plots above.