The monkeys chose the richer option more frequently, but not stereotypically.
Our two animals chose the richer option in 60.0% and 61.6% of trials, respectively.
An ideal agent who has information about the times of the payoff transitions will converge onto the richer option in one trial and continue to choose the richer option until the time of the next transition.
For a ratio of 3 : 1, the matching law dictates choosing the richer option on 75% of trials.
Specifically, for a ratio of 1.5 : 1, the strict matching law dictates choosing the richer option in 60% of trials.
Choosing the richer option at steady state in 100% of trials would constitute very strong over-matching.
This is as expected: the model should include as little noise in the value comparison as possible in order to correctly identify the richer option.
An ideal agent should transition to the richer option as rapidly as possible, and in the steady state should maintain as high a value of ML as possible.
We define the transition rate (TR) as the change in the proportion of choices of the richer option from trial 0 to trial 1 following transition, averaged across all four possible transitions.
The theoretical limit, achieved by an ideal agent that knows the transition times and so always selects the richer option, harvests 141.2 ms of valve open time per trial, which we label as 100%.
