1/3
Steve, I agree with your main point: a ranking against a uniform field of strategies and the outcome of evolutionary dynamics are different objects. But there is more to say, both about what Wolfram's post leaves out and about what the field has established since Nowak & Sigmund. The second part matters more, because much of it is proved rather than simulated. I also ran the experiment you propose on Wolfram's own finite-state machines; results at the end.
I. What Wolfram's post leaves out
1/ Finite automata as strategies for the repeated Prisoner's Dilemma are a forty-year literature: Rubinstein, Binmore & Samuelson and others started this in the 1980s. Linster (1992) studied the evolutionary dynamics of all 26 one- and two-state finite-state (Moore) machines, which is exactly Wolfram's 2-state space.
2/ Axelrod ran Wolfram's adaptive-evolution experiment in 1987, with a genetic algorithm (in Davis, Genetic Algorithms and Simulated Annealing). Evolving against a fixed field of eight strategies from his second tournament, 11 of 40 runs produced rules that scored substantially higher than TFT. Those rules defected on the first move and exploited one member of the field. When the rules evolved against each other instead, reciprocating rules eventually spread.
3/ The experiment you propose, iterating the tournament, has also been run on Wolfram's whole class of strategies. van Veelen (2012) evolved populations drawn from every strategy that can be encoded by a finite-state automaton. The populations showed perpetual oscillation between cooperation and defection, through neutral mutants that allow other mutants to invade.
4/ Without errors, 20 of Wolfram's 22 two-state machines are deterministic memory-one strategies. TFT, Grim and WSLS are three such machines. His 2-state ranking is therefore a ranking of corners of the memory-one strategy space, the four-dimensional cube of cooperation probabilities that Nowak & Sigmund studied in full.
5/ Wolfram's play has no errors, or trembling hand. Without errors, Grim never enters permanent defection against an opponent that does not defect first. Errors are the reason WSLS and generous strategies are favored in the Nowak–Sigmund work, and errors really change the outcome in Wolfram's setting too (see below)
6/ Wolfram writes that it will be difficult to "prove a theorem" about which strategy is best. Theorems about which strategies resist invasion do in fact exist, and they are what I would most add to your thread.
For my thoughts, criticisms, and suggestions re Wolfram's work, as cited by
@nntaleb below, see: