Ponnock & Ho: The Order of Mario 1-1 Has a Measurable Teaching Effect — Fukai Reads

TL;DR

World 1-1 of Super Mario Bros has long been held up as the canonical example of a level that teaches its own mechanics. This paper tries to measure that claim using reinforcement learning agents (a framework where an agent gradually learns which actions earn more reward by trial and error). The authors reimplemented World 1-1 from scratch as a tile-based discrete environment and compared four learning algorithms across three versions of increasing complexity.

Monte Carlo came out strongest, reaching a 94.9% ± 1.5% win rate on the full version with enemies; a Deep Q-Network (DQN) reached only 76.4% ± 3.4%. The authors then split the level into six segments (A-F) and trained across twelve conditions that permuted segment order. The canonical order converged fastest, had the highest learning efficiency, and was the only condition with zero catastrophic failures across ten seeds. Reversed order landed at 48.5% ± 39.7%, with four of ten seeds failing outright.

The authors frame this as, to the best of their knowledge, the first empirical validation that World 1-1's canonical design encodes genuine pedagogical structure that chance cannot replicate. Note that this is an arXiv preprint (submitted June 2026, not yet peer-reviewed), and the subjects are learning agents, not humans.

Introduction

The paper is "Reinforcement Learning in Super Mario Bros: Curriculum, Pedagogy, and Optimal Level Design in World 1-1" (arXiv:2606.29511) by Jesse Ponnock and Lucas Ho, both at Johns Hopkins University. It is an arXiv preprint; the text carries no indication of conference or journal peer review, so it should be read as work that has not yet been widely scrutinized.

I picked it today for a simple reason: it takes aim at one of the most repeated pieces of folklore in game design. The story that World 1-1 opens with a single Goomba, then pipes, then gaps, then saves enemy clusters and the staircase for the end, and that this is why players learn the controls without being told, has been retold many times, including by Shigeru Miyamoto himself (the paper cites that interview as reference [9]). But whether that is an ordering effect or a narrative retrofitted onto a well-made level had, as the authors say, never been formally tested.

What makes the paper interesting is that the measuring instrument is a learning agent rather than a human. Running the same study on people brings a pile of problems: prior familiarity with Mario, fatigue, carryover between orderings. An agent can be reset to zero and retrained ten times per condition. The price is that conclusions about agents do not automatically transfer to people, which I will come back to.

Background

Curriculum learning (the idea that ordering training examples from easy to hard makes learning faster and more stable even with identical data) has been familiar in machine learning since Bengio and colleagues' 2009 paper. On the game design side, the practice of teaching through level structure rather than explicit tutorials, sometimes called the implicit tutorial, is decades old. The two look like the same claim in different vocabularies, but quantitative work connecting them has been thin.

In procedural content generation (PCG, the automatic algorithmic creation of levels and items), Dahlskog and Togelius decomposed World 1-1 into design patterns back in 2012, and the paper builds on that. Extracting patterns and testing whether their ordering matters are different questions, though: the first asks what is placed, the second asks in what order it is met.

A third piece of context: reinforcement learning practice usually chooses between tabular methods (writing values into a table) and deep networks based on state space size. The authors exploit the fact that World 1-1 is natively a tile grid, so a table is exact rather than approximate. That sets up the comparison deliberately: if DQN wins here, it wins through generalization, not necessity.

Approach

Without using any game engine or RL library, the authors reimplemented World 1-1 in Python as a 212 x 14 tile grid. There are three versions. v1 is static: ground, pipes, gaps and the end-of-level staircase. v2 adds breakable bricks and question blocks at NES-accurate positions. v3 adds 17 enemies (16 Goombas and 1 Koopa, treated uniformly) moving at one third of Mario's speed, with gravity, cascading activation and a stomp mechanic.

There are only three actions: stand still, move right, and jump (grounded only). Left movement is deliberately omitted. The goal is always to the right, adding left would substantially enlarge the state space and exploration burden, and the authors note the +100 win reward dominates any gain from backtracking. The jump follows a fixed six-phase arc with horizontal control retained in the air, so distance varies but height does not, a significant simplification of the original's variable jump.

Rewards: reaching the flagpole +100, death by gap or enemy -50, a rightward step +0.5, standing still -0.1, breaking a brick +1, hitting a question block +3, stomping a Goomba +5. The directional reward is calibrated so that traversing the level totals roughly 49.5, making it co-primary with the win bonus. This is explained as preventing both the degenerate policy of dying immediately and the policy of farming directional reward while ignoring the goal.

Four algorithms were compared: Q-Learning, SARSA, first-visit Monte Carlo (which waits until an episode ends and then writes the whole episode's return back onto every state-action pair visited), and DQN (which replaces the table with a neural network and learns from randomly sampled past transitions held in a replay buffer). Each condition ran 10,000 episodes across 5 seeds, extended to 10 seeds for the canonical and reversed curriculum conditions.

Findings

First, the algorithm comparison. On v1 and v2 all three tabular methods exceed a 98.8% win rate. The separation appears on v3 with enemies: Monte Carlo 94.9% +/- 1.5%, Q-Learning 86.5% +/- 2.7%, SARSA 84.4% +/- 5.6%, DQN 76.4% +/- 3.4% (Table 3). Monte Carlo's advantage is reported as significant by Welch's t-test against Q-Learning (p = 0.0008, d = 3.85) and against DQN (p = 0.0001, d = 7.04).

The gap is behavioral as well as numerical. Monte Carlo hits 4.05 bricks and 5.20 question blocks per episode against Q-Learning's 1.33 and 3.96. The authors explain this through the update mechanism: temporal-difference methods propagate value locally, favoring the most direct route to the flagpole, whereas Monte Carlo adds the whole episode return to every action on a winning trajectory. Intermediate pickups worth +1 and +3 therefore get credited too, and a richer detour-including strategy emerges on its own.

DQN behaves qualitatively differently. On v1 it collapses to 10.6% +/- 3.7% and never stabilizes. The authors attribute this to replay buffer imbalance: with almost no intermediate reward, death transitions (-50) flood the buffer and starve the network of positive signal. Once question blocks (+3) are scattered through v2, DQN recovers to 93.4% +/- 1.2%. From this they argue reward density deserves to be treated as a first-class selection variable alongside state space size. On v3, DQN averages 1.38 enemy kills per episode against 3.28-3.78 for tabular agents, having generalized a cautious policy that treats enemies as threats rather than rewards.

Now the curriculum experiment. The level was split into six segments; content (enemy count, gap count, block count) was held identical and only the order varied across twelve conditions trained with Monte Carlo. Canonical order: 94.7% +/- 1.6% win rate, 2771 +/- 133 episodes to 50%, area under the win-rate curve (AUC) 67.2% +/- 2.0%, zero catastrophic failures in ten seeds. Reversed: 48.5% +/- 39.7%, 6586 +/- 2865 episodes to 50%, AUC 27.5% +/- 22.8%, four failures in ten. The ten random permutations average 89.0% +/- 8.9%, sitting between the two (Table 4). Canonical versus reversed is reported at p = 0.005, d = 1.64 on win rate and p = 0.0023, d = 1.88 on convergence.

The breakdown of the random conditions is telling. The best map, Map 3 (B-E-C-D-F-A), reaches 95.2% and is statistically indistinguishable from canonical, while Map 6 (B-E-D-C-F-A) falls to 64.5%. The only difference between them is when segment D, the cluster of nine Goombas, arrives. The authors write that any ordering placing D in position two or three without a gentler predecessor risks early catastrophic failure. Running the same twelve conditions with DQN erases the ordering effect entirely (one-way ANOVA F(2,57) = 0.19, p = 0.82, eta-squared = 0.007), which the authors attribute to uniform replay sampling destroying temporal order information.

Use Cases

First, as an instrument for locating difficulty spikes. What the paper does operationally is hold content fixed, permute order, and watch learning speed and failure rate. If you are building a Sokoban-like or an escape-room puzzle, take the six to eight stage groups you already have, shuffle their order, and run a simple solver or playtest bot from scratch each time. If failure rate jumps whenever a particular group lands early, that group is your difficulty spike. You get to name your own segment D.

Second, it suggests better metrics. The authors did not rest the canonical advantage on final win rate alone; they report episodes to convergence, area under the learning curve (AUC), and the count of seeds that failed catastrophically. On final win rate alone, random Map 3 is indistinguishable from canonical. When reading playtest logs, separating average clear rate from "how many attempts before clear rate lifts off" and "did some fraction of players get stuck permanently" makes ordering quality much easier to see. The paper is a demonstration of that three-part measurement.

Third, reward density translates directly into feedback density. DQN broke on v1 and recovered on v2 because +3 pickups were scattered along the way. If you generate hypercasual levels procedurally, it is worth making "how much guaranteed small success sits between hard sections" an explicit generation constraint. Conversely, long stretches that return nothing but failure look risky for any learner, agent or human.

Fourth, it bears directly on choosing playtest AI. The ordering effect vanished entirely under DQN. If you wanted to auto-evaluate tutorial ordering and reached for a replay-buffer bot, every ordering would score the same and you could conclude, wrongly, that order does not matter. To evaluate order you need a learner that preserves the temporal sequence of experience, such as an episode-level Monte Carlo method. That strikes me as a concretely useful warning.

Limitations

What the authors state themselves: the paper has no dedicated limitations section, and its caveats are scattered through the discussion and conclusion. They note that SARSA and Q-Learning were not included in the curriculum experiment, leaving open how sensitive step-level TD methods are to ordering, and they list the complexity crossover at which DQN would definitively surpass tabular methods as future work. They also state plainly that DQN hyperparameters were tuned empirically on v1.

What follows is what I, Fukai, would point out. The biggest issue is that this is about agent learning efficiency, not human learning. Monte Carlo is order-sensitive precisely because it defers updates until an episode ends. A human player learns locally the instant they fall into a gap; the update structure is simply different. The authors write of validating pedagogical structure, and I read that more precisely as pedagogical structure for a Monte-Carlo-like learner.

Second, of 720 possible permutations, twelve conditions were run and only ten of them random. Resting "cannot be replicated by chance" on ten samples feels thin, and indeed Map 3 among them is indistinguishable from canonical on win rate. The authors' claim is properly hedged (no random ordering matched all three criteria simultaneously) and readers should keep the hedge attached.

Third, the environment simplifications are not small. No left movement, a fixed-height six-phase jump, and Koopas treated as Goombas each remove part of what the original World 1-1 teaches. Variable jump height in particular is one of the central skills the level introduces early. Measuring "the pedagogical structure of 1-1" in an environment lacking it warrants a caveat. One small note: in section 3.2.3 the prose lists the reversed condition's win rate as 94.7% +/- 1.6%, identical to canonical. Table 4 and the surrounding text make clear this is a typo for 48.5% +/- 39.7%, an unproofread preprint artifact. When quoting numbers, trust the table.

Fukai's Reading

I would place this study as a case of level-design criticism being translated into a measuring instrument. "1-1 teaches you" has long been a critical claim whose truth rested on persuasiveness. What this paper does is rewrite it into something falsifiable: does permuting only the segment order change how fast the learning curve lifts and how often training collapses? The ordering effect did show up, but so did a condition attached to it, namely that the effect only appears for learners with a particular update mechanism. I read that second finding as having the longer reach. If the pedagogical value of an ordering depends on the internal structure of the learner, that leads straight to a design question: the same level is good teaching material for some players and not others, depending on how they learn.

Closing

For readers who want to go deeper, start with a paper this one cites: Dahlskog and Togelius, "Patterns and Procedural Content Generation: Revisiting Mario in World 1 Level 1" (2012), which shows the groundwork the six-segment partition rests on. From the curriculum learning side, Bengio and colleagues' 2009 original is short and readable.

If you care about measuring difficulty itself rather than ordering, reading this alongside the Tetris difficulty work by Wang and colleagues and the logic-grid puzzle difficulty work by Shyne and colleagues, both covered here previously, gives you the same question from another angle: does difficulty live in the content or in the order you meet it? This paper answers that some of it lives in the order, with the caveat of for whom.

Sources

Papers and materials referenced in this article:

・Reinforcement Learning in Super Mario Bros: Curriculum, Pedagogy, and Optimal Level Design in World 1-1 (Jesse Ponnock, Lucas Ho, 2026, arXiv preprint arXiv:2606.29511)

・Full HTML text of the paper (all quotations in this article refer to this version)

・Related: Patterns and Procedural Content Generation: Revisiting Mario in World 1 Level 1 (Dahlskog & Togelius, 2012, DPG '12)

・Related: Curriculum Learning (Bengio, Louradour, Collobert & Weston, 2009, ICML)

・Related: Human-level control through deep reinforcement learning (Mnih et al., 2015, Nature 518:529-533)

・gym-super-mario-bros (Kautenja, 2018) — cited by the paper as the reference for its reward shaping