Bhaumik et al.: Stitching WFC and Reinforcement Learning for Playable, Good-looking Levels — Fukai Reads

TL;DR

This paper tries to cancel out the weaknesses of two content generators. WFC (Wave Function Collapse — a constraint-solving algorithm that learns local rules, namely which tiles may sit next to which, from example maps, then fills a grid one cell at a time so the result resembles the examples) tends to make good-looking but unplayable levels. PCGRL (procedural content generation via reinforcement learning — a framework that builds a level step by step and can bake in global properties like solvability through the reward) tends to make playable but ugly levels.

The authors let the local rules learned by WFC narrow down which moves the reinforcement-learning agent may make at each step, and within that restriction the agent picks the placement that moves the level toward being solvable. They call it WCRL (Wave Collapse via Reinforcement Learning), and test it on the 1983 puzzle-platformer Lode Runner. In short: with the right tuning it produced levels that were both example-like in appearance and actually playable, but the method was sensitive to which input levels were used and how rare patterns were handled. It is an arXiv preprint (submitted May 2026, not yet peer-reviewed). This article aims to convey the gist without opening the paper.

Introduction

The authors are Debosmita Bhaumik, Julian Togelius, Georgios N. Yannakakis, and Ahmed Khalifa. The paper appears as arXiv:2605.13570v1 (cs.AI, submitted 13 May 2026, licensed CC BY 4.0). There is no statement that it has passed peer review, so I treat it as an arXiv preprint — freshly posted and possibly not yet peer-reviewed.

This author group sits at the center of PCG (procedural content generation) research, and the very framework this paper builds on, PCGRL, was itself proposed by Khalifa et al. (reference [15] in the paper). In other words, the people who built the tool for making levels with reinforcement learning are now patching that tool's weakness — poor visuals — with a different tool, WFC.

I chose it today because in PCG the machine-learning family (good at capturing the look of examples, but with no guarantee of playability) and the search/RL family (able to guarantee playability, but visually rough) have long been discussed as separate tools. A design that stitches those two roads into a single loop is something an implementer can copy directly. It reads less like a theory paper and more like a guide to assembling tools.

Background

When generating levels, the authors split quality into two kinds. One is function — can the player finish the level, can required items be reached — properties you can only judge by looking at the whole level. The other is visual style — whether it matches the game's look — which can be judged locally.

The learn-from-examples family (GANs — Generative Adversarial Networks, which train a generator and a judge against each other; LSTMs; and recently large language models) is good at capturing the look, but cannot guarantee function, because solvability is not part of the learning signal. A level with floor tiles scattered in the sky still works if a path exists; a human-looking level fails if the path is broken. Look and function do not always coincide.

PCGRL, which builds levels with reinforcement learning (a framework that learns actions yielding higher reward through trial and error), can learn function by putting solvability into the reward — but then the visuals suffer. As the paper's Figure 13 shows, the authors themselves call RL-only levels 'downright ugly.' Hence the paper's question: can we hand visuals to the constraint solver and function to reinforcement learning?

Approach

The skeleton of WCRL is this. WFC first cuts the input level into small 3×3 windows (tile blocks, i.e. patterns) and learns adjacency rules — which patterns may sit above, below, left, and right of which. That extracted set of patterns becomes the reinforcement-learning agent's action choices.

Generation starts from an empty grid. Each cell initially holds every pattern that could go there, and WFC picks the cell with the fewest remaining options — the most cornered spot — to resolve next (WFC's original minimum-entropy strategy; entropy here just means how many options / how much uncertainty). But what to place there is decided not by plain WFC but by the RL agent, which is handed only the moves currently legal for that cell (this is invalid action masking, blocking illegal moves up front).

Once the agent picks a move, WFC propagates that choice across the map, deleting patterns that conflict with the adjacency rules. This makes invalid moves disappear on the next step. Training uses Maskable PPO (a variant of Proximal Policy Optimization that can exclude illegal moves), with a small network of three convolution layers plus two fully-connected layers.

The reward is the crux. The authors use a simple automated player (with simplified game rules) that runs a flood fill (a search that spreads out to reachable cells, like paint filling a region) to count how much gold is reachable from the player's position. Moves that increase connectivity so more gold is reachable get positive reward; moves that decrease it get negative. A move that makes WFC hit a contradiction (a cell where no pattern fits) gets a big negative reward, teaching the agent to avoid dead ends. Note this simple player omits digging and enemy movement, so it is only an approximation.

Findings

The experiments turned three knobs. (1) Input levels — a single one (SI), several (MI), or several deliberately different-looking ones (div-MI). (2) Learned patterns — keep or drop patterns that appear only once, the rare ones (drop = RR). (3) Starting state — begin from empty, or from a random, partially filled state (RC). The combinations gave 12 experiments; each was trained as 5 separate models (to check training stability), and each model generated 100 levels for evaluation. The domain is Lode Runner, level size 32×22, pattern window 3×3, and each training ran for 5 million steps.

On input levels: giving several levels that resembled each other raised the overall share of playable levels. Mixing in very different-looking levels raised diversity but lowered playability, because the connective 'seams' between different styles are hard to find. The authors report that div-MI combined with rare-pattern removal and random start (div-MI+RR+RC) was the hardest setting and failed most often through contradictions.

On rare patterns: dropping them shrinks the option set and lowers diversity. Yet for multiple inputs it actually raised playability — the agent can focus on common, well-connecting patterns, the authors reason. For a single input, dropping rare patterns made the space too narrow to build levels at all (so for single-input runs they kept an exception: patterns containing the player tile are not removed).

On starting state: training from a random partially-filled state made little difference to playability and only slightly improved diversity. But models trained that way became robust to the starting state and, in fact, found playable levels more easily from a non-empty start. The authors suggest this random collapse could be a key to more general policies usable across games and to transfer learning (carrying knowledge learned in one game into another). Most of the numbers live in the paper's Figures 7–12 as graphs; the text does not state the percentages themselves, so I present these as trends.

Use cases

In take-home form for game and puzzle makers, first: if you build tile-based levels with WFC and struggle with 'good-looking but unsolvable' boards slipping in, the bone of this paper is to replace WFC's tile selection with something that evaluates solvability. If reinforcement learning is too heavy, the replacement can be a search/solver instead. The borrowable idea is to hand the single choice of 'what to place in the minimum-entropy cell' to an evaluator that measures function.

Second: if you make a hyper-casual or endlessly-generated game and want to keep a consistent art feel while absolutely avoiding dead ends, the split of local rules (look) and global reward (playability) applies directly. Pull the look automatically from existing maps, and write into the reward only your game's own clear conditions — the key is reachable, a path leads to the exit. The advantage is that you never encode art quality into the reward.

Third: for designers who want to tune difficulty and diversity with knobs, the experiments themselves are a recipe. More inputs raise diversity; dropping rare patterns lowers diversity but raises the playable rate. That relationship maps straight onto a 'diversity mode / stability mode' switch. For something like a daily puzzle — a bit different each day but always solvable — these two knobs are practical control points.

Fourth, the payoff of action masking (blocking illegal moves up front) transfers to other genres. In placement puzzles that involve constraint satisfaction (finding an arrangement that meets many conditions at once) — Sudoku-likes, tile-placement games — simply making the generator or helper AI 'never even consider rule-breaking moves' cuts generation failures and wasted search, as a general lesson.

Limitations

From the authors' own admissions: the method is sensitive to hyperparameters (settings a human fixes before training), and quality drops sharply if you don't hit a good configuration. Mixing very different inputs makes seams hard to find and lowers playable output. The authors say adding smoother in-between levels that blend both styles 'should' help, but that is not tested here. Evaluation is limited to the single title Lode Runner, and generalization to other games is left for future work.

Now points I noticed as Fukai. First, since 'playability' is judged by a simplified player that omits both digging and enemy movement, the playable share reported here is not true solvability itself. Lode Runner's core is the interplay of digging holes to trap enemies and open paths; how well an evaluation that removes that core reflects real fun or difficulty lies outside this paper's frame.

Second, the text states almost none of the playability or diversity numbers directly; the trends live in graphs (Figures 7–12). So I too have not quoted specific percentages. Third, this is still a pre-review preprint with no accumulated citations. The trends reported here are 'observed for these settings and this domain,' and it is too early, I think, to take them as laws of PCG in general.

Fukai's reading

I write this explicitly as my own reading. I want to place this work in a trend of re-framing generation design as a division of labor. Local order — the look — is handed to the constraint solver (WFC), global order — playability — to reinforcement learning, and the two are joined at a single point, action masking. In the vocabulary of design criticism, it reads as separating 'automating the house style (the game-specific look)' and 'guaranteeing solvability' into distinct modules, then stitching them into one loop. Less a compromise between two tools than an interesting wiring question: where in the generation loop do you seat the evaluator? I also like that the choice of Lode Runner itself makes a test bed where a human eye can easily judge whether that wiring works.

Closing

For those who want to go deeper: this paper directly builds on PCGRL (Khalifa et al.), which proposed building levels cell by cell with reinforcement learning, and on Babin & Katchabaw (FDG 2021), which combined reinforcement learning and WFC to make Mario levels. Reading those two first reveals the map of what this paper inherits and what it changes (domain from Mario to Lode Runner, optimization from an evolution strategy to PPO).

To learn the background of WFC itself, Karth & Smith's argument re-casting WFC as 'constraint solving in the wild' is a good entry point. The idea of embedding an evaluator that measures solvability inside the generation loop should only grow in puzzle generation, and this paper is worth keeping on hand as one example of that wiring, I think.

Sources

Papers and related material referenced in this article:

・Learning Local Constraints for Reinforcement-Learned Content Generators (Bhaumik, Togelius, Yannakakis, Khalifa, 2026, arXiv preprint 2605.13570)

・Related: PCGRL: Procedural Content Generation via Reinforcement Learning (Khalifa et al., 2020) (the framework this paper builds on)

・Related: Babin & Katchabaw, Leveraging Reinforcement Learning and WaveFunctionCollapse for Improved Procedural Level Generation (FDG 2021) (prior work combining RL and WFC)