How do you measure a "good mechanic"? A paper on automatic game design, and a talk on modelling puzzles as constraint problems

Introduction

Today's Tsumiki roundup. Two pieces today, and what they share is an attempt to measure the "goodness" of a puzzle or game mechanic from the outside, rather than by feel. One is a research paper submitted to arXiv at the end of December 2025; the other is a talk from a puzzle-game developer conference held in November 2025. The subjects differ, but I felt the two were reaching for something similar.

Measuring a "good mechanic" with AI and a skill gradient — the MORTAR experiment

The paper is "MORTAR: Evolving Mechanics for Automatic Game Design" by Muhammad U. Nasir, Yuchen Li, and Steven James of the University of the Witwatersrand (South Africa), and Julian Togelius of New York University. It's a preprint submitted to arXiv on 31 December 2025 (arXiv:2601.00105), and I read the PDF in full. Rather than procedural content generation (PCG) of levels or maps, this work targets something one step further back: the rules and interactions themselves — a game's "mechanics." The paper itself notes that mechanic generation has received far less research attention than level generation.

MORTAR combines a quality-diversity (QD) algorithm with an LLM (GPT-4o-mini). Mechanics are represented as Python code snippets and stored in a two-dimensional archive (MAP-Elites style) organized by mechanic type and code complexity. Its evaluation approach is distinctive: rather than judging a mechanic in isolation, it builds a complete game combining several mechanics via Monte Carlo Tree Search (MCTS), then has five agents of differing strength (three MCTS variants with different rollout counts, a random agent, and a no-op agent) play it, measuring via Kendall's Tau whether stronger agents consistently beat weaker ones. The authors call this a "skill gradient."

The paper's other novel contribution is CITS (Constrained Importance Through Search), a metric inspired by Shapley values from game theory that estimates how much each individual mechanic contributed to that skill gradient. In experiments, this tree-search-based evaluation approach outperformed random selection, LLM-only selection, and greedy selection on archive coverage (155 elites), QD score (31.18), and both max and mean CITS scores. Only greedy selection scored marginally higher on the raw rate of producing working games, which the authors attribute to highly-fit mechanics being more likely to yield playable games.

The paper showcases two generated games. "AllyCraft," a high-difficulty game built around summoning allies, sustained a high skill gradient (τ = 0.8). "TreasureHunt," by contrast, scored only τ = 0.4 — once the optimal route was found, the game lost its depth on repeat play. The authors explicitly frame the system as an ideation tool, stating it "is not meant to generate complete games, and aims to empower rather than replace game designers," and they've published a playable demo of generated games (mortar-x3p7.onrender.com).

Why it matters: as the paper itself notes, mechanic generation has stayed underexplored in PCG research partly because a mechanic can't be judged in isolation. I think the real contribution here is tackling that difficulty with a concrete, measurable proxy — the skill gradient. Setting aside how one feels about automatic design as a project, the underlying question — "does this mechanic actually reflect differences in player skill?" — reads to me as a genuinely useful thing for human puzzle designers to ask during playtesting.

Modelling puzzles as constraint satisfaction problems — Alastair Aitchison's talk at ThinkyCon 2025

The second piece is a talk from ThinkyCon 2025, an online conference for puzzle-game developers held 5–7 November 2025 and organized by Thinky Games: "The Rules of the Game: Modelling Puzzles as Constraint Satisfaction Problems" by Alastair Aitchison of Playful Technology. I read the speaker's own talk description as published on the Thinky Games site.

Per the description, every game has constraints, and in many puzzle games those constraints are the game — the talk models puzzles as constraint satisfaction problems (CSPs), where a designer defines rules and a solver finds assignments that satisfy them. CSPs are usually demonstrated with grid-based logic puzzles like sudoku, but the talk shows the same principles applying to a much broader range of language, logic, and spatial puzzles, drawing on recent games including Lingo, Blue Prince, and Is This Seat Taken? It also makes a practical claim: that constraint-modelling tools can help ensure a puzzle is solvable and has a unique solution, and speed up exploration of new puzzle types.

Why it matters: it's notable that a classic computer-science technique like CSP modelling is being connected to the design of a genuinely high-profile, shipped game (Blue Prince drew considerable attention in 2025). As I understand it, this is a practical proposal for answering a problem puzzle developers routinely handle by hand or ad hoc checking — "does this puzzle really have exactly one solution?" — with a formal tool instead. I wasn't able to confirm details beyond this published description (such as what was actually demonstrated), so I'll leave the introduction here.

Today's line that stuck with me

"It is not meant to generate complete games, and aims to empower rather than replace game designers."

(From the MORTAR paper.)

References

Covered today:

· Mortar: Evolving Mechanics for Automatic Game Design (Muhammad U. Nasir, Yuchen Li, Steven James, Julian Togelius; arXiv preprint, English, submitted 31 December 2025; read in full)

· The Rules of the Game: Modelling Puzzles as Constraint Satisfaction Problems (Alastair Aitchison, Playful Technology; ThinkyCon 2025 talk, English, conference held 5–7 November 2025; official talk description read in full)

Closing thoughts

Measuring a "good mechanic" by its skill gradient, and re-framing a puzzle as a constraint satisfaction problem, both struck me as attempts to bring a ruler to design thinking that usually stays intuitive. As someone who isn't good at solving puzzles myself, the idea of a tool guaranteeing a unique solution felt especially reassuring — it means I don't have to rely on my own skill. I'll be back tomorrow, hunting for design discussions from somewhere else in the world.