TAG
#llm
0 reviews · 20 essays
Related essays
Wu et al.: Rebuilding a Case Report Into a Chain of Decisions — Fukai Reads
A medical-education gamification paper by Qian Wu and colleagues (CUHK and others). MedGame is a dual-engine framework that converts static case reports into a three-level Act / Scene / Decision Node script and then into a dependency graph of multimodal generation tasks. Fine-tuning on a 5,000-case benchmark lifts structural validity from 79.4% to 99.1%, while medical accuracy plateaus around 7 out of 10.
Wang et al.: Making a Puzzle Solver the Teacher for Every Single Move — Fukai Reads
A game-AI paper by Yu Wang and colleagues. Where long-horizon puzzles reward only the final win, they convert the drop in a solver's remaining-distance-to-goal into a per-move score and mix it into training. Averaged over Sokoban, Minesweeper and Rush Hour, success rises from 16.6% to 62.1%, and on unseen difficulty from 5.9% to 28.4%. Querying the solver costs about 73 parts per million of training wall clock.
Zhou et al.: The Verifier is the Curriculum — Training Game Generation on a Launch Check Alone — Fukai Reads
A game-generation paper by Chenyu Zhou and colleagues. Starting from a diagnosis that the learned judge is gameable, they gate self-distillation on a single binary signal — does the generated Godot project launch cleanly — and over three rounds lift clean generation on four unseen families from 8.8% to 42.2%, with best-of-16 coverage going 18/25 to 25/25. Loosen the gate and the gain disappears.
Teo et al.: AI Assistants Overassist — Int-Bench Measures Intervention in Problem-Solving — Fukai Reads
Teo et al. on LLM intervention behavior. Using Int-Bench, a simulated setting, they measure when and how much AI assistants help during problem-solving, finding AI intervenes earlier and more often than humans, tends to leak the answer, and does not improve transfer. A useful read for puzzle hint design.
Johnson et al.: What Changes in a Game When You Build an LLM Into It — Fukai Reads
A qualitative study by Johnson and colleagues at the University of Calgary on developing two games with an LLM embedded in their structure. Reading developer reflections, it analyzes how embedding an LLM as a component (not decoration) changes gameplay, playability, and player experience. Variability and personalization increase, but new burdens of correctness, difficulty calibration, and coherence emerge, with schema enforcement and validation as the keys.
Zeng et al.: Automating Game Balancing with LLM-vs-LLM Self-Play — Fukai Reads
A paper by Zeng et al. on automated game balancing. It tackles balancing asymmetric strategy games by using multi-agent LLM self-play as an evaluator and Bayesian optimization to search rule parameters, reporting convergence to near-0% win-rate gaps on their own game, CivMini.
Waugh: Measuring AI's Reasoning with Sudoku and Slitherlink — Fukai Reads
A paper (arXiv preprint) by Justin Waugh of Approximate Labs on Pencil Puzzle Bench, a benchmark that measures LLM reasoning with pencil puzzles. From 62,231 puzzles across 94 types it selects 300, and its core is that a machine can verify every move against the rules; 51 models were evaluated. Even the strongest GPT-5.2 reached only 56.0% in agentic mode, with about half unsolved.
Triebel et al.: Does AI Have Both a Head and a Hand on a Classic Physics Puzzle? — Fukai Reads
A paper by Triebel et al. evaluating VLMs on the classic physics puzzle The Incredible Machine 2. Using VLATIM, a five-stage benchmark, it asks whether screen-operating AI can solve problems like humans; the cleverer large models can plan but cannot click precisely, and no model solved even one puzzle to completion.
Xu et al.: When Generative AI Becomes the Heart of Play — Fukai Reads the AI-Native Games Survey
A survey (arXiv preprint) by Zhiyue Xu and five co-authors on "AI-native games," where generative AI is the core loop itself. It defines them by a counterfactual — would play collapse if the AI were removed — and classifies 53 real artifacts along two axes: game type (G) and dominant AI mechanic (N), showing a skew toward narrative genres and a thin use of AI at the rule layer.
Letting an LLM build a whole game, and an AI playtest it: ScriptDoctor and the state of automatic game design
One piece today. I read, in the original English, "ScriptDoctor: Automatic Generation of PuzzleScript Games via Large Language Models and Tree Search" by Sam Earle, Julian Togelius and colleagues (arXiv:2506.06524; a short paper submitted to the IEEE Conference on Games). They pick PuzzleScript — the description language for turn-based 2D-grid puzzle games created by increpare (Stephen Lavelle) — as a "model organism," and have an LLM generate a whole game (rules, sprites, levels), iterating on it using compiler errors and the results of a breadth-first-search player agent. Feeding in a few human-authored games as examples clearly raises quality, and reasoning models (o1, o3-mini) beat GPT-4o. But the sharpest lesson is on the failure side: the games that looked most complex were often complex only because of broken mechanics — solvable is not the same as good. A rich read for anyone thinking about automatic game design.
Mirowski et al.: From Writing a Story to Finding One — Fabula, a Writing AI Grown With the Writers' Community — Fukai Reads
A paper on Fabula, a Google DeepMind writing-support AI. Its hierarchical story planner-generator, the Drama Manager, was critically co-developed with 42 experts; it proved strong at structure but weak at style and surprise. Fukai reads it for lessons that apply directly to game interactive narrative.
The strongest player is not the best tester: a paradox from a framework for measuring game difficulty with LLMs
One piece today. I read, in the original English, "LLMs May Not Be Human-Level Players, But They Can Be Testers: Measuring Game Difficulty with LLM Agents" by Chang Xiao (Adobe Research) and Brenda Z. Yang (Columbia University) (arXiv:2410.02829). It asks whether off-the-shelf LLMs can be used to measure game difficulty by letting them play a game and treating their performance as a difficulty proxy, tested on Wordle (a word puzzle) and Slay the Spire (a deck-building roguelike). The central finding is a paradox: LLMs play worse than the average human, yet the relative difficulty of challenges they struggle with correlates strongly with human data. Moreover, a near-optimal, information-theoretic Wordle solver that beats humans on move count showed almost no correlation with human-perceived difficulty. In other words, the entity that solves best is not the best difficulty tester. A thought-provoking read for anyone thinking about how to validate a difficulty curve.
Let the LLM Handle Story and Puzzles, Let the Symbolic Layer Keep the World From Breaking — Uruguay's IVIE on Incremental, Validated Generation of Interactive Fiction (ICCC'26)
One article today: IVIE, a paper headed to ICCC'26 by a team at the Universidad de la República in Uruguay (Vaucher, Silveira, Góngora, Chiruzzo), which I read in full in the original English on arXiv. The goal is to generate complete, playable interactive-fiction (text-adventure) worlds from scratch. The trick is a division of labor: creative decisions—setting, characters, puzzle design—go to an LLM, while a symbolic layer guarantees structural facts like spatial connectivity and objective solvability. Worlds are built backwards from the objective across four stages, each with a validation gate. In the puzzle stage, obstacle and solution are placed in different locations, solutions must be discoverable through exploration, and hints disclose in three escalating levels. Tellingly, in 3 of 16 evaluated worlds players slipped past puzzles simply by claiming they had solved them—surfacing a design tug-of-war: validate too strictly and you choke creativity; too loosely and the puzzles become hollow. Not a puzzle game per se, but a paper that touches the root of design: how to make 'validation' and 'freedom' coexist.
Monti et al.: Measuring AI's Planning Power on a Single-Corridor Sokoban — Fukai Reads
A paper by Monti and colleagues on SokoBench, a benchmark that measures reasoning models' long-horizon planning with Sokoban. By lining up only single-box straight corridors and narrowing difficulty to a single axis (corridor length), it shows that even state-of-the-art reasoning models break down once more than 25-30 moves of lookahead are needed. The authors locate the cause in accumulated miscounting.
Luo et al.: Can AI Agents Build Whole Playable Games in a Real Engine? — Fukai Reads
A paper by Luo, Wang and colleagues on GameCraft-Bench, a benchmark for end-to-end game generation by coding agents. It has agents build complete playable games on Godot from natural-language specs, judged by launch, input replay, and video-based scoring across 140 tasks in 15 families. Even the strongest configuration reaches only 41.46% overall, and the authors report that agents can build mechanics but fall short of finished games with content, readability, and polish.
Li et al.: AutoBG, an AI that supports board game design end-to-end from ideation to finish — Fukai Reads
A paper (arXiv preprint) by Zizhen Li et al. on AutoBG, a board game design assistant that covers the whole workflow—ideation, rulebook generation, and individualized feedback—via Verifier-Gated Iteration that splits the generator from the critic; the critic, BG-Critic, is reported to outperform GPT-5.4 on diagnostic quality.
Nasir et al.: Evolving the Rules of Play Themselves — Fukai Reads MORTAR
A paper on automatic game design by Nasir, Togelius and colleagues. Instead of levels, MORTAR evolves game mechanics themselves using a quality-diversity algorithm paired with a large language model, judging quality by whether stronger AI agents reliably beat weaker ones. Running on GPT-4o-mini, it generates diverse, playable games and even quantifies each mechanic's contribution.
Jiang et al.: Can a Sentence Build a Playable Game? — Fukai Reads OpenGame
A paper by Yilei Jiang et al. (CUHK) on OpenGame, an agent that generates whole 2D web games from natural language. Reusable skeletons and a 'living debug protocol' curb integration errors, setting a new state of the art across 150 tasks - though puzzles remained its weakest genre.
Li et al.: Can LLMs Play and Beat 2D Games? - Fukai Reads GVGAI-LLM
A paper by Li et al. (NYU and others) proposing GVGAI-LLM, a benchmark that has language models play 118 2D games to measure reasoning and spatial grounding. Translating boards into ASCII maps and solving zero-shot, GPT-4o-mini scored 0% on 477 of 540 levels and a 10.27% overall win rate, falling short of classic search algorithms. I unpack it as problem, method, findings, use cases, and limitations.
Can AI Build a Whole Puzzle Game? ScriptDoctor and Its Generate-Playtest-Repair Loop
ScriptDoctor has a large language model write an entire puzzle game — rules, sprites, levels — then lets a compiler and a search-based agent inspect the result and demand revisions. The testbed is PuzzleScript, a language indie developers know well. I walk through the paper in five parts — problem, method, findings, where you can use it, limitations — covering why human-authored examples boost success rates, why reasoning models win, and the distance between 'solvable' and 'fun'.
