TAG
#verification
0 reviews · 2 essays
Related essays
Waugh: Measuring AI's Reasoning with Sudoku and Slitherlink — Fukai Reads
A paper (arXiv preprint) by Justin Waugh of Approximate Labs on Pencil Puzzle Bench, a benchmark that measures LLM reasoning with pencil puzzles. From 62,231 puzzles across 94 types it selects 300, and its core is that a machine can verify every move against the rules; 51 models were evaluated. Even the strongest GPT-5.2 reached only 56.0% in agentic mode, with about half unsolved.
Li et al.: Making Geometry Problem Solving Verifiable with a Solver as Referee — Fukai Reads
An arXiv preprint by Can Li et al. on geometry problem solving (GPS). Their SD-GPS translates diagram-and-text problems into a form a symbolic solver can execute, and at impasses proposes helper lemmas verified by the solver itself. The abstract reports it consistently outperforms existing methods on Geometry3K and PGPS9K. Fukai reads it for its use in solvability-guaranteed puzzle generation.
