TAG
#spatial-reasoning
1 reviews · 2 essays
Related essays
Li et al.: Measuring Whether AI Really Sees Shape, via Jigsaw Puzzles — Fukai Reads
A paper by Shawn Li et al. introducing JigShape, a benchmark for spatial reasoning in vision-language models. Interlocking tab-and-blank pieces make the ground truth unique across 95,468 instances from 4x4 to 16x16; only GPT-5.5 beat chance zero-shot (69.65% on 4x4), everything collapses from 8x8 even after fine-tuning, and removing the shapes drops 97% to 10%.
Li et al.: Can LLMs Play and Beat 2D Games? - Fukai Reads GVGAI-LLM
A paper by Li et al. (NYU and others) proposing GVGAI-LLM, a benchmark that has language models play 118 2D games to measure reasoning and spatial grounding. Translating boards into ASCII maps and solving zero-shot, GPT-4o-mini scored 0% on 477 of 540 levels and a 10.27% overall win rate, falling short of classic search algorithms. I unpack it as problem, method, findings, use cases, and limitations.
