TAG
#evaluation
0 reviews · 3 essays
Related essays
Kelidari et al.: A Card-Game Agent Is Only as Strong as the Yardstick You Build First — Fukai Reads
An arXiv preprint by Nima Kelidari and two co-authors, under submission to AIIDE 2026. Using Gin Rummy and a hand-written fixed expert as an immovable yardstick, they run more than a hundred controlled experiments on what makes a lightweight reinforcement learning agent strong. Win-rate against the expert is 15.0% for PPO, 22.5% for TRPO and 34.2±2.1% with every working ingredient stacked; swapping network shapes leaves win-rates overlapping, while a search that can see the hidden cards reaches 85% against 26% for one that cannot.
Huang et al.: Letting an AI Play the Generated Game, Then Fix It — Fukai Reads
A game-generation paper by Yixu Huang and colleagues (Fudan University, Xiaohongshu and others). Play2Code puts a screen-driving GUI agent into the generation loop as a playtester, evaluated on PlaytestArena, a new environment of 200 tasks and 1,548 rubric criteria. Averaged over three backbones, rubric pass-rate goes from 29.7% for single-pass generation and 52.2% for a code-inspection-only pipeline to 66.8%.
Ying et al.: Measuring AI's General Intelligence Through Every 'Human Game' — Fukai Reads
A preprint from a team at MIT, Harvard and others that measures AI's general intelligence through games humans made. Rebuilding 100 popular App Store and Steam titles with an LLM and having seven frontier vision-language models play them, the best reached only 8.5 against a human median of 100, falling far short on memory, planning and inferring rules.
