More on the topic...
Generating detailed summary...
Failed to generate summary. Please try again.
We re-ran FrogsGame—the long-horizon challenge where AI agents improve a fixed base model (Qwen3-8B) on a puzzle—using Claude Fable 5, Opus 4.8 and GPT-5.5. In earlier tests, models often sketched solid high-level plans but then trained on their own weak traces and amplified errors. They struggled to spot when data was worthless, when to call a stronger model, or when a simple solver would beat fancy reward shaping. Evaluation also tripped them up: small test samples, noisy pass-rates and blown-up internal benchmarks led agents to think they were making progress when they weren’t.
Opus 4.8 and GPT-5.5 cut those obvious failures by threefold in pass@4: Claude hit 30.9% and GPT-5.5 reached 9.7%, compared to much lower scores before. Neither model dominated the baseline yet several runs actually improved checkpoint performance. Key shifts: no more overconfident wrong answers (agents now admit failure under time pressure), more curriculum use, and better algorithm choices. Claude stuck to GRPO from the start—no weak-trace SFT steps—while GPT still began with SFT and often barely touched reinforcement learning. Reward design settled on simple binary feedback; anything fancier usually hurt.
Fable 5 smashed the biggest barrier: it stopped training on bad traces by generating correct solutions with a backtracking algorithm, then distilled those into Qwen3-8B before hill-climbing with RL—all within 20 hours. Over 100 rollouts showed it’s the only model to produce reliable, high-quality SFT data across all board difficulties. Calibration errors fell to 1.2× overoptimism (versus 4.9× for Opus 4.8), agents used the full time budget instead of quitting early, and the best run churned out three times more diverse examples. It ran 512 rollouts per step for 95 steps plus 6,000 balanced SFT boards, compared with Opus’s 128 rollouts for 64 steps. That richer signal, especially on hard puzzles, let Qwen3-8B actually learn how to recover when its own reasoning went off track.
Questions about this article
No questions yet.