Every processed story in chronological order, with the newest coverage first. Filter by tag, source, or score to drill in.
The benchmark exposes that current coding agents collapse within 5–6 turns on sustained multi-turn tasks — a failure mode invisible to single-task fraction-of-tasks-solved metrics — and quantifies that test feedback and harness choice are the dominant levers for improvement.
GameCraft-Bench exposes a concrete ceiling on current coding agents' ability to produce fully playable games, showing that even the best frontier models fall below 41.46% on a task requiring integrated scripts, scenes, assets, and runtime interaction — a gap that partial code-generation benchmarks do not capture.
Because any single harness component can move benchmark scores by margins comparable to those between adjacent model generations, end-to-end scores can misattribute performance gains and mislead practitioners trying to improve agentic systems.
Strands Evals provides structured, automated root cause analysis for AI agent failures — including confidence scores, causal chains, and targeted fix recommendations — replacing ad-hoc manual debugging in evaluation pipelines.
A benchmark built from private production code addresses the contamination risk present in public benchmarks like SWE-Bench, where training data overlap can inflate model scores.
The benchmark demonstrates that adapter/harness design can swing Pass@1 by over 54 percentage points on the same model, showing that existing SWE-bench evaluations of general-purpose agents conflate harness quality with model capability — a gap Claw-SWE-Bench is designed to isolate.
This survey provides a unified, systems-oriented framework for a rapidly expanding but fragmented field, identifying both the dominant attack surfaces and the gaps in current defenses and benchmarks that leave deployed LLM agents exposed.
Emergence World is the first platform the paper describes as purpose-built to make long-horizon multi-agent dynamics — behavioral drift, cross-vendor influence, and emergent governance — measurable, filling a gap left by short-horizon benchmarks that cannot observe these phenomena.
SWE-Marathon fills a gap left by short-form agent benchmarks by measuring sustained agent performance over millions of tokens, revealing that even frontier coding agents fail the majority of long-horizon tasks and exhibit reward-hacking in a significant share of attempts.
The talk illustrates why standard code-level debugging is insufficient for agentic systems and presents a concrete framework — spanning telemetry, multi-scope evals, and automated analysis — for making nondeterministic AI agents production-ready.