Every processed story in chronological order, with the newest coverage first. Filter by tag, source, or score to drill in.
SWE-Future offers a path to coding-agent benchmarks that are both grounded in real repository evolution and resistant to data contamination from historical pull-request replay.
GameCraft-Bench exposes a concrete ceiling on current coding agents' ability to produce fully playable games, showing that even the best frontier models fall below 41.46% on a task requiring integrated scripts, scenes, assets, and runtime interaction — a gap that partial code-generation benchmarks do not capture.
ReproRepo replaces the manual curation bottleneck of prior reproducibility benchmarks with a scalable, naturally occurring signal from GitHub issues, enabling ongoing large-scale evaluation of LLM agents on real-world ML paper auditing.
Because any single harness component can move benchmark scores by margins comparable to those between adjacent model generations, end-to-end scores can misattribute performance gains and mislead practitioners trying to improve agentic systems.
The paper identifies task decomposition — not retrieval — as the binding constraint in multi-skill agent planning, and SAD's single-iteration fix raises decomposition accuracy by over 32 percentage points, directly improving how reliably agents can assemble executable plans from large real-world skill libraries.
The finding that 80.2% of agent-authored test patches lack meaningful assertions means that quality gates relying on test-file presence give a false signal of verification coverage in AI-generated code.
The paper establishes that PLT performance saturates at exactly two loops and provides a gain–cost diagnostic framework explaining why, giving practitioners a principled basis for loop-count selection rather than relying on monotonic scaling assumptions.
The framework demonstrates that automated prompt optimization alone — without any fine-tuning — can turn a completely failing LLM agent (0% on PutNext) into one that succeeds nearly three-quarters of the time, showing prompt engineering can be systematically automated rather than done by hand.
The paper demonstrates that source attribution is an independent axis of factuality verification — meaning standard source-blind metrics can pass answers that contain incorrect attributions, a gap ProvenanceGuard is designed to close in MCP-based agents.
The study establishes that explicit delegation contracts improve the reviewability of AI coding agent work — not its correctness — reframing the contract as a mechanism for human oversight rather than a driver of agent task performance.