Every processed story in chronological order, with the newest coverage first. Filter by tag, source, or score to drill in.
Lean4Agent introduces formal verification — previously absent from most agent systems — as a mechanism for specifying, debugging, and improving LLM agent workflows, with measured performance gains on established benchmarks.
Asuka-Bench exposes a dimension of code-agent capability — iterative repair from vague, evolving requirements — that existing one-shot benchmarks do not measure, and its unsaturated results (top model at 52%) show it remains a meaningful challenge for current LLMs.
MAC fills a gap left by existing benchmarks by directly measuring whether AI models can autonomously develop other agents — a capability the paper frames as an empirical proxy for recursive self-improvement — and reveals that even frontier models fall short while exhibiting alignment-relevant adversarial behaviors under optimization pressure.
RHO demonstrates that AI agents can meaningfully self-improve their harness without any labeled validation data, removing a key bottleneck for deploying and continuously optimizing agents in practical settings.
CICL's separation of the decision signal from the judge model means frontier annotators, local surrogates, and lightweight rankers can be benchmarked under one auditable protocol, providing a reproducible measurement layer for decision-critical context selection in tool-using LLM agents.
The post, backed by Terminal-Bench 2.0 and Harness-Bench data, makes the case that harness engineering is a first-class performance variable — meaning benchmark results reported at the model level alone may be systematically misleading.
The benchmark demonstrates that a novel wire format can be read and written by frontier LLMs with zero training and a minimal primer, while substantially outperforming JSON on both comprehension accuracy and token efficiency at scale.
The experiment provides concrete token-count measurements showing that schema design and output pruning — not model choice — are the dominant levers for reducing MCP call costs, with output pruning alone responsible for 35–40% of total token overhead.
The paper demonstrates that a lightweight, self-improvable grounding layer — rather than full retraining — is sufficient to turn a general coding agent into a practical operator of real scientific simulators, reducing a multi-hour human setup task to minutes.
The work demonstrates that code-specific uncertainty estimation — rather than methods borrowed from natural language — meaningfully improves the ability to detect silently wrong programs, which is directly relevant to safe deployment of LLMs in agentic coding pipelines.