Every processed story in chronological order, with the newest coverage first. Filter by tag, source, or score to drill in.
SWE-Explore provides a fine-grained diagnostic lens on coding agent capabilities that binary benchmarks like SWE-bench cannot offer, enabling targeted measurement of where exploration quality breaks down before the repair stage.
Socratic-SWE demonstrates that an agent's own solving traces can serve as a scalable, self-improving training substrate — overcoming the limitation of fixed synthetic data pipelines that are blind to the agent's actual weaknesses.
The discovery that LLM agent safety varies significantly based on conversational position — with a measurable 9–52% improvement after warm-up tasks — identifies a concrete, previously unnamed vulnerability in deployed agentic systems and proposes a benchmark and mitigation strategy grounded in empirical evidence.
The study reveals that a single instruction-tuned model cannot optimally serve both Flow and Command coding modes simultaneously, highlighting a concrete design tension that the authors argue must be carefully balanced in AI-powered coding assistant development.
The paper demonstrates that scaffolded method adaptation — not open-ended prompting — is what enables generalist coding agents to reliably advance data-curation research, a finding with direct implications for how agentic systems are designed for AI development workflows.
The study demonstrates that current LLM agents face substantial behavioral safety risks during task execution — with an average ASR of 47.1% and some models exceeding 70% — underscoring the inadequacy of static, output-only evaluation methods for agents operating with memory, tools, and environmental access.
The paper identifies that active agent control over memory storage and retrieval — rather than passive, pipeline-fixed stores — is the key driver of cross-scenario generality, a finding that directly informs how memory systems for deployed LLM agents should be designed.
Lean4Agent introduces formal verification — previously absent from most agent systems — as a mechanism for specifying, debugging, and improving LLM agent workflows, with measured performance gains on established benchmarks.
Asuka-Bench exposes a dimension of code-agent capability — iterative repair from vague, evolving requirements — that existing one-shot benchmarks do not measure, and its unsaturated results (top model at 52%) show it remains a meaningful challenge for current LLMs.
The hacker-fixer loop shows that automated, iterative verifier hardening can eliminate reward hacking that corrupts both benchmark leaderboards and RL training signal — without requiring per-task manual patching.