Gloaguen et al., Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?

• https://arxiv.org/abs/2602.11988
• Thibaud Gloaguen, Niels Mundler, Mark Muller, Veselin Raychev, Martin Vechev

LLMs, Software engineering, Benchmark

Evaluated whether repository-level context files (e.g., AGENTS.md, CLAUDE.md) actually help coding agents. Found that they tend to reduce task success rates while increasing inference cost by over 20% — the files encouraged agents to explore more broadly and test more thoroughly without improving outcomes. The recommendation is to keep context files minimal.

Directly relevant to Claude Code’s use of CLAUDE.md. A cautionary example of how imposing human-written structure on capable agents can backfire. Over-specifying context constrains rather than assists (see Claude Code).