Context engineering

How to organize instructions and context for LLM agents—CLAUDE.md, AGENTS.md, and skill files in harnesses like Claude Code. My core principle is Progressive disclosure: thin pointers at the top, detail discovered on demand. Much of this will be obvious to anyone steeped in software engineering.

The most common mistake: over-engineering and over-prescribing#

As I noted in the Claude Code page, the most common mistake when working with LLM agents is over-engineering: trying to build too much, instruct too much, constrain too much. It does feel productive—every rule you write down seems like accumulated wisdom—but it backfires, and it backfires harder as the models get smarter.

First, rules confine. What you prescribe may not be the best way to do the task; the model may know better—the Bitter Lesson applied to context engineering. Detailed rules lock the model into your—quite possibly inferior—way, so it cannot do better than what the rules say. A prescription written for much weaker models becomes a ceiling for new models. This is probably the most critical issue.

Second, rules distract. CLAUDE.md and AGENTS.md files are injected into every session, so the model is reminded of every minute detail—do this here, never do that there—in every single conversation, even when they are totally irrelevant. That not only exhausts valuable context but also reduces performance; Gloaguen et al. (2026) found that repository context files tend to reduce coding-agent success rates while increasing inference cost by over 20%.

Third, rules rot. They rot because the world changes. New models are different; your workflows evolve constantly. As rules become stale, they may mislead the agents. The more rules you have, the more of them have the risk of becoming outdated. Every instruction and rule you add becomes more maintenance you have to do.

The better way: progressive disclosure through thin pointers#

Instead, apply progressive disclosure. At the top level, provide mostly thin pointers: tell the model where things are at a high level, not everything about them. This drastically reduces what the model has to pay attention to. The information is still there; the model simply reads the relevant file when it needs to know more.

How to use a tool? Point to the tool’s documentation. Working in your wiki? Point to the wiki’s own agent instruction file.

The pointers can be organized hierarchically: the global file points to a certain directory that contains all your research projects; the agent file in that directory provides basic rules and “know-wheres”. Then each repo’s instruction file explains the project in details and points to other documents in the repo, and so on.

DRY principle#

Organized this way, the most fine-grained rules live in exactly one place and can be referenced from several—a single source of truth, the DRY principle (“don’t repeat yourself”) applied to instructions. My LaTeX style rules, for instance, sit in a single style file that the wiki, paper repos, and skills all point to—maintained once, discovered on demand.

Simplify, simplify#

Treat instructions as liabilities and prune them often, especially when a new model comes out. Many rules exist to patch the weaknesses of the model you wrote them for; a newer model may already handle those cases well, and the leftover rule becomes pure cost—or a straitjacket. Revisit the top-level files at each model upgrade and delete what the model now gets right on its own. This has been repeatedly observed. Many people reported that simply deleting the agent file and starting from scratch worked way better for Anthropic’s Opus 5 model. Even Boris Cherny, the creator of Claude Code, advises deleting your CLAUDE.md every six months and starting fresh.

Caveats#

Guardrails vs. knowledge#

Progressive disclosure is for knowledge, not guardrails. If you have few non-negotiables rules like “never rm” or “never push without asking”, it should not depend on the model reading the right file at the right time. Enforce them deterministically with permission rules and hooks, or keep them in the always-loaded file.

“When” cue#

Also, a pointer only works if the model knows when to follow it. Like a skill’s one-line description, each pointer needs a trigger. So the instruction should make it very clear when the model needs to read further.

Just a good engineering principle#

This is not new at all; it is basic software engineering (modularity, layered documentation, …) applied to agent context. It is also how Anthropic designed Agent Skills; they call progressive disclosure “the core design principle.” At startup only each skill’s name and one-line description enter the context; the full SKILL.md is read once the skill looks relevant; bundled reference files load only when a specific scenario needs them. Their analogy is a well-organized manual: table of contents, then a chapter, then the appendix.

Good information design works the same whether the reader is a human or a model: show a little first, and let the reader pull the rest.