Welcome to the latest issue of Engineering Enablement, a weekly newsletter sharing research and perspectives on developer productivity.
🗓 Join DX’s live research panel today as we introduce CAFE(S), a framework for improving AI agent effectiveness through better context. Register here.
This special issue marks the release of CAFE(S), a new framework for evaluating the quality of context assembled for AI systems. I had the privilege of co-authoring it with Max Kanat-Alexander (Capital One), Eirini Kalliamvakou (DX, formerly GitHub), Margaret-Anne Storey (University of Victoria), and Nicole Forsgren (Google).
It is out now in ACM Queue, and you can download the paper here.
Low-quality context now exacts a high price
For most of software’s history, knowledge management has been a major source of developer pain. Engineers have struggled with out-of-date documentation, unclear specifications, and conflicting requirements for decades. Poor context might have been a drag on developer experience, but that pain could be absorbed because a developer could go over and ask a peer for clarification.
What’s changed isn’t that context started mattering. It’s that the consequences now arrive immediately, at scale, and usually with a number attached.
Tokens. Every token sent to an agent is paid for, and context is where tokens accumulate. When Notion engineered its context for reuse through prompt caching, costs dropped roughly 90 percent with no loss in output quality, because reading cached context is far cheaper than retransmitting the same information.
Liability. In February 2024, the British Columbia Civil Resolution Tribunal found Air Canada liable for a refund its customer service chatbot promised and its actual policy did not provide. Air Canada argued the chatbot was effectively responsible for its own statements. The tribunal disagreed. The damages were modest. The precedent was not.
Security. EchoLeak (CVE-2025-32711) was the first documented prompt injection to achieve real-world data exfiltration from a production AI system. A single crafted email, never opened by the user, caused Microsoft 365 Copilot to pull internal documents into its working context and leak them. Untrusted information became part of the agent’s context, and the agent acted on it, faithfully.
Translating human intent for intelligent models
Our creative ideas aren’t automatically absorbed by intelligent models. They need to be translated before an agent can act on them. This simple pipeline describes how human intent becomes agent behavior.
Intent is what a person ultimately wants to accomplish. That intent must then be translated into context, which is the information that represents the task, its constraints, and the surrounding knowledge. A harness applies that context through orchestration, retrieval, tools, memory, and execution logic before passing it to a model, which supplies the underlying intelligence that produces a result.
Every layer matters, and three of them have serious disciplines behind them. Machine learning research improves models. Harness engineering improves how those models are orchestrated. Product and requirements work improves how we create and refine intent in the first place. The translation from intent into context is the one that gets relatively little attention, despite increasingly determining whether an otherwise capable agent succeeds or fails.
Some disciplines get close, but answer different questions. Knowledge management asks whether the information exists and is maintained. Information retrieval asks whether it can be found. Context engineering asks how to select and assemble it for a model. But none of those tell you whether the context itself is fit for the task at hand.
That’s the gap CAFE(S) is meant to fill.
The five properties
CAFE(S) names five properties of good context. The first four ask whether the context lets agents do good work. The fifth, set slightly apart in parentheses, asks whether the context is safe to use at all.
Clarity: can the agent interpret the request as intended? Clear context resolves the same way for the author and for the agent. When it is ambiguous, the agent may choose an incorrect interpretation, often silently, and every step after that compounds the choice.
Actionability: can the agent proceed, and know when it’s done? Actionable context states the objective, relevant constraints, and what “done” looks like. Without those, agents spiral into execution loops, stop early, or declare success against a target they inferred rather than one they were given.
Fidelity: can the agent trust that this context is true? Faithful context is accurate, current, internally consistent, and drawn from sources worth trusting. When it isn’t, agents reason confidently from stale guidance, or fill a gap with a guess that then becomes indistinguishable from fact.
Efficiency: can the agent focus on what matters? Efficient context carries a high signal-to-noise ratio. Irrelevant material doesn’t merely take up room, it competes for the model’s attention, which is how the one detail that mattered ends up lost in the middle. And every unnecessary token is paid for.
Security: should the agent have this context at all? Secure context is safe and appropriate for the agent to access and process. Failures here look like prompt injection, data leakage, and policy violations rather than merely disappointing results.
A fair question is why we stopped at five, when timeliness, cost, and provenance are all plausible candidates. Our bar was that a property earns its own pillar only if it isn’t derivable from the others, can be acted on independently, and names a failure the others miss. Most fold in cleanly. Stale context is context that’s no longer true, which makes timeliness a facet of fidelity, and cost is a downstream consequence of efficiency rather than a property of the artifact.
What’s left is genuinely independent. Context can be clear but not faithful, describing the system as it was rather than as it is. It can be efficient and faithful but not actionable, never saying what finishing looks like. A single bad instruction can fail several pillars at once, but each one names something you can reason about, review for, and fix on its own.
Security sits in parentheses because it evaluates the same artifact while asking a different kind of question about it. Failures of the first four make agents less effective. Failures of security make them unsafe.
Putting CAFE(S) into practice
Understanding the five properties is a start. The value comes from treating the context you hand to agents as a first-class engineering artifact, which means someone owns it, someone reviews it, and it gets maintained as the system changes.
First, the information must be accessible; the best context is useless if it is out of the agent’s reach. Second, the system must be capable of cutting through the noise to retrieve the exact slice of data needed across scattered systems, without exhausting token budgets or context windows. Finally, the knowledge must exist in the first place. No retrieval architecture or context engineering can surface the undocumented hallway decisions and unrecorded conversations that live only in people’s heads. Only after information is captured, connected, and retrievable do the properties of CAFE(S) become the operative standard for quality.
Once you’re past those, here are the highest-leverage moves against each property.
For clarity and actionability: introduce a review discipline for shared context files like AGENTS.md or checked-in specs. You wouldn’t merge unreviewed code. Review these the same way, and scale the scrutiny to how widely they get reused. A repository-wide file shaping thousands of sessions deserves far more attention than a working note between two engineers.
For fidelity: assign clear ownership and a review cadence to critical context sources, including runbooks, architecture decision records, and API schemas. Information decays as systems evolve, and an owner is what gets stale guidance pruned before an agent turns it into a confident mistake. Ownership should follow teams rather than individuals, so context survives reorgs.
For efficiency: put guidance where it’s actually used. The same words can be high quality in one location and harmful in another. An instruction about running the test suite belongs in a root AGENTS.md, because nearly every session needs it. An instruction about one rarely touched file does not, because it gets injected into thousands of sessions that have nothing to do with it. Move that instruction into a comment inside the file it describes and it becomes high quality, available exactly when it’s relevant. Nothing about the words changed. Only their location did.
For security: hold a hard boundary between trusted instructions and untrusted data. Emails, tickets, and pull request content are inputs, not instructions. Treating them as instructions is the mechanism behind the attack I described earlier.
The paper goes considerably further on what individuals, teams, and organizations can do to improve context quality along the dimensions of CAFE(S).
Final thoughts
CAFE(S) provides a definition of context quality, not a measurement system. Our goal is to give engineering teams a shared vocabulary to discuss context, review it as a first-class asset, and intervene before bad context causes failures. While empirical metrics and automated evaluations will follow, meaningful measurement first requires a precise definition of what high-quality context actually looks like.
As software development shifts toward greater human-agent collaboration, high-quality context becomes a fundamental prerequisite for agent autonomy. Clear, actionable, faithful, efficient, and secure context creates the baseline conditions where delegating complex work becomes reasonable, though never fully guaranteed. CAFE(S) doesn’t claim good context is sufficient to hand an agent the keys; it simply shows why, without it, you shouldn’t.
You can read the full paper here: CAFE(S): Your Agent Is Only as Good as Its Context
That’s it for this week. Thanks for reading.
-Brian
This is great, thank you!
The facts need to be correct and useful, not just plentiful. Old logs or missing release details can point an agent in the wrong direction. Which of the five parts is hardest to get right?