Peter D. BethkeSiteShell.netSpring Hollow Publishing, Inc.

An agent that can answer anything can be trusted about nothing

Bounding a corpus, not widening it, is what makes an agent testable — and the difference between context that is retrieved and context that is injected is the whole of it.

EXECUTIVE SUMMARY

The instinct when an agent gives a poor answer is to give it more data. That instinct is usually what caused the poor answer, and it quietly removes your ability to tell good answers from bad ones.

Three things break when the corpus is wide. Anything reachable becomes answerable, so the corpus is your permission model whether you designed it as one or not. Conflicting sources get resolved silently, because nothing in a pile of data says which source wins. And an agent with unbounded reach cannot be evaluated at all, because you cannot enumerate what it might have drawn on.

The fix is a distinction it's easy to lose track of: context an agent can retrieve is not the same as context that is injected into its instructions. The first is evidence and anyone may contribute to it. The second is authority and should require review. Keeping them separate is what makes an agent testable — and it is testable, with a specific experiment described at the end.

THE TEMPTATION

Every agent project reaches the same moment. It answers well on three sources, so someone connects a fourth. Then the data warehouse. Then the wiki, the ticket tracker, the shared drive. Each addition is individually reasonable, each one demos beautifully, and the trajectory is always the same direction: more.

The instinct is understandable. A model that can see everything feels like it should be able to answer anything, and when it fumbles a question the obvious remedy is that it must have been missing something.

It usually wasn't. Most bad answers I have chased were caused by the last source somebody added, not by the one they were about to add next.

THREE WAYS BREADTH FAILS

The first is the one people expect, and it is the least interesting. Anything an agent can reach, it can be asked about. Whatever your intranet says about who may see the compensation file, the agent's corpus is the real permission model, and it was assembled by whoever was wiring up connectors that week.

The second is the one that costs the most and gets discussed the least. Point an agent at your governed revenue table and also at a year-old export of the same numbers sitting in a shared folder, and it has no way to know which one is canonical. It will not raise an error. It will pick one, or blend them, and it will explain the result in exactly the same confident register either way. Nothing in a wide corpus expresses precedence. You know which table is authoritative because you know your organization, and none of that knowledge is in the data.

The third matters most to anyone who has to sign off on the thing. An agent whose reachable world is unbounded cannot be evaluated, because you cannot enumerate what it might have used. You end up assessing whether the model seems sensible rather than whether the system is correct. That is a vibe, not a test, and it does not survive contact with an auditor.

Which leads somewhere counterintuitive. Narrowing a corpus usually improves the answers. I spent the past few weeks building an application-level access-control model on Sigma, where every agent was deliberately grounded on a narrow, permission-scoped view of the data, and the constraint did not degrade anything. It made the agent correct, and it made it possible to state precisely what it could and could not know. A wide corpus does not make an agent smarter. It makes it unfalsifiable.

GIVE THE CORPUS A NAME

All of which raises the practical question the advice usually skips. Bounding a corpus sounds like discipline, and discipline does not survive a quarter. What does it actually look like to build?

The move that worked for me was to stop pointing agents at data and start pointing them at a named, governed view of it. In Sigma that view is an object — a defined set of rows and columns, with visible lineage back to the tables underneath, that a person can open and read. The agent is grounded on that, and only that.

Three things follow from the corpus being an object rather than a list of connections.

Precedence gets settled before the model is involved. The view is derived from the table you declared canonical, so the question of which source wins is answered upstream by whoever owns the model, not silently at inference time by a system with no way to know.

The corpus becomes countable. It has a row count, and a row count can be asserted. Every claim in that project is checked against a SQL model computing the expected answer independently, so "this identity can see 31,500 rows and no others" is a test rather than a belief. You cannot do that with a folder.

And scope can vary by identity. The same definition, written once, resolved to 31,500 rows for a store manager and 94,500 for a regional manager — and every agent, page and dropdown in the application reads through it. One boundary, a different corpus per person, because it is expressed in the data rather than in a prompt asking the model to be discreet. Nothing was ever refused. The rows were not there to refuse.

One caveat, stated plainly, because the rest of this piece asks you to be exact. The demo I published runs on Sigma's free tier, which uploads files rather than querying a warehouse. What it proves is narrower than what I am recommending: that scope can be expressed as data and checked against an independent model. Warehouse-backed governed views, writable tables and a queryable history are subscription features, and the approach above depends on all three.

That is the difference between bounding a corpus and intending to. Adding a source becomes a modeling change with a review trail attached, rather than a connector somebody switched on during a sprint.

RETRIEVED IS NOT INJECTED

Here is the distinction that resolves most of this, and it is rarely named out loud.

Some context an agent retrieves. It searches, finds something relevant, reads it, and treats it as evidence — one input among several, weighed against the rest.

Other context is injected. It is loaded automatically at the start of every run and lands in the instructions, where it functions as authority. The agent does not weigh it. It obeys it.

These are different objects with different trust requirements, and they are routinely stored in the same place, in the same format, and managed by the same process. That is the whole problem in one sentence.

Contributing to what an agent can retrieve is a low-stakes act. Someone adds a document, it becomes findable, and the agent may or may not use it — and when it does, the material is evidence that can be checked against other evidence. Contributing to what an agent is instructed with is an entirely different privilege. It is writing standing orders that execute unread.

A WORKED ANSWER

In one of my projects I built an audit gate where a herd of AI models shares one persistent, searchable memory. Shared memory is exactly the thing people warn about, but in this case it works because entries are tiered by provenance rather than by usefulness.

Lessons the herd discovered and a human then promoted are marked shared and injected into new instructions. They have been read by a person who decided they should carry authority. Humans hold the gate. Once a human has approved a lesson, it's merged back into the shared corpus.

The other tier stops short of that. Anything from a source we don't control is labeled with where it came from and stays in the reference pile. Agents can read it if they go looking. It never becomes an instruction, however sensible it sounds. So an outside document can inform an answer, but it cannot change how the agent behaves — and that holds because of how the system is built, not because someone remembered to be careful.

The promotion step is the entire design. Not the storage, not the search, not the model. Something moves from findable to authoritative only because a person decided it should, and that decision is recorded. Knowledge grows the way code grows: through review. Code review is the trust gate for knowledge exactly as it is for code.

WHO IS ALLOWED TO WRITE TO IT

Which turns a question that sounds administrative into a governance one. Who can add to what your agent reads?

In most organizations the answer is "rather a lot of people, and nobody decided that." A shared drive anyone can drop a file into. A wiki page anyone can edit. A document a supplier emailed over. None of that is unreasonable on its own — it is simply how companies work. It becomes a different question the moment those documents start shaping what an automated system tells your staff, in a confident voice, at scale.

The distinction above is what makes it manageable rather than alarming. Anyone may contribute to what the agent can look up, because look-up is evidence and evidence gets weighed. Adding to what the agent is instructed by should carry someone's name, the way a policy change does. Most organizations already have that review process. They have not yet noticed that a document can now behave like one.

YOU CAN TEST THIS

The part I care about most is that none of the above has to be a matter of opinion.

Take a document your agent can look up but that should never carry authority. A wiki page, a supplier's file, something collected from a public website. Put a sentence in it that instructs rather than informs — harmless, unmistakable, and written the way a real attempt would be: guidance about how the agent ought to behave, dressed as ordinary content.

Then ask the agent something unrelated, and watch whether it behaves differently.

If it does, you do not have two tiers. You have one tier and a naming convention. If the planted sentence stays look-up-able and never changes the agent's behavior, the boundary is real — and you can now say so with evidence instead of with an architecture diagram.

Run it again whenever the plumbing changes, because this is exactly the sort of boundary that degrades quietly. Nothing breaks when it stops working. The agent simply becomes a little more agreeable to whatever it happens to read.

The test is cheap. It takes an afternoon. I have not met many teams who have run it.

WHAT TO ACTUALLY DO

Give the agent less than you want to. Start from the smallest set of data that answers the question you are actually being asked, and add a source when a specific question fails — not in anticipation of questions nobody has asked yet.

Say which source wins, and say it in the data rather than in a meeting. That is what a governed view is for: it derives from the table you have declared canonical, so precedence is settled by the person who owns the model instead of silently by a system that has no way to know.

Separate what the agent can look up from what instructs it, and make the second require a human decision with a name attached.

And test that separation rather than trusting it, because a boundary nobody has tried to cross is not a boundary. It is an intention.

None of this is exotic, and none of it is really about AI. It is the same discipline your organization already applies to who may approve an invoice, change a price, or sign off a policy — applied to a new kind of input that happens to arrive looking like prose. The reason it feels unfamiliar is only that the thing carrying the authority is a paragraph.