The pattern is consistent enough to predict. A retrieval pilot goes well on a curated set of documents. It is pointed at the real corpus. The answers get worse in a way nobody can quite characterize, and the team starts tuning — chunk sizes, embedding models, reranking, prompts.
Some of that helps at the margin. None of it addresses the reason, which was fixed before the project started.
Retrieval surfaces what your content actually is. If the corpus cannot support a correct answer, no configuration produces one — it produces a confident answer instead, which is worse.
One: a document can be identified as current
Most enterprise corpora contain multiple versions of the same document, in multiple systems, with no reliable marker of which is authoritative.
A human resolves this without noticing. They see the 2019 policy, recognize the format as old, know the team reorganized since, and discard it. Retrieval has none of that. It sees three documents of similar relevance and surfaces all three, and the model synthesizes across them — producing an answer that is a blend of the current policy and two superseded ones, with no indication that blending occurred.
This is the single most common cause of retrieval failure in enterprises and it is not fixable at the retrieval layer. Either the corpus carries reliable currency metadata, or something upstream decides what is in scope.
Two: the answer exists in written form
Retrieval finds recorded things. A striking proportion of what people ask about was never recorded.
The exception process everyone knows. The reason the thresholds are what they are. Which of two contradictory policies actually governs. These live in people, and the fact that they are undocumented is usually invisible until a system is asked to operate without them.
Test this before building. Take twenty real questions from the intended users, and for each one, find the document that answers it. The proportion you cannot find is your ceiling, and it is frequently between a third and a half.
Three: permissions survive the pipeline
Documents in enterprise systems carry access controls. Documents in a vector index frequently do not, or carry a flattened approximation applied at ingestion and never refreshed.
The failure is quiet and serious. Someone asks a reasonable question and receives a synthesized answer drawing on a document they were never entitled to read. No access log records it as an access, because from the system’s point of view no document was opened.
This has to be designed at the start. Retrofitting permission-aware retrieval onto an index built without it generally means rebuilding the index, and the discovery usually happens after the pilot has been shown to executives.
Four: the corpus has a boundary
The instinct is to index everything, on the reasoning that more content means better answers. The opposite holds. Every irrelevant document is a competitor for retrieval slots, and volume dilutes precision.
The corpora that work are curated: a defined set, with an owner, and a rule for what enters. The corpora that fail are everything the connector could reach — which includes the draft nobody deleted, the personal notes in a shared folder, and the deck from the strategy that was abandoned.
The sequence that follows
| Before you build | What it tells you |
|---|---|
| Take twenty real user questions and find the answering document by hand | Your realistic ceiling. If you cannot find half of them, the project is a documentation project. |
| Check how many of those twenty have superseded versions in the corpus | Whether you need a currency mechanism before anything else. |
| Ask who owns each source system and whether permissions are exportable | Whether permission-aware retrieval is feasible or has to be designed around. |
| Define what is in the corpus and who decides | Turns an unbounded ingestion problem into a bounded one with an owner. |
A content project with an AI budget
Most failed retrieval projects were content projects that were funded as AI projects, and the mismatch is why they stall. The team has a model budget and a data problem, and the data problem belongs to people who were not in the room.
The twenty-question exercise costs a day and will tell you more than a three-month pilot. If most answers are findable in current, permissioned, identifiable documents, you have a retrieval project and it will probably work. If they are not, you have something else — still worth doing, considerably slower, and requiring people the AI budget does not currently pay for.
Knowing which one you have before you start is the difference between a project that ships and one that gets quietly rescoped.
This is a practitioner-facing sequencing argument rather than a research summary. It names no products and makes no performance claims.
Related Reading
- Enterprise Search & RAG Platforms
- Vector Database & AI Search
- Enterprise Content Management (ECM)
- Data Governance & Catalog Platforms
- An executive guide to open-weight models
- Most of your agent projects should be workflows
- Ten Google engineers explained why ML systems rot. Most MLOps advice still ignores them.
- From Models to Systems: How Enterprise AI Actually Gets Built
- Data Governance at Scale: Building a Federated Governance Operating Model
Independent. No sponsorships. Unsubscribe anytime.