RAG as shared organizational knowledge
- date
- category
- AI Agents
- also in
- Retrieval & Knowledge
- reading
- 2 min / 377 words
The Implementer Agent already has:
ticket
+
repo
+
AGENTS.md
That is enough when all the knowledge it needs lives in the repo.
But the ticket might say:
Implement retry according to the company resiliency standard.
And the standard might live in central documentation.
That is when the agent needs RAG.
RAG in practice
The most important mental model:
RAG is not a magic database the model "just knows what to fetch from". For an agent, RAG is usually a tool.
For example:
engineering_knowledge.search(query)
The tool has a description:
Use this tool when you need company-wide architecture
standards, ADRs or engineering documentation that is not
available in the current repository.
The agent sees:
task:
"implement retry according to company standard"
checks the repo and does not find the standard.
Then:
engineering_knowledge.search("retry standard")
|
v
RAG
|
v
relevant fragments
|
v
agent continues
How the agent knows which RAG to use
If the organization has several sources, it can expose several tools:
engineering_docs.search()
security_standards.search()
api_catalog.search()
Each one has:
- a name,
- a description,
- parameters.
The model picks a tool based on:
- the task,
- the system instructions,
- the tool description.
You do not need to code:
if task contains "security":
call security RAG
although for critical cases you can add more deterministic rules.
Where we configure it
Platform
Builds and maintains:
- data sources,
- indexes,
- embeddings,
- permissions,
- data refresh.
Agent repo
Declares which retrieval tools are available:
tools:
- engineering_docs.search
- security_standards.search
Product repo
Can carry an instruction:
If you need a company-wide standard,
use the engineering knowledge tool.
Tools on the market
| Solution | Cloud / type | Role |
|---|---|---|
| Azure AI Search | Azure | vector, keyword and hybrid search for RAG |
| Amazon Bedrock Knowledge Bases | AWS | managed RAG on Bedrock |
| Amazon OpenSearch | AWS | your own search/vector layer |
| Vertex AI RAG Engine | GCP | managed retrieval for agents |
| Pinecone | independent | managed vector database |
| Weaviate | independent | vector search / retrieval |
What the organization controls
- what counts as a knowledge source,
- who has access to which source,
- how fresh the data is,
- which metadata comes back,
- whether the agent sees the source of an answer.
In a local environment RAG helped a single agent find a document.
In an organization it becomes a controlled access layer to engineering knowledge.