Runtime, state and memory
- date
- category
- AI Agents
- reading
- 2 min / 354 words
The Implementer Agent now runs the full flow:
ticket
|
v
clone repo
|
v
RAG
|
v
implementation
|
v
tools
|
v
tests
If everything finishes within one short session, things are simple.
But a real task can look like this:
implementation
|
v
tests fail
|
v
fix
|
v
needs approval
|
v
wait
|
v
continue
The agent has to keep continuity.
Managed runtime
The runtime is responsible for where and how the agent runs:
- endpoint,
- process/container,
- scaling,
- identity,
- sessions,
- basic tracing.
Examples:
| Runtime | Cloud |
|---|---|
| Microsoft Foundry Agent Service | Azure |
| Amazon Bedrock AgentCore Runtime | AWS |
| Vertex AI Agent Engine | GCP |
State
State describes the current task.
Example:
{
"issue": "PAY-123",
"repository": "payments-service",
"branch": "agent/PAY-123",
"step": "waiting_for_approval",
"testsPassed": true
}
This is what lets work resume.
Memory
Memory is information that is useful later too.
For example:
team payments requires contract-test before every PR
That is not the state of one task.
It is information that can be reused in later tasks.
Where we define what to store
Agent repo
Decides:
- which data makes up state,
- when to checkpoint,
- what is worth keeping as memory.
Platform
Provides the storage mechanism.
Tools on the market
| Solution | Cloud | Role |
|---|---|---|
| Foundry Agent Service state / memory capabilities | Azure | sessions and managed agent state |
| Azure Cosmos DB | Azure | durable state, memory and semantic recall under your own control |
| Bedrock AgentCore Memory | AWS | short-term and long-term memory |
| Vertex AI Sessions / Memory Bank | GCP | session state and long-term memory |
| your own layer | any | more control at the cost of more code |
When you do not need an extra database
Not every agent needs Cosmos DB or a separate memory store.
If:
- the task is short,
- it does not come back hours later,
- it does not need memory across sessions,
the runtime's session state may be enough.
A separate layer only shows up when we really need durability beyond a single session.
Materials
- Dynamic Workflows and Beyond Dynamic Workflows, on who tracks state when the LLM does not
- Amazon Bedrock AgentCore Memory
- Memory Bank in Gemini Enterprise Agent Platform
- Anthropic: Effective context engineering for AI agents