6.9. Scale Out

Why a second replica breaks a conversation on SQLite sessions

A conversation lives in a session store. It records every user message and model reply, keyed by a conversation id, and the runtime reloads it before each turn. By default that store is a SQLite file inside AGENT_STATE_DIR, opened in agents/go/cmd/agent/session_store.go with a pool of exactly one connection. That is the correct design for one process — 2.4. Sessions explains why the single connection is load-bearing rather than lazy — and it is a conversation-shredder for two, because the file belongs to one pod’s disk. A follow-up routed to the second process reaches an agent that has never seen the conversation, and nothing reports it: no crash, no alert, nothing to grep for. A file is not a shared database just because two pods can both open it.

This page moves sessions onto PostgreSQL and runs one conversation across two independent processes, sizes the connection pool against the server’s usable budget, scales the read plane — the MCP server and its six read tools, as distinct from the agent that writes — to two replicas behind an autoscaler, and measures what the second replica bought.

Adding a replica takes the right lever. kubectl scale deployment/agentops-agent is a no-op with extra steps: nobody wrote that Deployment, the kagent controller renders it from infra/kagent/agent.yaml, and as 6.3. Platform Agents puts it, reconciliation converges the edit away without an error. Raise the declared intent instead:

kubectl --namespace agentops patch agent.kagent.dev/agentops-agent --type merge \
  --patch '{"spec":{"byo":{"deployment":{"replicas":2}}}}'

kagent reconciles and a second pod schedules. Nothing fails, and the two replicas now hold two unrelated session files.

Put it back before you read on. Two agent pods against one ReadWriteOnce claim is two writers of the runtime state, which is the arrangement the rest of this page argues against — and the namespace quota budgets thirteen pod slots for twelve workload and surge pods plus the nightly backup Job, so the extra replica is the slot your next rolling update needs:

kubectl --namespace agentops patch agent.kagent.dev/agentops-agent --type merge \
  --patch '{"spec":{"byo":{"deployment":{"replicas":1}}}}'

Run one conversation across two processes on shared PostgreSQL

Start a PostgreSQL server, then two agents that share nothing except that server — different ports, different state directories, different processes:

docker run -d --name agentops-sessions \
  -e POSTGRES_USER=agentops -e POSTGRES_PASSWORD=local-postgres -e POSTGRES_DB=sessions \
  -p 55432:5432 \
  postgres:18.3-alpine@sha256:54451ecb8ab38c24c3ec123f2fd501303a3a1856a5c66e98cecf2460d5e1e9d7

In two terminals, each exporting the same two settings and nothing else in common:

export AGENT_SESSION_BACKEND=postgres
export AGENT_SESSION_DSN='postgres://agentops:local-postgres@127.0.0.1:55432/sessions?sslmode=disable'
cd agents/go
AGENT_A2A_PORT=8090 AGENT_STATE_DIR=.state-a go run ./cmd/agent a2a
AGENT_A2A_PORT=8091 AGENT_STATE_DIR=.state-b go run ./cmd/agent a2a

go run rather than mise run a2a here on purpose: the task loads the repository .env, and this experiment is about the two variables you just exported rather than the ones your workspace already had.

The same store exists as a cluster fixture. infra/k8s/exercises/sessions-postgres.yaml is the Service, StatefulSet, and NetworkPolicies form of that docker run, kubeconform-validated by mise run check:infra and applied by no overlay: it has no backup, no restore drill, and a password in plain sight, which an exercise may have and a deployment may not. Read it once beside the command above and the difference between a database you started and a database somebody operates becomes a list of the things this file is missing.

Give replica A a phrase to remember, carrying a contextId you choose — that id is the key the store files the conversation under, so both requests must use the same one:

curl -s -X POST http://127.0.0.1:8090/ -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":1,"method":"message/send","params":{"message":{"role":"user","parts":[{"kind":"text","text":"Remember this exactly: the canary phrase is SCALEOUT-CANARY-42."}],"messageId":"m-1","contextId":"scaleout-proof-001","kind":"message"}}}' \
  | jq '{contextId: .result.contextId, state: .result.status.state, text: (.result.artifacts[0].parts[0].text // .result.status.message.parts[0].text)}'

{
  "contextId": "scaleout-proof-001",
  "state": "completed",
  "text": "Understood. I will remember the canary phrase \"SCALEOUT-CANARY-42\" for future reference."
}

Those three fields are a projection, not the wire format: what comes back is a full A2A task envelope — task id, artifact ids, message history, ADK metadata, timestamps — and the jq filter keeps the three this page’s claim depends on. 3.6. A2A shows the envelope whole. If the first turn comes back failed, the model timed out rather than the store; raise AGENT_MODEL_TIMEOUT_S and send it again.

Now the follow-up, to the other process, with the same contextId and nothing else in common. Predict the answer before you read it: replica B has its own empty state directory and has never seen the first message, so either the conversation lives somewhere both processes can reach, or B answers with nothing.

curl -s -X POST http://127.0.0.1:8091/ -H 'Content-Type: application/json' \
  -d '{"jsonrpc":"2.0","id":2,"method":"message/send","params":{"message":{"role":"user","parts":[{"kind":"text","text":"What was the canary phrase I asked you to remember?"}],"messageId":"m-2","contextId":"scaleout-proof-001","kind":"message"}}}' \
  | jq '{contextId: .result.contextId, state: .result.status.state, text: (.result.artifacts[0].parts[0].text // .result.status.message.parts[0].text)}'

{
  "contextId": "scaleout-proof-001",
  "state": "completed",
  "text": "The canary phrase you asked me to remember is \"SCALEOUT-CANARY-42\"."
}

Replica B answered with a phrase that was only ever typed at replica A. Look at what the two of them left behind; the query runs inside the PostgreSQL container, so it needs no client beyond the Docker you already have:

docker exec agentops-sessions \
  psql -U agentops -d sessions \
  -c "SELECT count(*) FROM sessions;" \
  -c "SELECT author, left(content::text, 52) AS content FROM events ORDER BY timestamp;"

 count 
-------
     1
(1 row)

     author     |                       content                        
----------------+------------------------------------------------------
 user           | {"role": "user", "parts": [{"text": "Remember this e
 agentops_agent | {"role": "model", "parts": [{"text": "Understood. I 
 user           | {"role": "user", "parts": [{"text": "What was the ca
 agentops_agent | {"role": "model", "parts": [{"text": "The canary phr

From a host that has psql installed, psql "$AGENT_SESSION_DSN" -c ... reads the same rows over the port you published, using the DSN already exported in that shell.

One session row, not two: the second process found the conversation instead of starting a new one. Four event rows in order, and each replica wrote two of them — A the first pair, B the second. That ordering also decides what a failed turn leaves behind: ADK’s runner appends the user event to the session service before it runs the agent at all, so a turn that ends failed still leaves the question behind for whichever replica picks the conversation up next.

Which state moves to PostgreSQL, and which stays on disk

Three settings do all of this, under the 1.4. Providers rules that govern every other setting — parse at the boundary, fail at startup, never at the first turn:

AGENT_SESSION_BACKEND=postgres    # sqlite (default) or postgres
AGENT_SESSION_DSN=postgres://...  # required by postgres, refused beside sqlite
AGENT_SESSION_MAX_CONNS=10        # connections one replica may hold

AGENT_SESSION_DSN is a Secret, so mise run config:check prints ********** and every error this store raises names the variable rather than the value. A connection URL carries a password and a container log carries forever; the tests in agents/go/cmd/agent/session_store_test.go assert that a refused connection, an unreachable server, and a DSN the driver rejects all fail without the credential in the message.

AGENT_SESSION_MAX_CONNS decides how far you can scale, and it is the one usually left at its default. The ceiling that matters is not this replica’s pool but the server’s total, which is smaller than the number normally quoted. Ask the container you just started:

docker exec agentops-sessions \
  psql -U agentops -d sessions \
  -c "SHOW max_connections;" \
  -c "SHOW superuser_reserved_connections;"

PostgreSQL ships 100 and 3. Those three slots are held back so an administrator can still log in to a server that has run out, which means ordinary clients — every agent replica — divide 97 between them, not 100. So the budget is (max_connections - superuser_reserved_connections) / AGENT_SESSION_MAX_CONNS, and the setting is really a statement about fleet size: at the shipped ten connections per replica, nine replicas fit with room to spare at 90 of 97, and the tenth is where it breaks. It does not break cleanly. The tenth replica starts, passes readiness, and answers turns on the seven connections still available; it is the eighth, wanted under load, that comes back sorry, too many clients already — as does every healthy replica’s next connection, because a pool grows on demand rather than at startup. Multiply against the usable budget before you scale, and note that the naive division is optimistic by a whole replica.

Setting AGENT_SESSION_BACKEND=sqlite while AGENT_SESSION_DSN is set is refused rather than ignored, because a DSN that looks configured while every session still lands on local disk is the same outage as before with a more convincing dashboard.

Sessions are the only state that moves. These stay exactly where they were, and each has a reason:

The audit seed is not moving to PostgreSQL, not now and not in a production port of this design. It is immutable committed input that every evaluation and every test measures against, and the moment it becomes a writable shared table, the thing your evidence is anchored to can drift underneath it.

Readiness, not sessions, was the first thing standing in the way of a second agent replica. a2aserver/probe.go opened runtime.db in AGENT_STATE_DIR unconditionally, so an agent configured for PostgreSQL served turns correctly and reported unready forever — a pod that works and never joins the load balancer. The probe now asks the backend it is actually using, resolving ADK’s four tables through information_schema on the PostgreSQL side and keeping the dedicated read-only file handle on the SQLite side. Point an A2A server at PostgreSQL and it answers:

curl -s localhost:8090/healthz

{ "status": "ready" }

Drop a column ADK needs — ALTER TABLE events DROP COLUMN content — and the same route returns 503 with the cause in the log rather than on the wire. Readiness is still a real question; it is no longer a question about a file.

Two bindings remain, and they are why the agent is not yet a fleet:

1. a2aserver/taskstore.go — the A2A task store is a per-replica SQLite file, so a tasks/get that lands on the replica which did not create the task finds nothing. Sessions are shared; tasks are not.
2. The a2aserver package documentation names the A2A process the single writer of runtime state: it recovers an interrupted restore, publishes and migrates the incident database at startup, and two of those against one claim is two writers.

So: sessions are shared and proved, readiness follows the backend, and the task store is next. Which half is measured and which is projected is the distinction 0.2. Evidence owns.

Scale the read plane to two replicas and measure the result

The MCP server is the workload replication was invented for. It opens the state volume read-only, publishes nothing, migrates nothing, and answers six read tools — so two of them are two identical readers rather than two writers racing. infra/k8s/overlays/scale is that change, layered on the local overlay rather than folded into it:

kubectl kustomize infra/k8s/overlays/scale | \
  yq 'select(.kind == "Deployment" and .metadata.name == "agentops-mcp") | .spec.replicas'

2

A fourth overlay instead of an edit to base is deliberate. The local overlay is the taught path and its ResourceQuota is sized to exactly what Chapter 6 runs; doubling a workload there changes the pod budget for every learner who never opens this page. Keeping it separate also makes the whole scale-out one readable diff: diff <(kubectl kustomize infra/k8s/overlays/local) <(kubectl kustomize infra/k8s/overlays/scale).

The overlay adds a HorizontalPodAutoscaler (HPA) with minReplicas: 2, because an autoscaler whose floor is one is a restart policy wearing a costume, and a ceiling of four, because the state claim is ReadWriteOnce and every replica therefore lands on the node that already holds the volume. Scaling past what one node can serve buys pending pods, not throughput. The anti-affinity rule is preferred, never required, for the same reason: the provisioner pins the volume’s node as a hard filter, so a required rule would make the second pod unschedulable while a preferred one costs nothing today and spreads correctly the day the claim becomes ReadWriteMany.

CPU is the HPA’s signal for this workload and must not be for the agent. The MCP server’s cost really is processor time. The agent’s latency is dominated by model inference happening in another process entirely, so its CPU sits low exactly when it is slowest, and an autoscaler reading that would remove replicas under load.

Now measure it: a replica count is a claim, a percentile is evidence. The scenario talks to the raw Go MCP server rather than the gateway, which isolates the reader and its SQLite file from everything else. Start one replica in its own terminal:

cd agents/go
MCP_HOST=127.0.0.1 MCP_PORT=8000 MCP_TRANSPORT=streamable-http go run ./cmd/agent mcp

From the repository root, run the shipped k6 scenario — k6 is a load generator that holds a fixed request rate and fails the run if a declared threshold is missed. RATE is tool calls per minute and defaults to 60:

MCP_URL=http://localhost:8000/mcp mise run load:mcp

Each run ends with a thresholds block and a totals block, printed here from the first heading down to the check totals; the per-metric tables that follow are cut:

  █ THRESHOLDS 

    http_req_duration{op:tools_call}
    ✓ 'p(95)<250' p(95)=6.96ms

    http_req_failed
    ✓ 'rate<0.01' rate=0.00%

    mcp_rate_limited
    ✓ 'count==0' count=0


  █ TOTAL RESULTS 

    checks_total.......: 188     3.133195/s
    checks_succeeded...: 100.00% 188 out of 188
    checks_failed......: 0.00%   0 out of 188

Sixty tool calls per minute against a 250 ms budget, answered in 6.96 ms at the 95th percentile. Predict what happens at a hundred times that rate before you run it — most people expect the SQLite reader to be the wall:

MCP_URL=http://localhost:8000/mcp DURATION=30s RATE=6000 mise run load:mcp

The threshold line out of that summary, on the same machine:

http_req_duration{op:tools_call}
✓ 'p(95)<250' p(95)=4.86ms

One replica served a hundred read-tool calls per second inside the same budget, across 9,008 checks with none failed. Now do it with two. Start a second server on another port and drive both at once, one k6 run per terminal:

cd agents/go
MCP_HOST=127.0.0.1 MCP_PORT=8001 MCP_TRANSPORT=streamable-http go run ./cmd/agent mcp

# terminal one
MCP_URL=http://localhost:8000/mcp DURATION=30s RATE=6000 mise run load:mcp

# terminal two, started at the same moment
MCP_URL=http://localhost:8001/mcp DURATION=30s RATE=6000 mise run load:mcp

Two hundred calls per second in total, and both runs stayed inside the budget: p(95)=6.15ms and p(95)=5.93ms, 9,008 checks each, zero failures on either side.

The shipped gateway rate-limits this listener to 120 requests per minute — the token bucket 5.2. MCP Gateway configures — which is one fiftieth of what a single replica already handles on a laptop. At the rates this course teaches, replication buys availability, not throughput — a rolling update with no gap, and a node drain that costs latency instead of an outage. For a throughput story from this stack, look at the model, where every second of an agent turn actually goes.

Two claims above are unproved and must not be repeated as if they were measured: no run on this page happened on a Kubernetes cluster, so the autoscaler has never been observed adding a replica, and the Service’s round-robin across two pods was never exercised — two processes were driven directly, one k6 run each. The overlay’s rendering, schema validation, and lint are proved; its behavior on a live cluster is yours to confirm.

Your turn: delete one replica’s state directory mid-conversation

Prove the session store is genuinely shared by taking away the only thing the two processes could have been sharing: their disks.

Predict first. If you delete replica A’s entire state directory between the two turns, will replica B still know the phrase? The answer tells you which store the conversation lives in.

• Mode: temporary experiment.
• Goal: run one conversation across two agent processes with PostgreSQL sessions, destroy one replica’s local state mid-conversation, and confirm the other replica still answers from the shared store.
• Files to touch: none tracked. Working from agents/go, the experiment creates the two disposable state directories .state-a and .state-b plus a container named agentops-sessions; all three are removed at the end.
• Preflight: from agents/go, confirm each target is absent rather than inherited — test ! -e .state-a, test ! -e .state-b, and docker ps -a --filter name=agentops-sessions listing no container. Stop rather than reusing a container or a state directory you did not create here.
• Steps: start the container and both agents as shown above. Send the canary message to :8090. Then delete the first replica’s disk with rm --recursive --force -- .state-a and send the follow-up to :8091.
• Gate that proves completion: the second turn returns the canary phrase, and psql shows exactly one row in sessions with four rows in events. Then set AGENT_SESSION_BACKEND=sqlite on both agents, repeat the same two turns, and watch the second replica answer without the phrase — that contrast, not the first result alone, is the proof.
• Final state: docker rm --force agentops-sessions and rm --recursive --force -- .state-a .state-b, then mise run check and mise run test from agents/go pass with no tracked file changed.

What you can do now

• You can say what carries one conversation across two processes: a shared contextId on AGENT_SESSION_BACKEND=postgres, one sessions row, not two.
• You can size a connection pool against a server’s usable budget — max_connections minus the superuser reservation — before adding a replica, instead of after.
• You can name the two bindings that still keep the agent at one replica, and show the readiness route answering ready against a PostgreSQL session store.
• You can drive one MCP replica at 100 read-tool calls a second, then two, read your own p(95) against the 250 ms budget, and say what replication bought: availability, not throughput.

“Runs at scale” is two claims here rather than one: you can demonstrate the half that is real, point at the exact file that blocks the other half, and tell the two apart in someone else’s architecture diagram.

Continue to 6.0. Platform for the chapter’s port and workload inventory, or on to 7. Observability, where the replicas you just added each start emitting their own traces.