5.0. Gateway

Why route the agent’s connections through one reverse proxy

A gateway here is a reverse proxy: one process in the request path, in front of shared backends. It holds the traffic decisions each connection would otherwise carry for itself — which model endpoint answers, which tools are callable, how fast a caller may push, what gets logged, and who is allowed in.

The agent from Chapters 2-4 opened those connections itself, so each decision lived in application code and was correct exactly once. The second process that talks to the same backends copies them, and the copy is where they drift. A control enforced once at a shared boundary cannot be forgotten by the next client — the same argument 5.5. Gateway Security makes for tool allowlists and rate limits.

This page maps each listener to its protocol and upstream, names the four configuration keys that produce them, checks that the security contract is identical in a laptop lab and on a cloud cluster, and draws the line at three controls that must not move to the gateway at any price.

The next page starts the agent with two variables that name gateway ports and no backend addresses:

AGENT_MCP_URL=http://127.0.0.1:3000/mcp
OPENAI_BASE_URL=http://127.0.0.1:4000/v1

The MCP server listens on :8000. Ollama listens on :11434. Neither number appears: the agent no longer knows where its tools or its model actually live, and something else decides what is behind those two ports.

A gateway does not replace the agent; it splits responsibilities. agentgateway owns traffic: transport, routing, per-listener rate limits, prompt guards, tool authorization, and structured access logs, metrics, and traces. ADK keeps owning application logic: sessions, the agentic loop, human confirmation, and the transaction that writes an action together with its audit record.

The diagram below labels agentgateway a data plane — the code on the request path that moves and inspects each call — as opposed to the control plane, the separate system that configures it.

    flowchart LR
    Client[A2A client / web UI] -->|A2A| GA2A
    subgraph GW["agentgateway data plane — named gateways plus shared routes and policy"]
        GA2A["gateway a2a :3001"] --> RA2A["route a2a"]
        GMCP["gateway mcp :3000"] --> RMCP["route mcp"]
        GLLM["gateway llm :4000"] --> RLLM["route llm"]
    end
    RA2A --> ADK
    subgraph ADK["ADK application (:8080) — confirmation, transactions, audit"]
        Loop[agentic loop]
    end
    Loop -->|MCP tools| GMCP
    Loop -->|OpenAI-compatible| GLLM
    RMCP --> MCP[MCP server :8000]
    RLLM --> Model[Ollama Qwen3 / Vertex Gemini]
    GW -. JSON logs, metrics :15020, OTLP .-> Obs[(logs / Prometheus / traces)]
  

Diagram in words: The A2A client enters the named a2a gateway and route. The ADK loop returns through separate named MCP and model gateways, whose routes apply traffic policy before their upstreams. Agentgateway emits traffic telemetry, while confirmation and writes stay inside ADK.

This course runs a data plane only. agentgateway reads a static, checked-in YAML file — infra/agentgateway/host/config.yaml, the secured config-auth.yaml beside it, and their two Kubernetes siblings — and serves it. There is no xDS stream, no dynamic service discovery, and no hot policy push: to change a route or a limit you edit the file and restart the process. Do not read “gateway” as “mesh”; the value here is a readable boundary and its policy, not dynamic fleet management. Any term on this page you do not recognize is defined in one line in 0.8. Glossary.

Which port carries which protocol, and what every profile shares

Each protocol gets its own listener, plus two operational ones the host wrapper injects:

Two of those names matter later: streamable HTTP is MCP’s HTTP transport, which lets the tool server be a separate process, and the OpenAI Responses API is the /v1/responses schema ADK’s model adapter speaks.

Separate ports keep routing unambiguous, because a catch-all rule is exactly how one protocol ends up at the wrong backend. That is why the smoke test asserts each listener by speaking its protocol rather than by opening a TCP connection and calling it proof. The last two rows appear in no config file on the host: the wrapper’s render step injects statsAddr and readinessAddr at start time, which 5.1. Gateway Setup takes apart line by line.

Those ports come from named gateways and top-level routes, and every later page in this chapter reuses that vocabulary. Reading the file’s keys top to bottom: gateways are the named entry ports; routes are forwarding rules, each attached to one named gateway through its own gateways list; policies are the ordered controls on a route — rate limits, authorization, prompt guards, CORS; and backends is the upstream a route reaches once every policy has passed.

The named gateways and the head of the MCP route make that relationship concrete:

gateways:
  mcp:
    port: 3000
  a2a:
    port: 3001
  llm:
    port: 4000

routes:
  - name: mcp
    gateways: [mcp]
    policies:
      # Only the six allowlisted reads reach this bounded retry policy, so it
      # cannot replay a confirmed write. The application does not retry remote
      # MCP calls, preventing stacked retry amplification.
      retry:
        attempts: 2
        backoff: 100ms
        codes: [502, 503, 504]
      timeout:
        requestTimeout: 5s
        backendRequestTimeout: 2s
      localRateLimit:
        - maxTokens: 120
          tokensPerFill: 120
          fillInterval: 60s

The mcpAuthorization and backends blocks continue under that same top-level route, and 5.2. MCP Gateway owns what they decide.

Three profiles ship that same shape: host/ for local processes, k3d/ for Kubernetes service DNS with a local Ollama upstream, and gke/ for Vertex AI with ambient workload identity. Those three directories hold four checked-in files, because host carries the secured config-auth.yaml next to its default. What stays invariant is the security contract, and you can read it out of the files yourself rather than trusting this paragraph:

for profile in host k3d gke; do
  ports=$(yq -r '[.gateways[].port] | sort | join(",")' "infra/agentgateway/$profile/config.yaml")
  tools=$(yq -r '[.routes[] | select(.name == "mcp") | .policies.mcpAuthorization.rules[]] | length' "infra/agentgateway/$profile/config.yaml")
  mode=$(yq -r '.routes[] | select(.name == "mcp") | .backends[].mcp.failureMode' "infra/agentgateway/$profile/config.yaml")
  printf '%-4s ports=%s tools=%s failureMode=%s\n' "$profile" "$ports" "$tools" "$mode"
done

host ports=3000,3001,4000 tools=6 failureMode=failClosed
k3d  ports=3000,3001,4000 tools=6 failureMode=failClosed
gke  ports=3000,3001,4000 tools=6 failureMode=failClosed

Same three ports, same six allowlisted tools, same fail-closed posture in a laptop lab and in a cloud cluster. Fail closed means the MCP backend denies a call whose policy or target check it cannot trust, rather than forwarding it. The per-listener request buckets (120/60s MCP, 60/60s A2A, 30/60s model) hold across profiles too. What changes is only what the environment forces: the upstream addresses, the model identity, whether callers must authenticate to :4000, whether the gateway exports traces, how it authenticates to a cloud backend — and the shape of the model listener’s prompt guard, which is the one control that could not be made identical.

The model route carries a second bucket, and on this one route the word “token” means two different things. Beside the 30-requests-per-minute limiter sits a maxTokens: 100000 entry declaring type: tokens, which meters model tokens rather than calls — the most AI-native control agentgateway has. Its timing matters: agentgateway charges that bucket after the call completes, so an oversized request is admitted and drains it, and the next call is the one refused with 429. A cap on spend is therefore a cap on the caller after the expensive turn, never on the expensive turn itself. 7.3b. Cost Governance owns that bucket, quotes it from this file, and has you set a per-caller budget and watch a refusal arrive.

A control belongs at this boundary when it is about traffic and applies to every caller uniformly: MCP tool authorization and a fail-closed backend, per-listener request rate limits, A2A protocol-aware forwarding, model request and response prompt guards, upstream model authentication at the deployment-identity boundary, and structured access logs, metrics, and tracing.

Validation, confirmation, and audit stay in the agent

Argument validation, human confirmation, and the write-plus-audit transaction stay in the application, and the reason is not caution — it is capability.

The gateway sees bytes on a connection. It cannot reconstruct an authenticated ADK agent.Context, the object a tool reads its caller, session, and approval from. It cannot decide whether a specific write was approved by a specific human, and it cannot hold a database transaction open across an action and its audit row. Push those to the boundary and you have not moved a control, you have replaced it with a weaker one that looks the same in a config file.

    flowchart TD
    Q{What does the control need?} -->|bytes on a connection| GW[agentgateway]
    Q -->|session, approval, transaction| APP[ADK application]
    GW --> GWc["rate limits, MCP allowlist, prompt guards, access logs"]
    APP --> APPc["Policy.ValidateActions validates actions<br/>RequireConfirmation function tools<br/>RestartServiceWithAudit / ResolveIncidentWithAudit transaction"]
  

Diagram in words: Connection-level controls belong to agentgateway; approval context, policy-plugin action validation, and atomic write-audit transactions remain in the application.

The rule generalizes past this chapter: a gateway is a policy point, not the only one. If a control’s failure would let an unapproved write through, that control lives in the application, never in a regex or an allowlist at the edge. 4.5. Guardrails owns all three; this page only draws the line.

func (t *Tools) validatedApproval(ctx agent.Context) (approval, error) {
	if ctx == nil {
		return approval{}, errors.New("the action must run through an ADK confirmation flow")
	}
	confirmation := ctx.ToolConfirmation()
	if confirmation == nil || !confirmation.Confirmed {
		return approval{}, errors.New("the action has not been confirmed")
	}

	// Session() is one of the accessors ADK stubs out inside a tool context, so
	// the identity is read from SessionID()/UserID(), which are not.
	identities := []struct{ label, value string }{
		{"approver identity", ctx.UserID()},
		{"session id", ctx.SessionID()},
		{"invocation id", ctx.InvocationID()},
	}
	var missing []string
	for _, identity := range identities {
		if strings.TrimSpace(identity.value) == "" {
			missing = append(missing, identity.label)
		}
	}
	if len(missing) > 0 {
		// All of them, not the first: an operator fixing a broken client wants the
		// whole list in one round trip.
		return approval{}, fmt.Errorf("the confirmed action is missing %s", strings.Join(missing, ", "))
	}
	approvedBy := strings.TrimSpace(ctx.UserID())
	networkPrincipal, networkState := principal.Network(ctx)
	switch networkState {
	case principal.NetworkUnauthenticated:
		return approval{}, errors.New("a network action requires an authenticated principal in addition to confirmation")
	case principal.NetworkAuthenticated:
		if networkPrincipal.Subject() != approvedBy {
			return approval{}, errors.New("the authenticated principal does not own the invocation")
		}
	}

	rationale := rationaleFrom(confirmation.Payload)
	if rationale == "" {
		return approval{}, errors.New("the approval carried no rationale")
	}
	if !fitsRationale(rationale) {
		return approval{}, fmt.Errorf(
			"the approval rationale exceeds %d characters", domain.MaxAuditRationaleLength,
		)
	}
	// Redaction runs before the row is written, because the audit trail is
	// append-only: a credential that lands there cannot be edited out afterwards.
	rationale = t.redact(rationale)
	// And the bound is re-checked afterwards, because a redactor rewrites rather
	// than deletes — "<EMAIL_ADDRESS>" is longer than most addresses it replaces.
	if !fitsRationale(rationale) {
		return approval{}, fmt.Errorf(
			"the redacted approval rationale exceeds %d characters", domain.MaxAuditRationaleLength,
		)
	}

	return approval{
		approvedBy:   approvedBy,
		rationale:    rationale,
		sessionID:    strings.TrimSpace(ctx.SessionID()),
		invocationID: strings.TrimSpace(ctx.InvocationID()),
	}, nil
}

Why the default profile is not a public security edge

The default course profile is not a public security edge. The honest list of what it lacks is short: end-user authentication, TLS termination, a distributed rate-limit store, multi-replica high availability, public ingress, and a trained content classifier. 5.5. Gateway Security adds an opt-in local JWT, API-key, and TLS profile while keeping the frictionless default, and it is equally blunt that the regex prompt guards are demonstrations rather than injection prevention. No profile in this repository creates an Ingress, a LoadBalancer, or a public endpoint; clients reach the gateway over loopback on the host and through kubectl port-forward in Kubernetes.

This chapter gives you a working mechanism rather than a hardened edge, a distinction 0.2. Evidence defines once for the whole course.

What you can do now

Before starting any process, check the profile contract with a command rather than with your eyes. Predict first: this gate re-derives every claim on this page against every profile, so when it agrees, does it print nothing, one line, or dozens?

mise run check:infra

Dozens. Seven of those lines are the real agentgateway binary reading a configuration file and approving it — the four checked-in profiles, plus the three variants the host wrapper renders from them:

Configuration is valid!
Configuration is valid!
Configuration is valid!
Configuration is valid!
Configuration is valid!
Configuration is valid!
Configuration is valid!

Around those, in the same run, scroll Kubernetes manifest validation, a backup-and-rollback drill, an openssl check on the lab certificate, and OpenTofu’s thirteen tftest cases. A gate that narrates is easier to trust than a silent one, because you can see which of its parts actually ran instead of inferring it from an exit code.

What it asserts is everything this page claimed: the same three protocol ports, the same six MCP allowlist entries, the same fail-closed posture, the same token buckets and prompt guards, and an OTLP destination appropriate to each environment. The allowlist comparison is worth knowing in detail. It compiles and runs compose.MCPReadToolNames(), the allowlist the agent’s own MCP client pins, and demands the gateway rules match it name for name; a separate Go test proves the server registers that same set. Gateway rules, client pin, server registration — three copies of one list, checked by machine, because an invariant a human re-checks by eye is one that drifts the week nobody looks.

• You can say which of :3000, :3001, and :4000 carries MCP tools, A2A clients, and OpenAI-compatible model calls.
• You have traced gateways → a route’s gateways attachment → policies → backends in infra/agentgateway/host/config.yaml.
• You can explain why a control enforced once at a shared boundary beats the same control copied into every client.
• You can name the three controls that stay in the agent, and say what the gateway would have to see in order to hold them.

Continue to 5.1. Gateway Setup when you can say what each of the three ports carries without looking it up — by way of 1.2. Container Engine if you deferred it, because the next page’s first command runs the gateway in a container.