0.7. Troubleshooting
In one glance
- You will: Match your symptom and leave with the fix, instead of reading this page top to bottom.
- You need: Nothing beyond a terminal.
- Time: no reading time — this is a lookup page. Find your symptom and go.
This page is a symptom index for the failures the course’s commands produce: each row of the table below is fixed by the section with the matching heading.
Bookmark it and come back when a checkpoint fails — the chapter-ending command that decides whether you may go on. Match the symptom, run that row’s command, and read only that section. If nothing matches, run the doctor for your tier, the staged check for the tools one learning stage needs: mise run doctor, then doctor:model, doctor:gateway, doctor:platform, or doctor:gcp, plus mise run config:check.
What symptom do you have?
| Symptom | First command |
|---|---|
| A task exits with a “tool not found” error | mise run install |
| Configuration validation fails before startup | mise run config:check |
| Model calls refuse to connect immediately | mise run doctor:model |
| Every model turn dies at about 60 seconds | ollama ps |
| A service won’t start or a port is taken | ss -ltnp | grep ‘:3001’ |
| MCP tools return a 4xx only through the gateway | mise run config:check |
| Pods stay Pending or CrashLoop | kubectl -n agentops get events --sort-by=.lastTimestamp |
cluster:start or doctor:platform stops before creating anything | stat -fc %T /sys/fs/cgroup/ |
| You need to inspect measured Go coverage | cd agents/go && mise run coverage |
Why does mise run install or a task fail with “tool not found”?
Every tool is pinned in mise.toml and nothing installs on demand, so hooks and CI fail fast instead of drifting. Run mise run install for the learner toolchain, mise run install:platform before the gateway and Kubernetes chapters, or mise run install:maintainer for the full contributor set.
Why does the agent fail at startup with a configuration error?
Configuration is validated once at startup, so a bad combination fails early with a message naming the fix; mise run config:check from agents/go prints both. Two rules catch most cases. Remove the retired AGENT_GATEWAY_ENABLED: topology comes from OPENAI_BASE_URL, not from switching providers. And AGENT_MODEL_PROVIDER=openai-compatible needs a compatible base URL plus a non-empty marker in OPENAI_API_KEY — http://127.0.0.1:11434/v1 in Chapters 2 to 4, http://127.0.0.1:4000/v1 once agentgateway is in front.
A third rule is about scope: the root .env reaches only configuration and model-backed tasks, so editing it changes nothing about test, check, redteam, or mcp, which exercise the committed defaults on purpose. To reach a variable from the deterministic suite, set it in the test.
Why can’t the agent reach Ollama or the model gateway?
Two failures look identical and have opposite remedies; the refusal names which one. Nothing is listening at the configured model endpoint comes back in under a second: no provider is running. The model did not answer within AGENT_MODEL_TIMEOUT_S is the deadline expiring while the model worked. The model provider rejected the request is an error status from a provider that did answer. None prints the provider’s own reply, which can echo the prompt, so ollama ps and mise run doctor:model remain how you see the detail.
For a real connectivity failure run mise run doctor:model, then check OPENAI_BASE_URL against the path you are on: direct Ollama in Chapters 2 to 4, the gateway on :4000 in Chapter 5 after mise run gateway:host. Inside Kubernetes, pods reach host Ollama through the gateway, never through localhost — a pod’s loopback is its own container.
Why does every model turn fail at about 60 seconds on my CPU?
This is the most likely failure on the required path, not a bug in your setup. The shipped deadline is 60 seconds per attempt, sized for a warm GPU, and two retries follow, so the error takes three deadlines to arrive and the wait reads as a hang. A local Qwen3-4B on CPU takes seconds to minutes per turn, and a multi-tool investigation makes several such calls. Check what the machine is doing:
ollama psNAME ID SIZE PROCESSOR CONTEXT UNTIL
qwen3:4b-instruct 0edcdef34593 3.2 GB 100% CPU 4096 4 minutes from now100% CPU in the processor column is the whole diagnosis: the model is loaded and working, with no GPU to accelerate it. The 3.2 GB is not the download either — that is about 2.5 GB on disk, and this larger figure is the resident footprint once the weights are loaded alongside the context buffer, which is the number your available memory has to cover. Raise the deadline in the gitignored root .env:
AGENT_MODEL_TIMEOUT_S=180 # CPU-only laptop; 600 and upwards for a slow or memory-tight machineTime one turn before you tune, and set the deadline above your slowest observed turn rather than at a round number. The typed setting rejects anything past 3600, and a turn that never completes under an unbounded deadline is a different problem.
Why does the agent card request or a port bind time out?
Something already owns the port; 0.4. Ecosystem holds the full contract. The likeliest squatter is your own earlier terminal — a detached gateway, or a server left running two chapters ago. The second is a bare upstream command. ADK’s web launcher defaults to :8080, which here belongs to the A2A server, and Hugo’s server defaults to :1313, so mise run web and mise run serve pin them to :8002 and :8003, clear of :8080 and of the raw MCP server on :8000.
Inspect the wrapper with mise run gateway:host:status and gateway:host:logs, and stop a stale detached instance with mise run gateway:host:stop. Otherwise find the process with ss -ltnp | grep ':3001', or lsof -nP -iTCP:3001 -sTCP:LISTEN on macOS. One collision survives care: the host Compose stack publishes Tempo on :3200 and Loki on :3100, and Chapter 7’s Kubernetes port-forwards ask for those same host ports, so running both either refuses to bind or leaves you querying the wrong stack.
Coming back after a break, four things outlive the terminal that started them, and each has its own stop: mise run gateway:host:stop for the detached host gateway, mise run observability:down for the telemetry stack, k3d cluster stop local for the local cluster (6.8. Platform Operations), and Ollama’s own service for the model. None of them deletes state, so the next session resumes where you stopped.
Why does an MCP call through the gateway return a 4xx?
The status code names the cause: 401 or 403 is authentication, 404 is a route mismatch, 421 is a rejected host. MCP_ALLOWED_HOSTS must include the gateway’s hostname, because the MCP server answers only for Host headers on that list — the check that blocks DNS rebinding, where an attacker’s page reaches your loopback under a name it controls. The gateway route must also match the path you call, and where the gateway enforces authentication, set AGENT_MCP_TOKEN to a minted demo token (5.5. Gateway Security).
Why do Kubernetes pods stay Pending, CrashLoop, or fail their probes?
Read the events first with kubectl -n agentops get events --sort-by=.lastTimestamp. Three causes cover almost everything. The image cannot be pulled from the local registry, so build and push it with the local skaffold profile. Or readiness reports that the process cannot serve: the agent’s /healthz checks the seed dataset, writable state, and the session store rather than the port, because a replica that listens but cannot read them joins the Service and fails every request. Or a default-deny egress NetworkPolicy blocks the pod, which kubectl -n agentops describe networkpolicy confirms.
How do you inspect measured Go coverage?
cd agents/go
mise run test
mise run coveragemise run test has already failed if any package in that module fell below its 80% line-coverage floor. Read the report to find which lines are unexercised, not whether some are. Whether the exercised lines behave correctly is a different question — see 0.2. Evidence.
Deeper: a secret scan passed, and key material is still on disk
Detection is heuristic, not a statement that no private material exists: the scanner may not recognize every age-key encoding, and the secured-gateway directory and local .env files are deliberate exclusions because they hold expected runtime secrets. Both are git-ignored, so the gitleaks pass in mise run secure still covers everything Git holds. Tear lab material down with each lab’s own command, and never force-add it past the ignore rules.
Why does the platform doctor require cgroup v2?
Kubernetes 1.35 and later refuse kubelet startup on a cgroup v1 host. Both mise run doctor:platform and mise run cluster:start therefore stop before creating a partial cluster. Confirm the hierarchy:
stat -fc %T /sys/fs/cgroup/Expect cgroup2fs; a tmpfs result is cgroup v1. Enable your distribution’s unified cgroup-v2 hierarchy, reboot, and re-run the doctor. A kubelet compatibility flag will not save you, because Kubernetes removed cgroup v1 from its supported v1.35 path — the host prerequisite is the only durable fix. See About cgroup v2.
How to use this page later
- Come back when a checkpoint fails: the symptom table routes you to one section.
- Come back when a model turn is slow rather than broken, and let
ollama pssay which it is. - Come back when nothing matches, and run your tier’s doctor plus
mise run config:check.
Continue to 0.8. Glossary, which gives the one-line definition of the course terms these fixes use in passing.