Smaller things you’ll find once it’s running.

God mode: Watch one key’s prompts and responses as they happen, from a dock on its page. Held in memory only, streamed only while the dock is open, and off unless you set VW_GODMODE_ENABLED.
Admin tokens: vwa_ tokens let scripts, CI and agents drive the control API without the admin password, a cookie jar or a CSRF header. Shown once, stored as a SHA-256 hash.
A HuggingFace cache manager: What is on disk and which model owns it. Garbage-collect orphans, and export or import the whole cache as a tarball.
One port: The UI, the control API and /v1 share :8080. Your own TLS terminator, ingress or SSO goes in front of it unchanged.
No route out needed: Three image tarballs and an optional model-cache tarball are the whole transport for an air-gapped host. HF_HUB_OFFLINE=1 stops even the weight downloads.
Router stats: Every decision on /router/stats: local, passed through, fell back, refused, with the reason, per rule and per target, and the breaker state. Per process, never a header or a body.
A setup page per client: Connect writes the setup for Claude Code, Codex CLI, OpenCode, Aider, Grok CLI, the Anthropic and OpenAI SDKs and three editors, with this warden’s address, a key and a loaded model, and sends a test request.
Codex CLI: Run against a warden on 5 October 2026 with Codex 0.160: shell tool calls, a file edit, the session kept on one replica. Thinking is on by default; model_reasoning_effort = "none" turns it off. Local models only: router rules do not apply.
A chat playground: Talk to a loaded model in the browser, images included, before you hand anyone a key.

Three things we learned the hard way.

The engine died four times in one day, and the proxy kept forwarding to it.

The vLLM engine core crashed while the vllm serve wrapper around it stayed up, so the process looked alive and requests went on being sent to it. Now a watchdog asks the engine’s own /health endpoint instead of the process table. When the engine stops answering, it keeps the evidence, including a bounded tail of the log, and reloads the model.

A model’s live log panel under load: vLLM API server lines answering GET /metrics, POST /v1/chat/completions from a client, engine lines reporting 10 requests running and 0 waiting, and one GET /health line, all returning 200 OK — **Fig. 15**The live log on a model’s page during the same load: scrapes of `/metrics`, chat completions coming in, the engine’s own count of 10 requests running and none waiting, and one `/health` probe between them. Scroll the image sideways to read the full lines.

A 12-minute load used to end in an error.

A large model can take twelve minutes or more to load. Until now the warden watched a fixed clock: at the deadline it wrote a red “still loading” error into the model, though the engine kept loading. The model page now shows every load step with its progress bar, the usual time for each from earlier loads, and the CPU and GPU-memory vital signs. A stalled load is stopped and its GPUs freed; nothing turns red while a load is still moving. GLM-5.2 NVFP4 at tensor-parallel 8 on eight NVIDIA B300s loads in about 11 to 12 minutes; a load that long used to hit the 600 s timeout.

The model page while GLM-5.2 NVFP4 loads on eight NVIDIA B300s: 9 min 17 s into a usual 10 min 46 s; Start engine, Load weights 47 of 47, Compile kernels and Size KV cache done; Tune kernels at op 2, 21 of 22; CPU and GPU-memory charts with 1948 of 2149 GiB in use; Capture CUDA graphs and Warm-up request still to come, each with its usual time — **Fig. 16**GLM-5.2 NVFP4 loading at tensor-parallel 8 on eight B300s, nine minutes in. Kernel tuning is the long step; the charts show the engine is alive while the log only repeats tuning progress.

A 16 GiB card holds less than the arithmetic says.

The weights of a bf16 8B model are about 15 GiB on their own, and it runs out of memory while loading. We tried each row below on 16 GiB RTX A4000 and Quadro RTX 5000 cards and judged it by reading the model’s answers in the chat playground, not by trusting a health check.

Model	What works	What does not
`openai/gpt-oss-20b`	Two cards, tensor parallel 2, `max_model_len` 32000, `gpu_memory_utilization` 0.7 to 0.9	One card: the 20B MoE doesn’t fit
Llama 3.1 8B, Mistral 7B class	One card, with an AWQ-INT4 checkpoint, `--enforce-eager` and `gpu_memory_utilization` 0.9	The bf16 repo: about 15 GiB of weights, out of memory during load
Two models at once	One model per card, each fitting its own card	Two models sharing one card

Checked against vLLM 0.20.0. The replica measurements above used 0.26.0. This release ships 0.32.0; the first load timed on it is GLM-5.2 NVFP4 at tensor-parallel 8 on eight B300s, about 11 to 12 minutes. The fit rules have not been re-measured on 0.32. The full table and the reasons behind it are in HAZARDS.md. Not on NVIDIA? Apple silicon has a Mac beta, Windows is next, and other GPUs are on the roadmap.

Smaller things you’ll find once it’s running.

Three things we learned the hard way.

The engine died four times in one day, and the proxy kept forwarding to it.

A 12-minute load used to end in an error.

A 16 GiB card holds less than the arithmetic says.

A 16 GiB card holds less than the arithmetic says.