LM Warden
Language: English
Documentation Download & Install

Grok CLI on your own GPUs.

xAI’s Grok Build CLI with LM Warden as a bring-your-own-key model.

Where a request from Grok CLI goes Grok CLI sends its requests to LM Warden with base_url set to the warden and env_key holding a warden key. LM Warden checks the key, speaks OpenAI Chat Completions, queues the request by key priority and hands it to the replica router, which keeps each session on the replica that already holds its cache, one model copy per GPU. Grok CLI base_url the warden, at /v1 env_key LMWARDEN_KEY holds vw_… POST /v1/chat/completions LM Warden checks the key: one per app speaks OpenAI Chat Completions scheduler, key priority usage, latency and live stats per key Replica router a session goes back to the replica that holds its cache one model copy per GPU your GPUs: vLLM or llama.cpp replica 0, GPU 0 replica 1, GPU 1 replica 2, GPU 2 a conversation stays on its replica
Where a request from Grok CLI goes Grok CLI sends its requests to LM Warden with base_url set to the warden and env_key holding a warden key. LM Warden checks the key, speaks OpenAI Chat Completions, queues the request by key priority and hands it to the replica router, which keeps each session on the replica that already holds its cache, one model copy per GPU. Grok CLI base_url the warden, at /v1 env_key LMWARDEN_KEY holds vw_… POST /v1/chat/completions LM Warden checks the key: one per app speaks OpenAI Chat Completions scheduler, key priority usage, latency, live stats per key Replica router a session goes back to the replica that holds its cache replica 0 GPU 0 replica 1 GPU 1 replica 2 GPU 2 your GPUs: vLLM or llama.cpp a conversation stays on its replica
Where a request from Grok CLI goes: the two settings Connect writes, the warden in the middle, then one copy of the model per GPU.

In words: Grok CLI sends its requests to LM Warden with base_url set to the warden and env_key holding a warden key. LM Warden checks the key, speaks OpenAI Chat Completions, queues the request by key priority and hands it to the replica router, which keeps each session on the replica that already holds its cache, one model copy per GPU.

Protocol
OpenAI Chat Completions
Support
Local models only

Local models only, over Chat Completions (api_backend = chat_completions). There is no pass-through to xAI, and a BYOK model needs no xAI login. session_summary keeps Grok’s session-title requests on this model; without it Grok asks this warden for grok-4.6 and gets a 404. Grok adds its own sections to config.toml when it runs. The community grok-cli (npm grok-dev) is a different tool: chat works, tool calls do not.

Requirements

  • Needs a model with tool calling (agentic edits).
  • Works best with at least 32,768 tokens of context.

Setup

In the console, Connect writes these files with your warden’s address, your key and the model you picked. Here they are with placeholders.

Grok Build (official)

A model entry in ~/.grok/config.toml; the key comes from LMWARDEN_KEY. Install: curl -fsSL https://x.ai/cli/install.sh | bash, or npm i -g @xai-official/grok

config.toml (~/.grok/config.toml)
[model.lmwarden]
model = "your-served-model-name"
name = "LM Warden · your-served-model-name"
base_url = "https://your-warden/v1"
env_key = "LMWARDEN_KEY"
api_backend = "chat_completions"
context_window = 32768

[models]
default = "lmwarden"
session_summary = "lmwarden"
# The model's context window is unknown: 32768 is a guess. Set it to the model's real window.
Shell
export LMWARDEN_KEY=vw_YOUR_KEY
grok -m lmwarden

Community grok-cli

The community grok-cli reads its base URL, key and model from the env and needs Bun installed. Chat works; tool calls fail, because its xAI SDK rejects tool-call chunks that arrive in parts, as vLLM streams them.

Shell
export GROK_BASE_URL=https://your-warden/v1
export GROK_API_KEY=vw_YOUR_KEY
export GROK_MODEL=your-served-model-name
npx -y grok-dev

Official documentation: https://docs.x.ai/build/settings/reference (read 2026-10-04).

Related: all coding agents · let your agent pick the model · API keys