Continue on your own GPUs.

Continue’s OpenAI provider pointed at this warden, one served model.

Where a request from Continue goes: the two settings Connect writes, the warden in the middle, then one copy of the model per GPU.

In words: Continue sends its requests to LM Warden with apiBase set to the warden and apiKey holding a warden key. LM Warden checks the key, speaks OpenAI Chat Completions, queues the request by key priority and hands it to the replica router, which keeps each session on the replica that already holds its cache, one model copy per GPU.

Protocol: OpenAI Chat Completions
Support: Local models only

Local models only, over Chat Completions. Router mode would need custom headers Continue does not document.

Requirements

Works best with at least 32,768 tokens of context.

Setup

In the console, Connect writes these files with your warden’s address, your key and the model you picked. Here they are with placeholders.

Local models

A model entry in ~/.continue/config.yaml.

config.yaml (~/.continue/config.yaml)

name: LM Warden
version: 0.0.1
schema: v1
models:
  - name: your-served-model-name
    provider: openai
    model: your-served-model-name
    apiBase: https://your-warden/v1
    apiKey: vw_YOUR_KEY
    roles: [chat, edit, apply]
    capabilities: [tool_use]
    useResponsesApi: false   # names like o*/gpt-5* would otherwise use /responses
    defaultCompletionOptions:
      contextLength: 32768
      maxTokens: 8192
# The model's context window is unknown: 32768 is a guess. Set it to the model's real window.

config.yaml (~/.continue/config.yaml)

name: LM Warden
version: 0.0.1
schema: v1
models:
  - name: your-served-model-name
    provider: openai
    model: your-served-model-name
    apiBase: https://your-warden/v1
    apiKey: vw_YOUR_KEY
    roles: [chat, edit, apply]
    useResponsesApi: false   # names like o*/gpt-5* would otherwise use /responses
    defaultCompletionOptions:
      contextLength: 32768
      maxTokens: 8192
# The model's context window is unknown: 32768 is a guess. Set it to the model's real window.

Official documentation: https://docs.continue.dev/reference (read 2026-10-04).