让你的编程智能体选择模型。
为你的显卡选择模型，意味着要选择量化方式、引擎（vLLM 或 llama.cpp）、如何在多块 GPU 之间切分、给它多少上下文、KV 缓存类型，以及推测解码是否有帮助。每一次猜测都要付出一次下载和一次加载的代价。编程智能体可以在你做别的事情时替你完成这项搜索，因为控制台所做的一切都是 API 调用。
三步完成设置
- 在 GPU 主机上安装 LM Warden（命令在下载并安装中），并完成设置向导。
- 在 Settings → Access Tokens 中签发一个管理员令牌，有效期 30 天。在智能体使用的 shell 中把它导出为
VW_ADMIN_TOKEN，并把 warden 的地址导出为W。 - 启动你的智能体：Claude Code、Codex CLI 或其他能运行 shell 命令的智能体。它做这项工作时可以使用托管模型；它要调优的是你显卡上的模型。
智能体会用到哪些接口
| 步骤 | 端点 |
|---|---|
| 查看显卡 | GET /api/system/gpus |
| 下载前检查模型是否放得下 | POST /api/models/fit-preview |
| 注册，获取初始配置 | POST /api/models, GET /api/models/{model_id}/suggest-config |
| 拉取并加载 | POST /api/models/{model_id}/pull, POST /api/models/{model_id}/load |
| 测量真实的上下文上限 | POST /api/models/{model_id}/stress，然后 POST /api/models/{model_id}/stress/apply |
| 读取 warden 的数据 | GET /api/stats/v2/throughput, GET /api/stats/v2/latency |
| 发送测试流量 | 来自 POST /api/tokens 的推理密钥，用于 /v1/chat/completions |
智能体会先阅读智能体指南：这是一份为语言模型编写的简短说明，介绍这些调用以及要避免的错误。
可直接粘贴的提示词
把 https://your-warden 替换为你的 warden 地址，把大写的名称替换为你想用的模型。
1. 盘点硬件并筛选模型
Read https://your-warden/agent-guide.md and follow it. My admin token is in $VW_ADMIN_TOKEN and the warden is at $W. List the GPUs this warden may use. Then shortlist three open-weight models from Hugging Face that fit them, for agentic coding with tool calling. Check each with POST /api/models/fit-preview and keep only those that fit with at least 65536 tokens of context. Show me a table: model, quantisation, engine, GPUs, context, and why. Do not download anything yet.
2. 为一个模型选择引擎和切分方式
Read https://your-warden/agent-guide.md and follow it. Credentials are in $VW_ADMIN_TOKEN and $W. For the model MODEL_REPO, decide between vLLM and llama.cpp, and between tensor parallel and data parallel replicas on my GPUs. Use fit-preview for every layout you consider. Register and load the best two layouts one after the other, send each the same 20 streaming requests through an inference key you issue, and compare time to first token and tokens per second at concurrency 1 and 8. Keep the winner loaded, unload the other, revoke the key, and report the numbers.
3. 找出真实上限并应用
Read https://your-warden/agent-guide.md and follow it. Credentials are in $VW_ADMIN_TOKEN and $W. The loaded model is MODEL_NAME. Nobody else is using this warden right now. Run a quick stress run on it, wait for it to finish, read its capabilities and apply the measured context. Then benchmark it at concurrency 1, 4 and 16 with prompts of about 2k and 30k tokens, and give me a table of time to first token and tokens per second.
4. 在我的工作负载上比较两个候选模型
Read https://your-warden/agent-guide.md and follow it. Credentials are in $VW_ADMIN_TOKEN and $W. Compare MODEL_A and MODEL_B for my workload: the prompts in ./workload/*.txt. Load each in turn with its best layout, replay the prompts at concurrency 4, and record latency, tokens per second, and whether each answer is correct by your judgement. Write the results to ./model-comparison.md with a recommendation, and leave the recommended model loaded.
开始之前
- 管理员令牌拥有对 warden 的完整管理权限：模型、密钥和设置。目前还没有更细的权限范围。请给它设置较短的有效期。
- 压力测试会故意让引擎崩溃，以找出它的上限。请在没有其他人依赖的 warden 上运行这些提示词。
- 下载量很大。提示词会要求智能体在拉取之前先检查是否放得下，但请留意你的磁盘空间。
- 工作完成后，在 Settings → Access Tokens 中吊销管理员令牌。