rapid-mlx-dsh-provider

一句话定位

为 DeepSeek Harness (DSH) 添加本地 Rapid-MLX 模型路由，直接从服务端读取模型事实（上下文窗口、是否支持推理/工具调用等），免去手动维护 settings.yaml；并附带 5 个模型管理工具和 /rapid-mlx 总览命令，让 Agent 在会话内就能查看、下载、删除本地模型。

核心能力

• 自动发现模型元数据：注册时调用本地 Rapid-MLX 服务端的 /v1/models，把上下文窗口、推理/工具解析器、MoE/hybrid 架构、视觉能力等信息写进路由
• 优先使用内存拟合的容量上限：上下文窗口取服务端 max_model_len（按 Apple Silicon 统一内存估算的容量），老版本服务端回退到 context_window，确保 dsh-compaction-basic 压缩时机正确
• 让推理档位"说实话"：服务端报告 reasoning_parser: null 的模型不再展示 off/low/medium/high 选择器（选了也没用）
• 提供 5 个模型管理工具：rapid_mlx_serving（查看正在服务的模型）、rapid_mlx_cached（查看下载缓存）、rapid_mlx_pull（下载模型，可取消）、rapid_mlx_remove（删除缓存，强制 -y）、rapid_mlx_health（独立报告 API 和 CLI 健康）
• 注册 /rapid-mlx 总览命令：一行输出服务端/CLI 健康、当前服务模型事实、缓存总占用

技术实现

• 语言: JavaScript（ESM，纯 JS 无构建步骤）
• 关键依赖: @deepseek-ai/dsh-llm（LlmAdapter 基类与 LlmError 契约）、@deepseek-ai/schemastery（Config schema 校验与默认值）、@deepseek-ai/dsh-tools（defineTool 注册工具）、@deepseek-ai/dsh-subprocess（跑 CLI 的 subprocess seam）
• 架构模式: 通过 cordis.patch.yml 作为 profile bundle layer 注入；插件 inject ['llm', 'tools', 'subprocess', 'commands'] 四个服务，依赖 dsh.bundle 字段被 profile 加载；Config 用 schemastery 声明 + 环境变量回退（RAPID_MLX_BASE_URL / RAPID_MLX_CLI）
• 入口文件: lib/index.js（apply(ctx, config) 是 cordis 钩子入口）

适用场景

DSH 用户希望让 Agent 直接连本机跑的 Rapid-MLX 服务端（典型如 Apple Silicon 上的 MLX 模型）时使用本插件。最直接的痛点是：通用 openai-completions 路由让你必须手动填 baseURL、上下文窗口和推理档位，换模型还得改；而本插件从服务端拉元数据，无需维护就能跟随服务端切换模型，并让 dsh-compaction-basic 在合适的时机触发压缩。

前置依赖与兼容性

安装方式

dsh plugin --profile web add @raullenchai/dsh-provider

配置项

常见问题

Q: 安装后需要在 settings.yaml 里写模型配置吗？

A: 不需要写上下文窗口、推理档位、视觉能力这些字段了。只需要把 agent-default-model 的 provider 设为 rapid-mlx、model 设为服务端实际跑的模型名（短别名或 Hugging Face repo id 都行）。其他元数据由插件从服务端拉。

Q: max_model_len 和 context_window 有什么区别？

A: context_window 是模型训练时定的"理论最大上下文"；max_model_len 是服务端按当前机器的统一内存（权重 + KV 缓存）估算的"实际能塞下的上限"。插件优先用后者喂给压缩模块，避免压缩时机晚于机器实际能承载的长度。

Q: 能在非 Apple Silicon 机器上跑吗？

A: 插件代码是纯 JS、跨平台。但实际可用必须本机能运行 Rapid-MLX（MLX 后端限制 Apple Silicon）。在 Intel Mac 或 Linux 上安装插件本身没问题，只是没有可用的服务端连。

Q: 服务端响应慢或 502，插件会怎样？

A: stream() 路径下：HTTP 非 2xx 抛 LlmError（401/403 → 凭证错误码、429 → QUOTA、413 → 上下文超限、5xx → PROVIDER_ERROR）。/v1/models 拉取失败时静默返回空列表，让 DSH 走"未知模型"的分支而不是中断整个会话。

Q: 工具调用和思考链支持吗？

A: 支持。流式输出里 tool-call 增量按 argumentsDelta 拼接，完整 ToolCallBlock 在 block-end 一次性给出；reasoning_content 单独走 reasoning 通道，不混入正文文本。但图片内容会抛 UNSUPPORTED，不会悄悄丢弃。

Q: 怎么卸载？

A: dsh plugin --profile web remove @raullenchai/dsh-provider。移除后 rapid-mlx 路由和 5 个工具、 /rapid-mlx 命令一并下线，不影响你已有的 settings.yaml 内容。

Q: dsh 升级到新版本会不会坏？

A: 截至 rc.8，LlmAdapter 合约与 rc.7 字节一致，插件向上兼容。dsh 是开发者预览，演进很快，建议关注 cordis.patch.yml 的 inject 字段（当前 ['llm', 'tools', 'subprocess', 'commands']）和 Config schema 是否需要新增。

上手难度

入门 — 本地有现成的 Rapid-MLX 服务端跑着的话，dsh plugin add 一行即可使用，settings.yaml 只改 provider 和 model 两行。

已知问题与限制

• recommended_sampling（每模型推荐采样参数）只读取，未自动应用
• tool_call_parser 只读取，未用于"模型不支持工具调用时快速失败"，目前仍可能进入循环
• is_hybrid / is_moe / capabilities 只读取，尚未在路由层据此改变行为
• 真正的内存感知容量仍未接入：resolveModel() 目前返回服务端报告的 max_model_len，这本身已经是"按机器估算的"值，但更精细的"当前可用容量"需要 Rapid-MLX 侧先暴露 usable-capacity 字段
• 图片输入目前直接抛 LlmError('UNSUPPORTED')，不静默丢弃；如果需要看图，请改用通用 openai-completions provider
• 路由名固定为 rapid-mlx，如果你的 settings.yaml 已在 llm-pi-ai.providers 下声明过同名 provider，registerAdapter 的 provider 排他性会导致冲突——需二选一或重命名

A native Rapid-MLX provider for DeepSeek Harness — so dsh gets its model facts from the server instead of from whatever you typed into settings.yaml.

Status: published to npm as @raullenchai/dsh-provider. The end-to-end dsh run in Verified was on an M3 Ultra against dsh 0.1.0-rc.7; dsh 0.1.0-rc.8 is API-compatible — the LlmAdapter contract is byte-identical and the only changes are additive — and the adapter is re-verified against rc.8 at the protocol and unit-test level. DSH is still a developer preview that moves fast, so treat this as tracking a moving target, not a frozen compatibility promise.

What it does for you

DSH can already talk to a local Rapid-MLX server through its generic openai-completions provider. That route works — but it knows nothing about your model beyond what you hand-wrote:

# what the generic route makes you maintain, by hand, per model
llm-pi-ai:
  providers:
    rapid-mlx:
      baseURL: http://localhost:8000/v1
      defaultContextWindow: 262144      # you looked this up. is it still right?
      models:
        - id: qwen3.6-35b-8bit
          contextWindow: 262144
          reasoningEfforts: {off: none, low: low, medium: medium, high: high}

Rapid-MLX's /v1/models already publishes all of that and more. This adapter reads it, so:

1. Nothing to hand-write, and nothing to re-write when you switch models. Swap what rapid-mlx serve is running and dsh follows. No re-running setup, no stale numbers.

2. The reasoning control tells the truth. Rapid-MLX reports whether a model actually has a reasoning parser. A model that can't reason no longer shows an off/low/medium/high selector that does nothing.

3. Compaction is timed with the capacity that actually fits this Mac, not a number that drifted. This is the one that quietly costs you. dsh-compaction-basic asks the provider for the route's capacity and compacts at thresholdRatio × capacity (0.8 by default). The provider prefers the server's max_model_len — Rapid-MLX's memory-fitted ceiling (what fits in unified memory: weights + KV cache), in the vLLM/SGLang-standard field — over the native context_window, and falls back to context_window on an older server that doesn't report it. So compaction is timed to what the machine can actually hold, not the model's advertised window (which it may not have room for) and not a hand-written number copied from another model.

Install

Needs Node ≥ 22.15 (dsh imports Node's Zstd stream API without declaring it) and a running Rapid-MLX server.

# From npm:
dsh plugin --profile web add @raullenchai/dsh-provider

# …or straight from source — the package ships plain JS with no build step:
dsh plugin --profile web add github:raullenchai/rapid-mlx-dsh-provider

export RAPID_MLX_BASE_URL=http://localhost:8000/v1     # optional; this is the default
dsh web

Then point the agent at the route:

# $DSH_HOME/settings.yaml
agent-default-model:
  provider: rapid-mlx
  model: qwen3.6-35b-8bit

Verified: that command installs and activates as a profile layer against dsh 0.1.0-rc.7. To hack on it locally instead, see Local development.

Model management (v0.2.0)

Beyond the provider route, the plugin registers five tools and a /rapid-mlx command so the agent can see and manage models without leaving the session. The split follows which surface actually owns each fact: served-model facts come from the structured HTTP /v1/models; the download cache and pull/remove are CLI-only, so those — and only those — shell out to rapid-mlx through the harness subprocess seam.

/rapid-mlx prints a one-shot overview: health, the served model and its facts, and total cache disk usage.

The CLI is resolved from the cliCommand config (default rapid-mlx on PATH, or $RAPID_MLX_CLI); set it to an absolute path if the binary is not on the harness's PATH. rapid_mlx_pull/rapid_mlx_remove are the only tools that change anything on disk, and they run non-interactively (rm is forced with -y because the subprocess seam ignores stdin).

Verified

Against dsh 0.1.0-rc.7 on an M3 Ultra:

• Installs and activates as a profile layer (no "declares no dsh.bundle" warning; the entry shows up in dsh --profile headless --dump-config).
• Registers the rapid-mlx route with ctx.llm and serves real queries.
• Plain chat, a single tool call, and the multi-step bug-fix task that gates Rapid-MLX releases — the last one fixed the bug and made the target repo's own test pass, verified independently, in 36 s on qwen3.6-35b-8bit.

Not done yet

Being explicit, because the point of the adapter is to use what the server says and some of it is still only read:

• recommended_sampling — should be applied automatically per model.
• tool_call_parser — should let dsh fail fast on a model that cannot emit tool_calls, instead of looping.
• is_hybrid / is_moe / capabilities — read, not yet acted on.
• Memory-aware capacity. Today resolveModel() reports the model's advertised context window. On a Mac the real ceiling is unified memory, and reporting that instead is the biggest remaining win — it needs Rapid-MLX to expose a usable-capacity figure first.
• Images are not carried through stream() — but they now refuse with LlmError(..., 'UNSUPPORTED') rather than being dropped, per the cookbook. Text, reasoning and tool calls are carried.
• The route is registered as rapid-mlx. If your settings.yaml also declares a rapid-mlx provider under llm-pi-ai, the two compete for one route name (registerAdapter owns provider exclusivity). Use one or rename ours.

Conformance with the official adapter contract

Built against docs/cookbook/adding-an-llm-adapter.md and its "protocol obligations" section. Each item has a test:

finish.replayState is not emitted: Rapid-MLX needs no native response ids or signatures on follow-up calls, so there is nothing lossless to project.

Three things worth knowing before you edit this

Each of these cost real debugging time:

1. dsh.bundle in package.json is what makes this a plugin. Without it the package installs as an inert dependency and dsh only warns. It is also what gets it appended to the profile's dsh.profile.bundles. CI fails if it goes missing.
2. LlmReasoningEffortInfo.name is required. Returning {id} alone fails the whole model with INVALID_MODEL_REASONING — an error that names the model, not the missing field.
3. DSH has no tool role. Message.role is only system|user|assistant; a tool result is a user-role message whose source.kind === 'tool' carries the callId and whose content holds a ToolResultBlock. Flatten those into plain user text and the model reissues the same call forever — the symptom is an empty answer and a non-zero exit, with nothing on stderr.

Local development

pnpm links a local path outside the profile tree, so Node's parent-walk never reaches $DSH_HOME/profiles/node_modules and the peer deps fail to resolve. Symlink them in — dev only, node_modules is gitignored and excluded from the published files:

mkdir -p node_modules/@deepseek-ai
ln -sfn <dsh-install>/node_modules/@deepseek-ai/dsh-llm node_modules/@deepseek-ai/dsh-llm
ln -sfn <dsh-install>/node_modules/@deepseek-ai/cordis  node_modules/@deepseek-ai/cordis

export DSH_HOME=/tmp/dsh-dev          # never your real ~/.dsh
dsh plugin --profile headless add "$PWD"
export RAPID_MLX_BASE_URL=http://127.0.0.1:8000/v1
dsh --profile headless "say hello"

A real npm install needs none of this: the package lands inside the profile tree, where the flat fallback resolves bare names normally.

When testing agent behaviour, use a strong 8-bit model. A multi-step task here failed on qwen3.5-9b-4bit and passed on qwen3.6-35b-8bit — 4-bit confounds "weak model" with "broken integration".

The engine side guards these fields

Living in its own repo means a rename in Rapid-MLX would break this package silently — nothing there imports it and this CI does not run there. So the fields are pinned on the side that owns them, by tests/test_model_card_client_contract.py in Rapid-MLX, which names this package as its reason. It pins the wire shape: field names, nullability, and the fact that ModelInfo does not set exclude_none — which is what makes "reasoning_parser": null distinguishable from an older server that omits the key entirely.

If you start reading a new /v1/models field here, add it there too. Otherwise the guard silently stops covering what this package actually uses.

License

Apache-2.0, matching Rapid-MLX.