dsh-approval-llm

One-Line Description

This plugin adds an "Auto-Approve" permission mode to DeepSeek Harness: in this mode, an independent review model replaces human judgment for approval/request, providing a three-value decision of ALLOW / DENY / ESCALATE; in other permission modes the plugin remains completely silent, and the human approval process is unaffected.

Core Capabilities

• In model-approval permission mode, the review model automatically determines approval/request, giving pass, reject, or escalate-to-human decisions
• Before decision-making, deterministic routing is performed via whitelist, blacklist, and human-only lists; matched requests skip model invocation
• On model timeout, parsing errors, or upstream failures, always escalate to human (fail-to-human), not counted toward consecutive denial circuit breaking
• After consecutive denials reach the threshold in a single session, automatically route subsequent requests back to human; the review model stops acting as judge
• During review, replay tool real parameters from session logs to avoid being misled by agent self-reports
• Bundled with configure-approval-llm skill; AI can write review configuration following a "suggest first, then confirm" workflow

Technical Implementation

• Language: TypeScript (ESM, strict mode tsconfig)
• Key Dependencies: @deepseek-ai/dsh-user-approval (approval outcome vocabulary), @deepseek-ai/dsh-permission-presets (preset services), @deepseek-ai/dsh-llm (streaming review calls), @deepseek-ai/schemastery (config schema validation)
• Architecture Pattern: cordis plugin, registered via inject = ['llm','tools','permissionPresets','skills'] and listens to approval/request events, processing via "deterministic routing → circuit breaking → review model" waterfall; inserts plugin line and model-approval preset via package.json#dsh.bundle.patch
• Entry File: src/index.ts (apply(ctx, rawConfig))

Use Cases

Developers who want to implement low-risk automation workflows like "let model review first, escalate to human when uncertain" in DSH. Suitable for scenarios requiring models to continuously execute autonomously in workspace-write mode while still retaining human oversight for high-risk tools like delete, terminal commands, job_kill; not suitable for unattended high-risk tasks.

Prerequisites & Compatibility

Installation

dsh plugin --profile web add github:Letter2025/dsh-approval-llm

Configuration Options

FAQ

Q: Can it work without configuring provider/model?

A: Yes. When not set, it falls back to the conversation routing from the last request/header in the session; if there's also no routing in session logs, review fails and escalates to human, it won't fake a decision.

Q: Does human approval stop working after installing?

A: No. The plugin only responds under the model-approval preset; other presets (including the default 请求批准) still go through the original human approval, behavior is identical to before the plugin was installed.

Q: What happens when the review model says "uncertain" or crashes?

A: Always escalate to human: timeout, parsing errors, upstream errors all produce ESCALATE and pass the request to the next answerer; these failures are not counted toward consecutive denial circuit breaking to avoid misinterpreting model failures as "repeated rejections".

Q: I'm worried the model might be fooled by tool outputs. What to do?

A: This is a known risk. Suggest tightening humanOnlyList (high-risk tools requiring human), denyList (blacklist), maxConsecutiveDenials (circuit breaking threshold), and avoid enabling in unattended high-risk scenarios.

Q: How to configure the review model after installation?

A: Have any agent load the configure-approval-llm skill or say "configure approval review model" to it; it will follow an AI-suggests-then-you-confirm workflow to write the approval-llm line into ~/.dsh/profiles/web/cordis.patch.yml, restart dsh web to take effect.

Q: How to uninstall?

A: Remove via the host DSH's plugin removal command and restart dsh web; the model-approval preset declared by the plugin is injected via bundle patch, after uninstall the preset will disappear.

Difficulty Level

Advanced — requires understanding DSH's permission presets, cordis patch mechanism, and model selection for review; built-in configure-approval-llm skill can reduce configuration work, but strategy tuning for whitelist/blacklist/circuit breaking still requires deployer judgment.

Known Issues & Limitations

• No dedicated machine-readable audit event type yet: review decisions appear as user/message in the main pipeline and can be persisted/replayed; waiting for host to open custom event registration channel before adding session/approval-llm-request events
• Client badge not provided yet (browser half): review decisions displayed as conversation messages, not as shield icon on tool cards
• One request equals one model call: batch review (PLAN-0063) requires approval service to first open batch entry; single request latency constrained by timeoutMs and selected review model speed
• Review model may be prompt-injected by tool outputs: configure humanOnlyList / denyList / circuit breaking parameters well, do not enable in unattended high-risk scenarios
• Lists only exact name matching: allowlist / denyList / humanOnlyList do not support wildcards or parameter pattern matching

English | 中文

Model-based permission approval (approve-for-me) for DeepSeek Harness.

A community plugin that adds a "model approval" permission mode (approve-for-me) to DeepSeek Harness: in that mode, approval/request asks are answered by a separate reviewer model instead of a human — the reviewer decides ALLOW / DENY / ESCALATE, and the request only reaches a human when the reviewer cannot decide or fails. In every other permission mode the plugin stays silent, so human approval is never front-run by the model.

It is the dsh equivalent of Codex's approvals_reviewer=auto_review (--approve-for-me), and it follows the review design of AGENTSCOPE-PLAN-058 / 062 / 063 (orthogonal reviewer dimension, three-way decision, routing policy, prompt isolation, fail-to-human, circuit breaker).

Warning: an AI reviewer is a policy choice, not a security guarantee. It can be fooled by prompt injection from tool output or the agent's own reason. Prefer it for low-risk workflows; keep humanOnlyList, the denyList, and the circuit breaker tight.

Design

How it works

approval/request (waterfall)
   │
   ├─ enabled? no ────────────────────────────────► next() (unchanged)
   │
   ├─ mode gate: session preset ≠ modePreset ─────► next() (human approval unchanged)
   │    (default modePreset = model-approval)
   │
   ├─ RoutingPolicy (deterministic, no model call)
   │    ├─ denyList hit ──────────────────────────► 'rejected'
   │    ├─ allowlist hit ─────────────────────────► 'allowed-once'
   │    ├─ humanOnlyList hit ─────────────────────► next() (human decides)
   │    └─ else: REVIEW
   │
   ├─ circuit breaker: consecutive DENY ≥ max ────► next() (human takes over)
   │
   ├─ Reviewer model (isolated security-policy prompt)
   │    input: tool name + description (ctx.tools)
   │          + reason + tool arguments (from the session log `tool/call`)
   │    decision: ALLOW ──────────────────────────► 'allowed-once' (counter resets)
   │              DENY ───────────────────────────► 'rejected' (counter +1)
   │              ESCALATE ───────────────────────► next()
   │              timeout / parse error / provider error ─► next() (fail-to-human)
   │
   └─ ALLOW / DENY also append a user-visible decision message to the session,
      so the main chain records why the call was approved or denied.

• ALLOW / DENY / ESCALATE map 1:1 onto the dsh outcome vocabulary (allowed-once / rejected / delegate). ESCALATE and model failures never fabricate a rejection — they hand the request to the next answerer (the human UI), and a deployment with no human answerer fails closed (unavailable), exactly like Codex's fail-closed guardian.
• The mode gate makes the modes exclusive. In the 帮我批准 preset the reviewer answers; in 请求批准 (and every other preset) the plugin delegates, so human approval behaves exactly as before. The two presets share sandbox/approval knobs; the recorded permission/preset selection tells them apart.
• One terminal answerer per deployment. The dsh approval chain is not a priority list of competing judges — compose one answerer. To keep human override, put a human UI answerer behind this plugin (the waterfall next() reaches it).
• Arguments are read from the session log, not from the approval request (the request deliberately carries no arguments to avoid a second rendering that could drift).

Configuration

All fields are validated by the Loader schema; defaults apply when omitted.

Example overlay (cordis.patch.yml of your profile):

- id: approval-llm
  config:
    provider: deepseek-official
    model: deepseek-v4-flash
    allowlist: [read, read_image, glob, grep]
    humanOnlyList: [delete, terminal_send]
    denyList: [job_kill]
    maxConsecutiveDenials: 3

Install

Copy-paste for an AI agent — hand this one sentence to any AI coding agent to have it install the plugin for you: "Read https://github.com/Letter2025/dsh-approval-llm/blob/main/README.md and follow its ## Install section to install the dsh-approval-llm bundle into the DeepSeek Harness web profile, restart the dsh web server, and verify that the permission selector shows the model-approval (帮我批准) preset with its shield-sparkle icon."

As an installable bundle (recommended)

This package declares dsh.bundle.patch in its package.json, so installing it activates a configuration layer that inserts the plugin row and adds the model-approval ("帮我批准") preset to the permission table — no manual preset config needed:

dsh plugin --profile web add dsh-approval-llm   # installs the published npm package

Restart dsh web, then pick 帮我批准 in the permission selector (the Access chip in the input bar, which carries a shield-sparkle glyph) to switch that session's reviewer to the model. The preset table is process-level, so changing presets requires a dsh restart.

In-box bundle rows resolve from the dsh installation itself; the @deepseek-ai/* imports are peerDependencies provided by the host dsh, so pin your dsh version (the project is in developer preview with breaking changes). Installing from a local checkout instead: pnpm run build, then dsh plugin --profile web add ./dsh-approval-llm from the parent directory.

Bundled skill: configure the reviewer

The package ships one bundled skill (configure-approval-llm, source bundled), so installing the plugin also puts a configuration guide in the skill catalog. Ask any agent to "configure the approval reviewer", or load the skill directly — it walks an AI-proposes / user-confirms flow: probe the current model and provider settings, write the approval-llm overlay into ~/.dsh/profiles/web/cordis.patch.yml, then present the full config for your confirmation before a restart takes effect. The guide covers choosing a reviewer model (same provider preferred, contextWindow ≥ the main model), and tightening allowlist / denyList / humanOnlyList / maxConsecutiveDenials for your deployment.

As a source overlay (dev)

- insert:
    - id: approval-llm
      name: './src/index.ts'        # path to this package's entry, or an absolute path
      config:
        provider: deepseek-official
        model: deepseek-v4-flash

Run dsh with the overlay (dsh web --patch ./cordis.patch.yml), or merge the row into your profile's cordis.patch.yml. The source overlay inserts only the plugin row, not the preset — either also install the bundle layer above, or add the model-approval preset to the permission row yourself (a patch replaces the whole row config, so restate every preset):

- id: permission
  config:
    presets:
      read-only:
        sandbox: read-only
        approval: ask
        name: 只读
      workspace-write:
        sandbox: workspace-write
        approval: ask
        name: 请求批准
      model-approval:
        sandbox: workspace-write
        approval: ask
        name: 帮我批准
        description: 审批由独立的评审模型决定；拿不准或模型故障时转人工。
      danger-full-access:
        sandbox: danger-full-access
        approval: never
        name: 完全放开

Build & test

The plugin lives inside the DeepSeek Harness checkout at custom_plugin/dsh-approval-llm; @deepseek-ai/* resolves against the checkout's own node_modules (built lib declarations + @types), so keep the harness built (pnpm run build at the repo root). The node_modules junction into the harness is provided by the checkout.

pnpm run typecheck   # tsc --noEmit (strict)
pnpm run test        # vitest: 39 unit tests, no network
pnpm run build       # tsc emit to lib/ (ESM, relative imports rewritten)

Roadmap

• Client-side badge & toggle: a browser half (dsh.client in this package) can render a shield icon on tool cards whose ask the reviewer decided, and a settings row that writes the plugin's enabled/modePreset to a hot-reloaded settings namespace. The host loader already discovers dsh.client packages from the same row, so the install path is unchanged.
• Batch review (PLAN-063): one reviewer call for several pending tools needs a batch entry point in the approval service.
• Wildcard / argument-pattern routing in the deterministic policy.

Security model

• Mode gating keeps human approval unchanged: outside the configured preset the plugin delegates every request, so 请求批准 behaves exactly as before the plugin existed.
• Prompt isolation: the reviewer prompt is assembled by this plugin from its own config; the main agent never sees the security policy, so it cannot tailor asks to pass review. Tool descriptions come from the live registry (ctx.tools.schemas()), arguments from the durable log — the reviewer judges the real call, not the agent's claim.
• Fail-to-human: TIMEOUT, PARSE_ERROR, and provider errors produce ESCALATE (delegate), never a fabricated denial, and are not counted in the circuit breaker (PLAN-058 §4.8 separation of model failure from policy decision).
• Fail-closed by composition: a deployment with no human answerer resolves unavailable, which callers treat as denial.
• Circuit breaker: maxConsecutiveDenials consecutive DENY on one session hands the rest of the session's asks to a human — the reviewer stops being the judge when it keeps saying no.
• Sensitive data: review input (tool arguments, reason) is used only for the reviewer request and structured logs; it is not stored beyond the ordinary session log and console output. Turn includeArgs off if arguments are sensitive.

Known limitations

• No dedicated log-only audit event. Model decisions are visible in the main chain as user/message notices (durable and replayable), and the built-in approval/asked + approval/decided pair records the ask/outcome. A dedicated machine-readable audit event (like a session/approval-llm-request with the full review context) is still blocked by the harness persistence policy for out-of-repo event types; when the harness ships a registration surface, this plugin should add one.
• No client-side badge (yet). The decision notice appears as a transcript message; a Codex-style shield icon anchored to the tool card needs a client plugin (the package ships no browser half yet). See the roadmap below.
• One request per model call. PLAN-063's batch review (one call for several pending tools) needs a batch entry point in the approval service; the dsh seam processes one request at a time. Per-request latency is bounded by timeoutMs and the reviewer model choice.
• AI-reviewer trust is a deployment decision. The reviewer can be prompt-injected through tool output. Keep humanOnlyList, denyList, and the breaker configured; do not enable this for high-risk, unattended workflows.
• Exact-name routing only. Lists match whole tool names; there is no wildcard or argument-pattern matching. Add patterns as a follow-up if needed.

License

MIT