一句话定位

在 DeepSeek Harness 的审批链上挂一个第二模型，让沙箱外的工具调用先由一个只读子 agent 读上下文并返回允许/拒绝；失败默认拒绝，全程记录会话日志。配套风险等级策略、拒绝熔断器和 Web 可视面板。

核心能力

• 第二模型审批：在 approval/request answerer 链上注册一个钩子，命中 ai 策略的工具交给只读子 agent 裁决，其它请求一律走 next() 放行给原有人类审批链。
• 结构化裁决：审查子 agent 通过结构化输出返回 { decision, reason, riskLevel }，只有 read/glob/grep 这三个只读工具，不会真正执行被审查的动作。
• 失败默认拒绝：审查超时、崩溃、子 agent 拉不起、verdict 结构不对都走 fallbackPolicy（默认 rejected），不会被悄悄放行；可改为 delegate（交给人类）或 allow-once（一次性放行，文档明确标注为危险）。
• 风险等级 + 熔断器：允许裁决附带的 riskLevel 超过 maxAutoAllow 时可配置 delegate 或 deny；同一回合连续拒绝或滑窗拒绝过多时熔断，触发 delegate / reject / abort-turn 三种动作之一。
• deny 原因回流到模型：被拒的工具结果里会注入 [auto-review] / [auto-review-fallback] / [auto-review-never] 标记 + 拒绝理由 + 防绕过提示，模型能看到为什么被拒、为什么不能换个姿势重试。
• 完整审计 + Web 面板：autoReview/state、autoReview/verdict、autoReview/rejection、autoReview/circuit、autoReview/override 五类事件写入会话日志；Web profile 多一个会话头 AI Review 按钮，展示开关、预算、累计统计、最近裁决和一次性批准。

技术实现

• 语言: TypeScript（ESM，type: module）
• 关键依赖: @deepseek-ai/cordis（插件宿主框架）、@deepseek-ai/schemastery（配置 schema + 默认值）、@deepseek-ai/dsh-session + @deepseek-ai/dsh-session-projection（会话日志 + Web 面板数据通道）、@deepseek-ai/dsh-subagent（审查子 agent 编排）、zod（projection wire schema 校验）
• 架构模式: 走 Cordis 函数插件契约（name + inject + apply），通过 ctx.on('approval/request', ...)、ctx.on('tools/post-execute', ...)、ctx.commands.register(...) 三个 effect 注册副作用；通过 ctx.inject(['sessionProjections']) 在 host 支持时挂载 autoReview 投影单元供 Web 面板读取
• 入口文件: src/index.ts（插件导出），src/runtime.ts（审批回答器 + 命令），src/review.ts（审查子 agent 编排），src/config.ts（Schemastery schema + resolveConfig）

适用场景

你希望把 DSH 跑在无人值守环境，但又不放心让主模型独断"写文件"或"跑命令"这类有副作用的操作；装上这个插件后，主模型要做的每一步沙箱外动作会先被第二个模型读上下文、给出裁决和理由，命中失败兜底会保守拒绝，整个过程在会话日志里可审计、可回放。

前置依赖与兼容性

安装方式

dsh plugin --profile web add github:PerryLink/dsh-auto-review

配置项

常见问题

Q: 默认会自动审查哪些工具？

A: 只审查 bash 和 write，其余工具交给人类审批。edit（原地修改）刻意没列入默认 AI 审查列表——如果你接受原地修改也走自动审查，把 edit: ai 加到 toolsPolicy.overrides 即可。

Q: 审查子 agent 自己也能发起工具调用吗？会不会再次触发审查？

A: 不会。审查子 agent 走的是 fork provider 的子会话，插件按 session id 识别审查子会话并直接走 next() 跳过审查；同时审查只允许 read/glob/grep 三个只读工具，它既不能写、不能跑 shell、也不能调用别的子 agent（maxDepth 限定）。

Q: 审查挂掉了怎么办？

A: 走 fallbackPolicy，默认 rejected——也就是保守拒绝，并把审查失败原因回灌给主模型。如果你想让人类兜底，把 fallbackPolicy 设为 delegate；非要把失败当成功，设为 allow-once（README 明确标注这是无人值守部署才会选的危险选项，grant 是无条件的）。

Q: 我能临时关闭吗？

A: 可以。在会话里跑 /auto-review off 即可，关闭状态写进会话日志，下次回放仍然有效。/auto-review status 看当前状态、当回合预算、累计统计、熔断情况；/auto-review approve [n] 给最近第 n 条拒绝一次性放行。

Q: 它会写入会话日志吗？我能事后审计吗？

A: 会。一条审批请求对应的 approval/asked → autoReview/verdict（或 autoReview/rejection）→ approval/decided 链条都能从 session log 复原。早期的 harness 版本（0.1.0-rc.1 ~ rc.7）会丢掉 ignorable 标记，所以插件会在第一次写入前做能力探测，如果探测不通过就自动降级为内存审计并一次性提醒，必要时用 dsh-permission-rules 仓库里的 scripts/repair-session-logs.mjs 修复已经被污染的历史日志。

Q: 风险等级策略和拒绝熔断器有什么区别？

A: 风险等级策略作用于单次裁决：审查返回 allow 但 riskLevel 超过 maxAutoAllow，该次允许会被升级成 delegate 或 deny；熔断器作用于回合累计：连续 3 次拒绝、或最近 10 次裁决里出现 6 次拒绝，触发后整回合后续请求按预设动作（delegate/reject/abort-turn）落。两者互不冲突。

Q: 数据存哪里？会联网吗？

A: 不写本地文件、不额外发网络请求。审计数据全在内存 + session log（受 MEMORY_DENIES_CAP=200 / MEMORY_OVERRIDES_CAP=50 等有界缓冲约束）。reviewerModel 不填则继承当前会话模型；要换成别的 provider，等于把已展示的会话内容呈现给另一个 provider，自己拿捏一下。

Q: 怎么卸载？

A: dsh plugin --profile web remove dsh-auto-review，或从 profile patch 里删 auto-review 那一行；manifest 标的是 transactional 卸载，不会动 profile 之外的配置。

上手难度

进阶 — 需要理解 DSH 的审批链、risk rule 正则、会话日志 audit 事件这三条核心概念，且要在 cordis.yml 里调风险策略和熔断阈值才能发挥全部作用；只装默认配置直接用也是可以的。

已知问题与限制

• 早期 harness 兼容性：0.1.0-rc.1 ~ rc.7 的 @deepseek-ai/dsh-session 不识别 ignorable 标记，会导致 autoReview/* 事件落地后让更严格 harness 无法 resume 该会话。插件默认会预检并降级为内存审计（功能照常，但无审计日志），需要持久审计可设 allowUnmarkedAudit: true（危险）或用 dsh-permission-rules 的 repair 脚本修历史日志（CHANGELOG.md:9-17 / src/audit.ts:36-52）。
• 审查模型需要可用的 LLM 路由：审查默认继承主会话模型路由；路由不通时每个审查都走 fallbackPolicy——绝对不会静默放行（README.md:232-241）。
• reviewerTools 必须非空：空白名单会让审查子 agent 无工具可用，加载时直接报错；且名单里的工具名必须是 profile 已注册的全局工具，否则审查子 agent 启动时失败。
• /auto-review approve 授权的是"下一次同工具审查"而不是历史那一次调用：如果下一次同工具的调用参数、上下文已经不一样，授权照样会被消费，审查仍按当时上下文判定（README.md:236-237）。
• never 是单向锁：命中 never 策略直接拒绝，不会进入人类链——这是个 lockdown 旋钮，不是日常选项（README.md:230）。
• git 渠道安装需要 pnpm.allowBuilds: { esbuild: true }：pnpm 11 忽略 package.json#pnpm 字段，必须在 pnpm-workspace.yaml 里写 allowBuilds（repo 自带）。typescript + tsdown 在常规 dependencies 里以保证 git 隔离 prepare 环境能装上。
• invariant 配套需要 invariants 服务：只在 headless/ACP 这类 agent-spine composition 下能用；普通 web profile 没有该服务，配套 patch 在仓库里默认注释掉。

---

Compatibility

What you get

dsh-auto-review puts a second model on the approval/request answerer chain:

1. Official seam — an answerer that claims only the requests it owns (ai policy) and delegates everything else via next(); the human approval flow is never short-circuited.
2. Read-only reviewer subagent — a one-shot fork with a read/glob/grep tool allow-list returns a structured verdict { decision, reason, riskLevel }. Reviewer asks are recognized by identity and delegated; maxDepth + the allow-list keep the reviewer non-delegating.
3. Fail closed — reviewer crash, timeout, or schema mismatch resolves through fallbackPolicy (default rejected); a deny verdict feeds its reason back to the calling model.
4. Config-driven routing — per-tool policies (ai/human/never) plus regex risk rules, all changeable from cordis.yml.
5. Deny reasons reach the model — the reviewer's reason is injected into the denied tool result (callId-linked); fallback and never-policy rejections inject auditable markers too ([auto-review] / [auto-review-fallback] / [auto-review-never]).
6. Full audit trail — log-only autoReview/verdict + autoReview/rejection session events (envelope ignorable: true) plus an optional invariant companion enforcing marker ⟺ event.
7. Safety knobs — a rejection circuit breaker (3 consecutive denials, or 6 of the last 10 verdicts, per turn), a risk-level policy, a one-shot /auto-review approve override, and a never-policy hard disable that explains itself to the model.
8. Optional reviewer context — a bounded compact transcript (contextBudget) plus a Codex-style Markdown ruling policy (reviewerPolicyText).

Every decision reconstructs from the session log: approval/asked → autoReview/verdict (or autoReview/rejection) → approval/decided.

Why a second model instead of rules?

Pattern-based auto-approvers decide before dispatch, with no evidence. dsh-auto-review gives the decision to a reviewer subagent that reads the actual workspace (through its read-only tool face), the already-streamed tool-call arguments (sensitive values redacted), the request reason, and your risk rules — then returns a structured verdict. A deny verdict feeds its reason back to the calling model, so the agent learns why instead of retrying blindly.

Quick start

# 1. install the bundle into your profile
dsh plugin --profile web add "github:PerryLink/dsh-auto-review#main"

# or from npm (published releases)
dsh plugin --profile web add dsh-auto-review

# 2. restart and verify the row
dsh --profile web --dump-config | grep -A4 'id: auto-review'

Out of the box the shipped patch AI-reviews bash and write; every other tool (including edit — in-place modification) delegates to the human chain. Add edit: ai explicitly if you accept in-place edits without a human in the loop.

Install & uninstall

• git channel (latest main): dsh plugin --profile web add "github:PerryLink/dsh-auto-review#main" — the isolated prepare build needs the single allowBuilds: { esbuild: true } key the dsh CLI prints for dsh-auto-review.
• npm channel (published releases): dsh plugin --profile web add dsh-auto-review.
• tarball channel: pnpm pack in this repo, then dsh plugin --profile web add ./dsh-auto-review-<version>.tgz.
• uninstall: dsh plugin --profile web remove dsh-auto-review (or remove the row from the profile patch).

Configuration

All tunables are Schemastery Config fields (changeable from cordis.yml). An id-targeted override replaces the whole row — restate every key you need.

Example (annotated full form: fixtures/config/config-full.yaml):

- insert:
    - id: auto-review
      name: dsh-auto-review
      config:
        toolsPolicy:
          overrides: { bash: ai, write: ai }
        riskRules:
          - pattern: '(?i)(rm\s+(-[a-z]+\s+)*/|git\s+push\s+--force)'
            policy: never
          - pattern: 'write'
            policy: never
            field: toolName
        reviewerTimeoutMs: 30000
        fallbackPolicy: delegate
        riskPolicy: { maxAutoAllow: medium, onHighRisk: delegate }
        circuitBreaker: { consecutiveDenies: 3, windowDenies: 6, windowSize: 10, action: delegate }

Tools & surfaces

Session command

/auto-review on|off|status|approve [n]

on/off append the durable autoReview/state override (the fold survives restart/resume — replay IS the state) and inject a switch notice the model sees (logged as a user/message event). status reports the effective state, both per-turn budgets (AI verdicts and reviewer failures), a tripped circuit breaker when one is active, and the session's cumulative statistics (allows/denies/fallbacks/never rejects, mean duration, recent verdicts). approve [n] records a single-use autoReview/override for the n-th most recent denial (1 = most recent): the next same-tool review within overrideTtlMs carries the authorization as reviewer context — the reviewer still decides, and the override is consumed by that review regardless of its outcome.

Web review panel

In the Web GUI (web profile), the package contributes a session-header action (AI Review) that opens a panel with the session's auto-review state: the switch with on/off buttons (they execute /auto-review on|off), both per-turn budgets, cumulative statistics (including hard-disable rejections), the circuit trip, the recent verdicts, and one-shot approve buttons for recent denials (they execute /auto-review approve [n]).

How it is wired:

• The host registers an autoReview session projection (folded from the log-only autoReview/* events) and serves it through the session-projection channel.
• The browser half is a client module (auto-discovered from the dsh.client declaration) registered on the conversation.session.header.actions seat.
• No extra patch rows are needed: the panel loads whenever the plugin is installed in a profile whose web build provides the session-projection capability (the web profile does). Without that capability the panel reports itself unavailable; the answerer is unaffected.

The panel reads only whole projection values — it never receives the raw session event stream.

How it works

                       approval/request waterfall (answerer chain)
                        │
┌───────────────────────┴──────────────────────┐
│ dsh-auto-review answerer                     │
│  · session enabled?  · policy = ai?         │   no ── next() ──▶ human answerer (UI)
│  · risk rules → toolsPolicy → default       │
└───────────────────────┬──────────────────────┘
                        │ yes
                        ▼
        ┌───────────────────────────────────┐
        │ reviewer subagent (fork, one-shot)│
        │  · toolFilter: read/glob/grep     │
        │  · outputSchema: {decision,       │
        │    reason, riskLevel}             │
        │  · timeout + req.signal abort     │
        └───────────────┬───────────────────┘
                        │ verdict / failure (fail-closed fallback)
                        ▼
 allow → allowed-once        deny → rejected + reason injected into the
                                       denied tool result (callId-linked)
                        │   never → rejected + [auto-review-never] feedback
                        │            (hard disable, no reviewer runs)
                        ▼
 audit: approval/asked → autoReview/verdict | autoReview/rejection
        → approval/decided (session events, log-only, invariant-checked)

Composition order. The answerer runs at its registration position in the waterfall: if a human UI answerer is composed BEFORE the auto-review row, humans answer first and the reviewer only sees what is delegated downstream. Verify with dsh --profile <name> --dump-config and place the auto-review row before your human answerer rows when you want ai-policy tools routed to the reviewer first.

dsh-eval — agent evaluation engine

Beyond the approval reviewer, dsh-auto-review ships dsh-eval: a YAML-driven agent evaluation platform that runs real headless DSH sessions (one isolated agent + scratch workspace per case, the official Minimal persona as the baseline system prompt), collects the tool-call trace from the session event log, and evaluates structured assertions plus an optional second-model review — the same reviewer seam as the approval answerer.

# eval/cases/demo.yaml (abridged)
suite:
  name: my-suite
  cases:
    - id: math-output
      input: Solve 17 × 24 and reply with only the final number, nothing else.
      expect:
        output: { contains: "408" }
    - id: glob-trace
      seedFrom: '.'
      input: Use the glob tool with pattern "src/**" to list the source files…
      expect:
        toolCalls: [{ tool: glob, arguments: { contains: { pattern: "src" } } }]
        results: [{ tool: glob, contains: "index.ts" }]

Run it (a DeepSeek API key must be in the environment):

dsh-eval eval/cases --model deepseek-v4-flash --timeout-ms 240000 --out .eval-reports

CI gate: the process exits 0 only when every case of every suite passed — drop it into a GitHub Action step and failing evaluations fail the build. Each case leaves a replayable session JSONL and a trace JSON beside report.md/report.json; assertion results, token usage, and the review verdict are all written into the report files.

Permissions & data

• Permissions: the workshop manifest declares session:append, approval:answer, subagent:spawn, command:register, and tools:observe.
• Data: nothing is stored on disk; the report ring buffer is in-memory and bounded. No network requests of its own.
• Session log: autoReview/* events carry reviewer identity, verdict, reason, risk, and duration — appended with the envelope's ignorable: true marker so any build loads the log. Hosts whose Session.append predates the marker (every released rc line through 0.1.0-rc.8 — no release stamps it yet) are detected before the first append (peer-version pre-check, then a probe of the returned envelope) and audit degrades to an in-memory mirror with marker-free feedback, so sessions stay loadable everywhere.

Security boundaries

• The reviewer is a model. Its verdicts are advisory policy, not a security kernel; prefer human/never rules for irreversible operations.
• Fail closed. Every abnormal path (provider missing, capability gaps, start rejection, timeout, non-completed stop reason, missing/malformed verdict, audit-correlation failure) resolves through fallbackPolicy, default rejected — and the rejection feeds an auditable reason back to the model. allow-once grants unconditionally; it exists only for unattended deployments whose admin accepts that risk.
• Read-only reviewer. The reviewer's toolFilter allow-list (read/glob/grep) cannot write, edit, run bash, fetch the network, or delegate (maxDepth = its own depth). Its session log is persisted and auditable.
• Sensitive arguments are redacted (key-name matching: token, password, api_key, Authorization, credentials, private keys …) before entering the reviewer prompt; the plugin never executes the reviewed arguments. Redaction is key-based, not content-based — do not AI-review tools whose argument values you cannot afford to show a model.
• Hard disables explain themselves. A never tool or risk rule rejects deterministically AND records a log-only autoReview/rejection event, then injects a [auto-review-never] marker into the denied tool result — the model learns the action is hard-disabled instead of retrying it (invariant-checked: marker ⟺ event).
• Rejection circuit breaker. A run of denials in one turn trips the breaker (consecutiveDenies / windowDenies inside windowSize), recorded as a log-only autoReview/circuit event; later requests follow its action (delegate / reject / abort-turn).
• Reviewer context is presented transcript. contextBudget feeds already-presented session content to the reviewer. With the default same-route reviewer model that content stays inside one provider; configure reviewerModel to a different provider only if you accept presenting that transcript to it.
• never is one-way at this layer. A never tool or risk rule rejects before the human chain sees the request — a lockdown knob, not a default.

Known limitations

• The reviewer needs a working LLM route (inherited by default); without one every review falls back per fallbackPolicy — never a silent grant.
• reviewerTools names must exist as global tools in the profile; an unknown name fails the reviewer child loudly at the earliest point and falls back.
• Risk rules match the request reason, the toolName, or the redacted call arguments per their field; other conditions belong in toolsPolicy.overrides.
• The /auto-review approve override authorizes the next same-tool review, not the exact historical call; a different action on the same tool consumes it.
• The verdict events are log-only; the Web review panel reads the folded autoReview projection (the raw event stream never reaches browser plugins).
• autoReview/state and autoReview/verdict are appended with the envelope's ignorable: true marker on hosts that honor it, so any harness build loads the log — readers that do not know the out-of-repo types simply skip those records. On released rc hosts (rc.1–rc.8) the runtime detects the dropped marker and never writes these events (the in-memory mirror keeps the command, budgets, breaker, and approve working for the session); sessions already polluted by pre-0.5.1 versions can be repaired with scripts/repair-session-logs.mjs from dsh-permission-rules (its default target set covers all five autoReview/* event types).
• The git channel needs the single allowBuilds key the dsh CLI prints for dsh-auto-review itself. The repo ships its own pnpm-workspace.yaml with allowBuilds: { esbuild: true }; typescript + tsdown are regular dependencies.
• The optional invariant companion needs the invariants service (agent-spine compositions such as headless/ACP); the plain web profile does not provide it, so the row ships commented out in the bundle patch.

Related work

• Andy8647/dsh-auto-approval — two-state allow/deny classifier on the tools/pre-execute waterfall with file-log audit. dsh-auto-review deliberately differs: official answerer chain, always delegates what it does not own, read-only second model with a structured verdict, deny reasons fed back to the model, session-log audit.
• ACP automation bridge — one-shot machine decisions for its own ACP-owned agents. dsh-auto-review is session- and tool-policy-scoped for the interactive harness; it never infers durable grants.

Development

pnpm install                # node ^22.19 || >=24
pnpm run typecheck          # tsc: src + tests against the local harness checkout
pnpm test                   # vitest: 202 tests, 16 files
pnpm run build              # tsc declarations + tsdown bundles (lib/, incl. the client bundle)
pnpm run verify:self-contained
pnpm pack                   # the published tarball

Repository layout: src/index.ts (plugin contract) · src/config.ts (Schemastery schema + resolution) · src/runtime.ts (answerer, command, deny-reason injection) · src/review.ts (reviewer orchestration, prompt, sanitization) · src/events.ts (session-event vocabulary + folds) · src/audit.ts (host ignorable-marker capability detection) · src/projection.ts + src/projection-types.ts (the autoReview session projection) · src/invariant.ts (invariant companion) · src/eval/ (the dsh-eval engine) · eval/ (shipped evaluation composition) · bin/dsh-eval.mjs (CLI launcher) · src/client/ (browser half) · test/ · fixtures/.

Topics

deepseek-harness, dsh, dsh-plugin, cordis, approval, auto-review, second-model, ai-safety, sandbox, subagent

Contributors

• @PerryLink — creator and maintainer: the approval answerer, the reviewer subagent, risk policy and circuit breaker, the session-projection review panel, the invariant companion, dsh-eval, and the five-language docs.

PerryLink DSH Plugin Family

This project is one of the DeepSeek Harness plugins maintained by PerryLink. If this one helps you, the others likely will too:

License

Apache License 2.0 © 2026 dsh-auto-review contributors