One-line Positioning

Attach a second model to DeepSeek Harness's approval chain, letting sandbox-external tool calls first be read by a read-only sub-agent that returns allow/deny; failures default to deny, with full session logging. Comes with risk level policies, deny circuit breaker, and a web visual panel.

Core Capabilities

• Second Model Approval: Register a hook on the approval/request answerer chain; tools matching the ai policy are routed to a read-only sub-agent for judgment, all other requests go through next() to the original human approval chain.
• Structured Verdict: The review sub-agent returns { decision, reason, riskLevel } via structured output; only read/glob/grep这三个只读工具 are available, so it cannot actually execute the action under review.
• Fail-Open is Not an Option: Review timeout, crash, sub-agent unavailable, or malformed verdict all trigger fallbackPolicy (default rejected)—nothing gets silently passed through; can be changed to delegate (to human) or allow-once (one-time allow, explicitly marked as dangerous in docs).
• Risk Level + Circuit Breaker: When the verdict's riskLevel exceeds maxAutoAllow, you can configure delegate or deny; consecutive denials or sliding-window denials trigger a circuit breaker with one of three actions: delegate / reject / abort-turn.
• Deny Reason Reflow to Model: Denied tool results get injected with [auto-review] / [auto-review-fallback] / [auto-review-never] markers plus the denial reason and bypass-prevention hints, so the model can see why it was denied and why it can't try a different approach.
• Full Audit + Web Panel: Five event types—autoReview/state, autoReview/verdict, autoReview/rejection, autoReview/circuit, autoReview/override—are written to session logs; the web profile adds an AI Review button in the session header, showing toggle, budget, cumulative stats, recent verdicts, and one-time approvals.

Technical Implementation

• Language: TypeScript (ESM, type: module)
• Key Dependencies: @deepseek-ai/cordis (plugin host framework), @deepseek-ai/schemastery (config schema + defaults), @deepseek-ai/dsh-session + @deepseek-ai/dsh-session-projection (session log + web panel data channel), @deepseek-ai/dsh-subagent (review sub-agent orchestration), zod (projection wire schema validation)
• Architecture Pattern: Uses Cordis function plugin contract (name + inject + apply), registers side effects via ctx.on('approval/request', ...), ctx.on('tools/post-execute', ...), ctx.commands.register(...); mounts the autoReview projection unit to the web panel via ctx.inject(['sessionProjections']) when supported by host
• Entry Files: src/index.ts (plugin export), runtime.ts (approval answerer + commands), review.ts (review sub-agent orchestration), config.ts (Schemastery schema + resolveConfig)

Use Cases

You want to run DSH in an unattended environment but don't trust the main model to unilaterally perform side-effecting operations like "write file" or "run command"; after installing this plugin, every sandbox-external action the main model wants to take is first read by a second model, which provides a verdict and reasoning. Failed fallbacks conservatively deny, and the entire process is auditable and replayable from session logs.

Prerequisites and Compatibility

Installation

dsh plugin --profile web add github:PerryLink/dsh-auto-review

Configuration Options

FAQ

Q: Which tools are automatically reviewed by default?

A: Only bash and write are reviewed; other tools go to human approval. edit (in-place modification) was intentionally left out of the default AI review list—if you want in-place modifications to also go through auto-review, just add edit: ai to toolsPolicy.overrides.

Q: Can the review sub-agent also make tool calls? Would that trigger review again?

A: No. The review sub-agent runs in a sub-session via the fork provider; the plugin identifies review sub-sessions by session ID and directly goes through next() to skip review. Also, review only allows the three read-only tools read/glob/grep—it cannot write, run shell, or call other sub-agents (limited by maxDepth).

Q: What if the review crashes?

A: It goes to fallbackPolicy, defaulting to rejected—conservative denial with the review failure reason fed back to the main model. If you want humans to handle it, set fallbackPolicy to delegate; if you want to treat failure as success, set it to allow-once (README explicitly marks this as a dangerous option only for unattended deployments, with unconditional grant).

Q: Can I temporarily disable it?

A: Yes. Run /auto-review off in the session; the disabled state is written to session logs and remains effective on replay. /auto-review status shows current status, current turn budget, cumulative stats, and circuit status; /auto-review approve [n] grants one-time allow for the n-th most recent denial.

Q: Does it write to session logs? Can I audit afterward?

A: Yes. The chain approval/asked → autoReview/verdict (or autoReview/rejection) → approval/decided for each approval request can be reconstructed from session logs. Early harness versions (0.1.0-rc.1 ~ rc.7) lose the ignorable tag, so the plugin does capability probing before the first write; if probing fails, it auto-downgrades to in-memory audit with a one-time reminder, and you can use scripts/repair-session-logs.mjs from the dsh-permission-rules repo to fix already-polluted historical logs if needed.

Q: What's the difference between risk level policy and deny circuit breaker?

A: Risk level policy applies to individual verdicts: if review returns allow but riskLevel exceeds maxAutoAllow, that allow gets upgraded to delegate or deny; the circuit breaker applies to turn-level accumulation: 3 consecutive denials, or 6 denials in the last 10 verdicts, triggers the circuit, and subsequent requests in the turn follow the preset action (delegate/reject/abort-turn). They're not mutually exclusive.

Q: Where is data stored? Does it connect to the internet?

A: No local files, no extra network requests. Audit data stays in memory + session log (bounded by buffers like MEMORY_DENIES_CAP=200 / MEMORY_OVERRIDES_CAP=50). If reviewerModel is empty, it inherits the current session model; if you switch to another provider, you're essentially presenting the displayed session content to another provider—use your own judgment.

Q: How to uninstall?

A: dsh plugin --profile web remove dsh-auto-review, or delete the auto-review line from the profile patch; manifest marks transactional uninstall, leaving profile config untouched.

Difficulty Level

Advanced — requires understanding DSH's approval chain, risk rule regex, and session log audit events; you'll need to tune risk policies and circuit thresholds in cordis.yml to get full value. However, it works fine with just the default config.

Known Issues and Limitations

• Early Harness Compatibility: @deepseek-ai/dsh-session in versions 0.1.0-rc.1 ~ rc.7 doesn't recognize the ignorable tag, which causes autoReview/* events to make stricter harnesses unable to resume the session. The plugin defaults to pre-check and downgrade to in-memory audit (functionality intact but no audit log). For persistent audit, set allowUnmarkedAudit: true (dangerous) or use the repair script from dsh-permission-rules to fix historical logs (CHANGELOG.md:9-17 / src/audit.ts:36-52).
• Review Model Needs Working LLM Routing: Review inherits main session model routing by default; when routing fails, every review goes through fallbackPolicy—never silent allow (README.md:232-241).
• reviewerTools Must Be Non-Empty: An empty whitelist leaves the review sub-agent with no tools, causing load-time error; tool names in the list must be globally registered in the profile, otherwise the review sub-agent fails on startup.
• /auto-review approve Authorizes "Next Same-Tool Review" Not Historical Call: If the next same-tool call has different parameters or context, the authorization is still consumed, and review still judges based on that context (README.md:236-237).
• never Is a One-Way Lock: Matching never policy denies directly, never entering the human chain—it's a lockdown knob, not a daily option (README.md:230).
• Git Channel Install Needs pnpm.allowBuilds: { esbuild: true }: pnpm 11 ignores package.json#pnpm field, must write allowBuilds in pnpm-workspace.yaml (repo ships with this). typescript + tsdown are in regular dependencies to ensure git-isolated prepare environment can install them.
• Invariant Companion Requires invariants Service: Only works under headless/ACP agent-spene composition; regular web profile lacks this service, so the companion patch is commented out by default in the repo.

---

Compatibility

What you get

dsh-auto-review puts a second model on the approval/request answerer chain:

1. Official seam — an answerer that claims only the requests it owns (ai policy) and delegates everything else via next(); the human approval flow is never short-circuited.
2. Read-only reviewer subagent — a one-shot fork with a read/glob/grep tool allow-list returns a structured verdict { decision, reason, riskLevel }. Reviewer asks are recognized by identity and delegated; maxDepth + the allow-list keep the reviewer non-delegating.
3. Fail closed — reviewer crash, timeout, or schema mismatch resolves through fallbackPolicy (default rejected); a deny verdict feeds its reason back to the calling model.
4. Config-driven routing — per-tool policies (ai/human/never) plus regex risk rules, all changeable from cordis.yml.
5. Deny reasons reach the model — the reviewer's reason is injected into the denied tool result (callId-linked); fallback and never-policy rejections inject auditable markers too ([auto-review] / [auto-review-fallback] / [auto-review-never]).
6. Full audit trail — log-only autoReview/verdict + autoReview/rejection session events (envelope ignorable: true) plus an optional invariant companion enforcing marker ⟺ event.
7. Safety knobs — a rejection circuit breaker (3 consecutive denials, or 6 of the last 10 verdicts, per turn), a risk-level policy, a one-shot /auto-review approve override, and a never-policy hard disable that explains itself to the model.
8. Optional reviewer context — a bounded compact transcript (contextBudget) plus a Codex-style Markdown ruling policy (reviewerPolicyText).

Every decision reconstructs from the session log: approval/asked → autoReview/verdict (or autoReview/rejection) → approval/decided.

Why a second model instead of rules?

Pattern-based auto-approvers decide before dispatch, with no evidence. dsh-auto-review gives the decision to a reviewer subagent that reads the actual workspace (through its read-only tool face), the already-streamed tool-call arguments (sensitive values redacted), the request reason, and your risk rules — then returns a structured verdict. A deny verdict feeds its reason back to the calling model, so the agent learns why instead of retrying blindly.

Quick start

# 1. install the bundle into your profile
dsh plugin --profile web add "github:PerryLink/dsh-auto-review#main"

# or from npm (published releases)
dsh plugin --profile web add dsh-auto-review

# 2. restart and verify the row
dsh --profile web --dump-config | grep -A4 'id: auto-review'

Out of the box the shipped patch AI-reviews bash and write; every other tool (including edit — in-place modification) delegates to the human chain. Add edit: ai explicitly if you accept in-place edits without a human in the loop.

Install & uninstall

• git channel (latest main): dsh plugin --profile web add "github:PerryLink/dsh-auto-review#main" — the isolated prepare build needs the single allowBuilds: { esbuild: true } key the dsh CLI prints for dsh-auto-review.
• npm channel (published releases): dsh plugin --profile web add dsh-auto-review.
• tarball channel: pnpm pack in this repo, then dsh plugin --profile web add ./dsh-auto-review-<version>.tgz.
• uninstall: dsh plugin --profile web remove dsh-auto-review (or remove the row from the profile patch).

Configuration

All tunables are Schemastery Config fields (changeable from cordis.yml). An id-targeted override replaces the whole row — restate every key you need.

Example (annotated full form: fixtures/config/config-full.yaml):

- insert:
    - id: auto-review
      name: dsh-auto-review
      config:
        toolsPolicy:
          overrides: { bash: ai, write: ai }
        riskRules:
          - pattern: '(?i)(rm\s+(-[a-z]+\s+)*/|git\s+push\s+--force)'
            policy: never
          - pattern: 'write'
            policy: never
            field: toolName
        reviewerTimeoutMs: 30000
        fallbackPolicy: delegate
        riskPolicy: { maxAutoAllow: medium, onHighRisk: delegate }
        circuitBreaker: { consecutiveDenies: 3, windowDenies: 6, windowSize: 10, action: delegate }

Tools & surfaces

Session command

/auto-review on|off|status|approve [n]

on/off append the durable autoReview/state override (the fold survives restart/resume — replay IS the state) and inject a switch notice the model sees (logged as a user/message event). status reports the effective state, both per-turn budgets (AI verdicts and reviewer failures), a tripped circuit breaker when one is active, and the session's cumulative statistics (allows/denies/fallbacks/never rejects, mean duration, recent verdicts). approve [n] records a single-use autoReview/override for the n-th most recent denial (1 = most recent): the next same-tool review within overrideTtlMs carries the authorization as reviewer context — the reviewer still decides, and the override is consumed by that review regardless of its outcome.

Web review panel

In the Web GUI (web profile), the package contributes a session-header action (AI Review) that opens a panel with the session's auto-review state: the switch with on/off buttons (they execute /auto-review on|off), both per-turn budgets, cumulative statistics (including hard-disable rejections), the circuit trip, the recent verdicts, and one-shot approve buttons for recent denials (they execute /auto-review approve [n]).

How it is wired:

• The host registers an autoReview session projection (folded from the log-only autoReview/* events) and serves it through the session-projection channel.
• The browser half is a client module (auto-discovered from the dsh.client declaration) registered on the conversation.session.header.actions seat.
• No extra patch rows are needed: the panel loads whenever the plugin is installed in a profile whose web build provides the session-projection capability (the web profile does). Without that capability the panel reports itself unavailable; the answerer is unaffected.

The panel reads only whole projection values — it never receives the raw session event stream.

How it works

                       approval/request waterfall (answerer chain)
                        │
┌───────────────────────┴──────────────────────┐
│ dsh-auto-review answerer                     │
│  · session enabled?  · policy = ai?         │   no ── next() ──▶ human answerer (UI)
│  · risk rules → toolsPolicy → default       │
└───────────────────────┬──────────────────────┘
                        │ yes
                        ▼
        ┌───────────────────────────────────┐
        │ reviewer subagent (fork, one-shot)│
        │  · toolFilter: read/glob/grep     │
        │  · outputSchema: {decision,       │
        │    reason, riskLevel}             │
        │  · timeout + req.signal abort     │
        └───────────────┬───────────────────┘
                        │ verdict / failure (fail-closed fallback)
                        ▼
 allow → allowed-once        deny → rejected + reason injected into the
                                       denied tool result (callId-linked)
                        │   never → rejected + [auto-review-never] feedback
                        │            (hard disable, no reviewer runs)
                        ▼
 audit: approval/asked → autoReview/verdict | autoReview/rejection
        → approval/decided (session events, log-only, invariant-checked)

Composition order. The answerer runs at its registration position in the waterfall: if a human UI answerer is composed BEFORE the auto-review row, humans answer first and the reviewer only sees what is delegated downstream. Verify with dsh --profile <name> --dump-config and place the auto-review row before your human answerer rows when you want ai-policy tools routed to the reviewer first.

dsh-eval — agent evaluation engine

Beyond the approval reviewer, dsh-auto-review ships dsh-eval: a YAML-driven agent evaluation platform that runs real headless DSH sessions (one isolated agent + scratch workspace per case, the official Minimal persona as the baseline system prompt), collects the tool-call trace from the session event log, and evaluates structured assertions plus an optional second-model review — the same reviewer seam as the approval answerer.

# eval/cases/demo.yaml (abridged)
suite:
  name: my-suite
  cases:
    - id: math-output
      input: Solve 17 × 24 and reply with only the final number, nothing else.
      expect:
        output: { contains: "408" }
    - id: glob-trace
      seedFrom: '.'
      input: Use the glob tool with pattern "src/**" to list the source files…
      expect:
        toolCalls: [{ tool: glob, arguments: { contains: { pattern: "src" } } }]
        results: [{ tool: glob, contains: "index.ts" }]

Run it (a DeepSeek API key must be in the environment):

dsh-eval eval/cases --model deepseek-v4-flash --timeout-ms 240000 --out .eval-reports

CI gate: the process exits 0 only when every case of every suite passed — drop it into a GitHub Action step and failing evaluations fail the build. Each case leaves a replayable session JSONL and a trace JSON beside report.md/report.json; assertion results, token usage, and the review verdict are all written into the report files.

Permissions & data

• Permissions: the workshop manifest declares session:append, approval:answer, subagent:spawn, command:register, and tools:observe.
• Data: nothing is stored on disk; the report ring buffer is in-memory and bounded. No network requests of its own.
• Session log: autoReview/* events carry reviewer identity, verdict, reason, risk, and duration — appended with the envelope's ignorable: true marker so any build loads the log. Hosts whose Session.append predates the marker (every released rc line through 0.1.0-rc.8 — no release stamps it yet) are detected before the first append (peer-version pre-check, then a probe of the returned envelope) and audit degrades to an in-memory mirror with marker-free feedback, so sessions stay loadable everywhere.

Security boundaries

• The reviewer is a model. Its verdicts are advisory policy, not a security kernel; prefer human/never rules for irreversible operations.
• Fail closed. Every abnormal path (provider missing, capability gaps, start rejection, timeout, non-completed stop reason, missing/malformed verdict, audit-correlation failure) resolves through fallbackPolicy, default rejected — and the rejection feeds an auditable reason back to the model. allow-once grants unconditionally; it exists only for unattended deployments whose admin accepts that risk.
• Read-only reviewer. The reviewer's toolFilter allow-list (read/glob/grep) cannot write, edit, run bash, fetch the network, or delegate (maxDepth = its own depth). Its session log is persisted and auditable.
• Sensitive arguments are redacted (key-name matching: token, password, api_key, Authorization, credentials, private keys …) before entering the reviewer prompt; the plugin never executes the reviewed arguments. Redaction is key-based, not content-based — do not AI-review tools whose argument values you cannot afford to show a model.
• Hard disables explain themselves. A never tool or risk rule rejects deterministically AND records a log-only autoReview/rejection event, then injects a [auto-review-never] marker into the denied tool result — the model learns the action is hard-disabled instead of retrying it (invariant-checked: marker ⟺ event).
• Rejection circuit breaker. A run of denials in one turn trips the breaker (consecutiveDenies / windowDenies inside windowSize), recorded as a log-only autoReview/circuit event; later requests follow its action (delegate / reject / abort-turn).
• Reviewer context is presented transcript. contextBudget feeds already-presented session content to the reviewer. With the default same-route reviewer model that content stays inside one provider; configure reviewerModel to a different provider only if you accept presenting that transcript to it.
• never is one-way at this layer. A never tool or risk rule rejects before the human chain sees the request — a lockdown knob, not a default.

Known limitations

• The reviewer needs a working LLM route (inherited by default); without one every review falls back per fallbackPolicy — never a silent grant.
• reviewerTools names must exist as global tools in the profile; an unknown name fails the reviewer child loudly at the earliest point and falls back.
• Risk rules match the request reason, the toolName, or the redacted call arguments per their field; other conditions belong in toolsPolicy.overrides.
• The /auto-review approve override authorizes the next same-tool review, not the exact historical call; a different action on the same tool consumes it.
• The verdict events are log-only; the Web review panel reads the folded autoReview projection (the raw event stream never reaches browser plugins).
• autoReview/state and autoReview/verdict are appended with the envelope's ignorable: true marker on hosts that honor it, so any harness build loads the log — readers that do not know the out-of-repo types simply skip those records. On released rc hosts (rc.1–rc.8) the runtime detects the dropped marker and never writes these events (the in-memory mirror keeps the command, budgets, breaker, and approve working for the session); sessions already polluted by pre-0.5.1 versions can be repaired with scripts/repair-session-logs.mjs from dsh-permission-rules (its default target set covers all five autoReview/* event types).
• The git channel needs the single allowBuilds key the dsh CLI prints for dsh-auto-review itself. The repo ships its own pnpm-workspace.yaml with allowBuilds: { esbuild: true }; typescript + tsdown are regular dependencies.
• The optional invariant companion needs the invariants service (agent-spine compositions such as headless/ACP); the plain web profile does not provide it, so the row ships commented out in the bundle patch.

Related work

• Andy8647/dsh-auto-approval — two-state allow/deny classifier on the tools/pre-execute waterfall with file-log audit. dsh-auto-review deliberately differs: official answerer chain, always delegates what it does not own, read-only second model with a structured verdict, deny reasons fed back to the model, session-log audit.
• ACP automation bridge — one-shot machine decisions for its own ACP-owned agents. dsh-auto-review is session- and tool-policy-scoped for the interactive harness; it never infers durable grants.

Development

pnpm install                # node ^22.19 || >=24
pnpm run typecheck          # tsc: src + tests against the local harness checkout
pnpm test                   # vitest: 202 tests, 16 files
pnpm run build              # tsc declarations + tsdown bundles (lib/, incl. the client bundle)
pnpm run verify:self-contained
pnpm pack                   # the published tarball

Repository layout: src/index.ts (plugin contract) · src/config.ts (Schemastery schema + resolution) · src/runtime.ts (answerer, command, deny-reason injection) · src/review.ts (reviewer orchestration, prompt, sanitization) · src/events.ts (session-event vocabulary + folds) · src/audit.ts (host ignorable-marker capability detection) · src/projection.ts + src/projection-types.ts (the autoReview session projection) · src/invariant.ts (invariant companion) · src/eval/ (the dsh-eval engine) · eval/ (shipped evaluation composition) · bin/dsh-eval.mjs (CLI launcher) · src/client/ (browser half) · test/ · fixtures/.

Topics

deepseek-harness, dsh, dsh-plugin, cordis, approval, auto-review, second-model, ai-safety, sandbox, subagent

Contributors

• @PerryLink — creator and maintainer: the approval answerer, the reviewer subagent, risk policy and circuit breaker, the session-projection review panel, the invariant companion, dsh-eval, and the five-language docs.

PerryLink DSH Plugin Family

This project is one of the DeepSeek Harness plugins maintained by PerryLink. If this one helps you, the others likely will too:

License

Apache License 2.0 © 2026 dsh-auto-review contributors