# modsearch > Adds web search, X search, and single-page scraping to text models lacking network capability in dsh. Registers x_search and read_page tools, overriding web_search to provide models with structured, source-attributed evidence. ## Metadata - Author: [@liustack](https://github.com/liustack) - Repo: - GitHub: [liustack/modsearch](https://github.com/liustack/modsearch) - Stars: 190 - Language: TypeScript - License: [MIT](https://spdx.org/licenses/MIT.html) - Homepage: - Topics: `agent-skills`, `agentic-workflow`, `claude-code`, `claude-skills`, `codex`, `cordis`, `deepseek`, `deepseek-harness`, `dsh`, `dsh-plugin`, `glm`, `harness`, `harness-engineering`, `hermes-agent`, `openclaw`, `opencode`, `pi-agent`, `web-fetch`, `web-search` - Forks: 9 - Open Issues: 1 - Last push: 2026-08-20T15:56:00.000Z - Added: 2026-08-14T00:00:00.000Z ## Install ```bash dsh plugin --profile web add github:liustack/modsearch ``` ## Wiki ## One-Line Pitch modsearch is DeepSeek Harness (dsh)'s web search plugin, adding web search, X (Twitter) search, and page fetch capabilities to text models that originally have no internet access, allowing the model to obtain structured evidence with source links directly in conversations instead of relying only on training data. ## Core Capabilities - In the dsh host, takes over the search backend of dsh's native `web_search`; the tool name and schema seen by the model remain unchanged, citation cards are preserved, while the backend is switched to modsearch's engine chain - Registers two additional tools that dsh doesn't have: X (Twitter) search via `x_search` and focused page reading via `read_page`, which automatically appear in the model's tool list - Runs multiple data sources simultaneously (`--source web,x`): web and X execute concurrently, results returned separately, each with `source`, `requestedSource`, `engine`, `status` - Page fetching (`-u`) requires no configuration: built-in local engine uses HTTP direct pull, with SSRF protection and per-hop IP pinning; DNS rebinding cannot bypass it - Engines are organized by role with priority (antigravity-cli → tavily/exa/firecrawl; X uses grok-cli; fetch falls back to local); quota exhaustion automatically triggers failover; `cooldown` is enabled by default, temporarily moving quota-exhausted engines to the end of the chain ## Technical Implementation - **Language**: TypeScript (Node.js ESM) + pure JS dsh plugin side (`dsh/index.js`, uses only Node built-in modules, no bundling) - **Key dependencies**: `commander` (CLI subcommand routing), `undici` (HTTP client and Agent pool for local fetcher), `@biomejs/biome` / `vite` / `vitest` / `typescript` (devDependencies only) - **Architecture pattern**: Dual-form package — the same npm package is both a CLI (`bin: modsearch` → `dist/main.js`, also exposes `modsearch ` externally) and a dsh cordis plugin. The `package.json` `dsh.bundle.patch` points to `cordis.patch.yml`, which injects `@liustack/modsearch` into the host dsh's cordis layer and points the web `searchProvider` to this plugin; the plugin side `dsh/index.js` exports `apply(ctx, config)`, spawns the same package's `dist/main.js` to invoke CLI, no PATH lookup, plugin and engine versions are tightly bound - **Entry files**: CLI entry `src/main.ts`; DSH plugin entry `dsh/index.js`; schema generated by CLI callers is in `src/schema.ts` and `dsh/search-schema.json` / `dsh/fetch-schema.json` (synchronized by unit tests) ## Use Cases When using dsh daily to converse with DeepSeek/GLM and other text models, need to look up latest news, want to see reactions on X about a certain topic, or have a URL you want to read specific content from — after installing this plugin, you don't need to remember any commands, just chat normally or paste a URL. Also suitable for developers doing keyword research, needing citations with clickable source links, or wanting the model to read a complete blog post or API documentation page. ## Prerequisites and Compatibility | Dependency | Minimum Version | Description | |---|---|---| | Node.js | >= 22.13 | `engines.node` in `package.json`; `src/doctor.ts` also lists 22.13.0 as the minimum threshold | | dsh | No specific version declared | Injected into host cordis layer via `dsh.bundle.patch` (cordis.patch.yml); host must expose `ctx.web.registerSearchProvider` and `ctx.tools.register`; package.json declares no peerDependencies | | Platform | macOS / Windows / Linux | package.json doesn't restrict os/cpu; Antigravity CLI / Grok Build binaries are invoked as subprocesses, target platform follows respective official installers | | Native modules | None | No node-gyp dependencies introduced; HTTP fetching uses undici's built-in Agent, process communication uses Node's built-in `node:child_process` | | External accounts | At least one for web search | Default Antigravity CLI (no key required, browser login once); can alternatively configure any of Tavily/Exa/Firecrawl with free keys; X search requires Grok Build + SuperGrok or X Premium subscription | ## Installation ```bash dsh plugin --profile web add github:liustack/modsearch ``` ## Configuration Options Config file is at `~/.modsearch/config.json` (written as 0600, keys automatically masked when rendered via `modsearch config show`). Priority: CLI arguments > environment variables > config file > built-in defaults. File is optional; it runs without it but cannot fix engines or customize endpoints. | Config | Type | Description | Default | |---|---|---|---| | `engine` | string | Preferred engine for web search; empty string means auto-select based on machine conditions. Valid values: `antigravity-cli`, `tavily`, `exa`, `firecrawl`; aliases `agy`/`antigravity`/`grok`/`http`/`direct` will be normalized to canonical names | empty (auto) | | `cooldown` | `on` / `off` | Quota cooldown failover switch; when enabled, moves quota-exhausted engines to end of chain and writes state file; when disabled, completely skips reading/writing state | `on` | | `allowPrivateNetwork` | boolean | Whether to allow local fetcher to reach reserved and internal network address ranges (for VPN scenarios); Firecrawl fetching is always unaffected by this switch | `false` | | `engines..apiKey` | string | API key for that engine, `` ∈ `tavily`/`exa`/`firecrawl`; same-named environment variables (`TAVILY_API_KEY`/`EXA_API_KEY`/`FIRECRAWL_API_KEY`) override file values | none | | `engines..baseURL` | string | Point that engine to a compatible third-party gateway/self-hosted endpoint; must be a full http(s) URL, empty string cancels override; `` ∈ `tavily`/`exa`/`firecrawl`, corresponding env vars `TAVILY_BASE_URL`/`EXA_BASE_URL`/`FIRECRAWL_BASE_URL` | each engine's official endpoint | | `engines..bin` | string | Executable path for that engine; `` ∈ `antigravity-cli` (default `agy`) / `grok-cli` (default `grok`) | each engine's default | | `engines..model` | string | Model name used by that engine; currently applies to `antigravity-cli`, default `gemini-3.6-flash-low` | each engine's default | | `TAVILY_API_KEY` / `EXA_API_KEY` / `FIRECRAWL_API_KEY` | environment variables | Environment variable form of same-named apiKey, higher priority than config file | none | | `MODSEARCH_DSH_CLI` | environment variables | CLI path override used by dsh plugin side when running tests (see `src/dshPlugin.test.ts`) | none | Subcommands: `modsearch config init` creates skeleton, `modsearch config set ` writes values, `modsearch config show` renders effective config (source tags + key masking), `modsearch doctor [--json]` health check, `modsearch state clear` clears all cooldown state. ## FAQ **Q: Can modsearch be used directly after installation, or do I need to configure an API key?** A: Page fetching (`-u`) works with zero configuration because the built-in local engine is always available; web search (`-q`) requires at least one search engine. It defaults to Antigravity CLI, which requires no key and only needs one browser login; the three backup engines (Tavily/Exa/Firecrawl) all have free tiers without requiring a card. When nothing is configured, error messages will list all available activation methods. **Q: What changes will appear in dsh after installation?** A: The native `web_search` tool still uses the same name, the model sees the native schema and citation cards, but the backend is replaced with modsearch's engine chain; additionally, two tools that dsh doesn't have are added: `x_search` for searching X and `read_page` for focused page reading, appearing directly in the model's available tool list without manual triggering. **Q: Which account's quota is deducted during search? Which engine processes it?** A: When not forced with `-e`, all ready engines form a failover chain by role priority. The `results[i].engine` in results indicates the actual responder, `attempts` lists the attempt order, and `warnings` write reasons when an engine fails, gets demoted, or gets cooled down — never silently deducting from another account. **Q: Why does the same X query sometimes return non-X content?** A: When Grok Build is not installed, not logged in, or fails this round, the router automatically falls back to the web search engine to answer, marking `source` as `web`, `requestedSource` as `x`, `status` as `degraded`, and writing the fallback reason in `warnings`; it won't pretend to be real X results. When `--source web,x` requires both simultaneously and X is unreachable, a separate empty entry with `status: "unavailable"` is returned, explicit but as a placeholder. **Q: How do I know what's actually configured on my machine and what's wrong?** A: Run `modsearch doctor`, which performs purely local checks, doesn't consume quota or send network requests. It reports Node version, readiness status and reasons for each engine, config source and permissions, private network switch, cooldown state, and provides a copy-pasteable fix command for each unready engine; add `--json` for machine-readable output for script processing. **Q: What if Antigravity CLI says quota is exhausted?** A: Quota cooldown failover is enabled by default: this round moves antigravity-cli to the end of the chain, next time it directly uses other engines first. Wait for weekly reset (error message will state `Resets in …`) or add a keyed engine (any one of Tavily/Exa/Firecrawl) to recover. `modsearch state clear` can immediately clear all cooldown. **Q: How is it handled when VPN maps target address to an internal IP?** A: Private/reserved address ranges are rejected by default (SSRF protection). You can allow once with `--allow-private-network`, or set globally with `modsearch config set allowPrivateNetwork true`; Firecrawl fetching always rejects private/reserved addresses even with the switch on, preventing internal hostnames from being sent to third-party clouds. **Q: What remains after uninstallation?** A: Just uninstall this plugin. What stays on the machine is only `~/.modsearch/config.json` (your explicitly configured keys and endpoints) and `~/.modsearch/state.json` (cooldown state); no hooks are left, nor does it modify the host's own configuration. ## Learning Curve Introductory — one dsh installation command plus one default engine (Antigravity CLI requires no key, browser login once) lets you use all three capabilities; reading the config manual is only needed for "advanced configuration" (customizing third-party endpoints, switching engines, disabling cooldown). ## Known Issues and Limitations - Local fetcher doesn't execute JavaScript; pages rendered by pure JS frameworks come back very thin; the returned `uncertainty` field explicitly states this, results should be paraphrased accordingly rather than saying the page is empty - Quota is shared by role bucket: Antigravity CLI's free tier is weekly quota, shared with Antigravity desktop App and SDK; concurrent sub-agents can exhaust it quickly - Firecrawl fetching always rejects private/reserved addresses even with `allowPrivateNetwork` enabled (prevents internal hostnames from being sent to third-party cloud) - Explicit `-e/--engine` is hard forced: uses only this one engine, doesn't go to fallback or local fallback, fails with error if unable, won't secretly switch to another engine to deduct from another account - Old config files with `provider`/`providers` and `search`/`fetch`/`social` three-role structure are automatically migrated to new `engine` + `engines` form, no manual file editing needed - For schema-enforced subprocess engines (Antigravity CLI, etc.), output is strictly constrained via `--json-schema`; HTTP engines (Tavily/Exa/Firecrawl) are guaranteed structure by their servers, but CLI forcibly strips `source`/`requestedSource`/`engine`/`model`/`status`/`warnings`/`attempts`/`durationSeconds` from engine-returned fields to prevent forgery --- This document is auto-generated by [deepseek-plugin.org](https://deepseek-plugin.org). HTML page: [modsearch](https://deepseek-plugin.org/plugins/liustack/modsearch) Wiki generated by AI (model: `MiniMax-M2.5`)