# dsh-ego-browser

> 把 ego-lite（AI Agent 专用 Chromium）接入 DSH，提供 32 个结构化浏览器工具加实时观察窗，AI 操作网页时用户能看见、可接手。

## Metadata

- Author: [@Fisfzy](https://github.com/Fisfzy)
- Repo: <https://github.com/Fisfzy/ego-browser.git>
- GitHub: [Fisfzy/dsh-ego-browser](https://github.com/Fisfzy/dsh-ego-browser)
- Stars: 29
- Language: JavaScript
- Topics: `agent-browser`, `browser-automation`, `dsh-plugin`, `dshx`, `ego-lite`
- Forks: 3
- Open Issues: 5
- Last push: 2026-08-21T10:30:35.000Z
- Added: 2026-08-19T00:00:00.000Z

## Install

```bash
dsh plugin --profile web add github:Fisfzy/ego-browser
```

## Wiki

## One-Line Positioning

Connect ego-lite (a Chromium designed for AI Agents, with the runtime embedded in the plugin package) to DeepSeek Harness, exposing 32 structured `ego_*` tools that enable AI to click, fill forms, read content, and pass verification like a real human on web pages; plus a real-time observation window in the bottom-right corner that lets you watch as AI operates on web pages and take over control when necessary.

## Core Capabilities

- Register 32 structured `ego_*` tools covering task spaces (open/close), page reading (semantic tree snapshots, attribute reading, screenshots, element info), navigation and waiting, keyboard/mouse interaction, forms and dialogs, human verification detection, network request and download capture, and free script escape hatches
- Provide a "🌐" real-time observation window at the bottom-right of DSH Web: uses SSE to push JPEG (CDP backend, default) or fMP4 video stream (FFmpeg backend) to live display the page the agent is browsing, supporting tab switching and history drawer review
- Observation window supports direct takeover: click, drag, scroll, Ctrl+scroll zoom, Ctrl+drag pan, type Chinese after clicking, keyboard shortcuts—all routed back to the same Chromium used by the agent without interrupting the agent to restart
- Built-in human verification detection (reCAPTCHA / hCaptcha / Turnstile / Cloudflare, etc.) and login guidance bar; proactively prompts when agent gets stuck on login or verification
- Cross-platform self-adaptation: Linux/macOS/Windows all supported; root / Docker / no-display environments automatically inject `--no-sandbox` wrapper with headless fallback; no manual Chrome configuration needed
- Expose operation tools like `ego_doctor` for environment check, `ego_auth_flush` for login state persistence, `ego_help` for tool index, etc.

## Technical Implementation

- **Language**: TypeScript (build output: `lib/index.js` + `lib/client.js` + `bin/ego-cast-worker.mjs` three bundles)
- **Key Dependencies**: `@deepseek-ai/dsh-tools` (tool registration base via `defineTool`), `@deepseek-ai/schemastery` (Config Schema validation), `@deepseek-ai/dsh-settings` (`ego-browser` settings namespace and persistence), `react` (frontend floating window UI), `runtime/ego-linux` (embedded ego-lite runtime, vendored)
- **Architecture Pattern**: Cordis dual-sided plugin (`dsh.bundle.patch` injects into host services list, `dsh.client` injects floating window frontend into Web host); each `ego_*` tool follows the pattern "params → assemble JS script → `ctx.subprocess` call `ego-browser nodejs` → parse by `@@DSH_RESULT@@` sentinel"; all tools serialized via in-process mutex lock to prevent browser contention
- **Entry Files**: `src/index.ts` (plugin apply entry + 32 tool registrations), `src/config.ts` (Schemastery Config + resolveConfig), `src/cast-server.ts` (host routes `/api/ego/*`), `src/client/index.ts` (floating window frontend single file)

## Use Cases

When you want AI to accomplish tasks that can only be done with a browser—capturing pages requiring login state, cross-page form filling, mobile simulation, operating sites with captcha or requiring real human sessions—without opening a separate test account for it, this plugin loans the same Chromium to AI; meanwhile, the observation window makes "what AI is doing, where it's stuck" visible throughout, and you can personally take over when necessary without interrupting the agent.

## Prerequisites and Compatibility

| Dependency | Minimum Version | Description |
|---|---|---|
| DeepSeek Harness | 0.1.0-rc.8 | Derived from `@deepseek-ai/dsh-*` series in `peerDependencies`; requires host to provide `tools` / `subprocess` / `webServer` (observation window) services |
| Node.js | >=22 | `package.json#engines.node`; observation window and vendored runtime both rely on Node 22+ built-in capabilities |
| Operating System | macOS / Windows / Linux | Plugin layer supports all three platforms; macOS is official experience, Linux/Windows use community port (see "Known Issues and Limitations") |
| Browser Runtime | Google Chrome / Chromium / Brave / Microsoft Edge (any one) | Auto-detect common paths; root users use plugin's built-in `--no-sandbox` wrapper script; can manually specify via `EGO_LINUX_CHROME` or `chromePath` field in settings |
| Native Modules | None | Plugin body doesn't introduce native modules; vendored runtime is pure JS; FFmpeg binary is optional download (on-demand托管到 `~/.dsh/cache/ego-browser/ffmpeg/`) |

## Installation

```bash
dsh plugin --profile web add github:Fisfzy/ego-browser
```

## Configuration Options

| Config | Type | Description | Default |
|---|---|---|---|
| chromePath | string | Custom Chrome/Chromium executable path; empty triggers auto-detection of common locations | empty (auto) |
| captureBackend | string | Observation window video backend: `auto` / `cdp` (default JPEG stream) / `ffmpeg` (video stream, requires manual install) | `auto` |
| streamProfile | string | Quality preset: `low` / `balanced` / `high`, determines FPS, max width, and bitrate defaults | `balanced` |
| cdpFps | number | CDP preview frames per second, 5–30 | 20 |
| cdpQuality | number | CDP JPEG quality, 1–100 | 55 |
| cdpMaxWidth | number | CDP frame max width (pixels), 320–1920 | 960 |
| cdpBackstopIntervalMs | number | Static page fallback screenshot interval (ms), 1000–10000 | 3000 |
| ffmpegFps | number | FFmpeg backend frames, 5–30 | Per profile 15/20/30 |
| ffmpegMaxWidth | number | FFmpeg backend max width, 320–1920 | Per profile 960/1280/1600 |
| ffmpegBitrateKbps | number | FFmpeg target video bitrate (kbps), 500–20000 | Per profile 2000/4000/8000 |
| ffmpegEncoder | string | FFmpeg H.264 encoder: `auto` / `software` / `h264_mf` / `h264_nvenc` / `h264_qsv` / `h264_amf` / `h264_videotoolbox` / `h264_vaapi` | `auto` |
| ffmpegPath | string | Custom FFmpeg path; empty triggers detection in order: PATH → managed cache | empty |
| githubMirror | string | GitHub mirror prefix for managed downloads, e.g., `https://gh-proxy.com/github.com` | empty |
| egoCliArgs | string | Extra CLI args appended to `ego-browser nodejs` call; takes effect on next `ego_*` call; some control args will be filtered | empty |
| chromeArgs | string | Extra options appended to Chrome launch args; takes effect on next browser cold start (browser is singleton); some control args will be filtered | empty |
| egoBin | string | Custom `ego-browser` CLI path; empty uses vendored `runtime/ego-linux/bin/ego-browser.mjs` | vendored runtime |
| defaultSpace | string/number | Default task space name; subsequent `ego_*` calls without `space` fall back to this | `dsh-agent` |
| maxOutputBytes | number | Byte limit for each subprocess stdout/stderr collection | 4194304 (4 MiB) |
| graceMs | number | Grace period milliseconds for subprocess timeout | 15000 |

## FAQ

**Q: I'm starting as root on a Linux server. Can it run without Chrome?**

A: Yes. `resolveEgoEnv` (`src/index.ts`) automatically detects root and no-display environments, injecting its built-in `--no-sandbox` wrapper script with headless fallback; as long as the system has any Chrome / Chromium / Edge / Brave installed, it can start. When not found, `ego_status` immediately gives the executable path and reason.

**Q: Can I run multiple AI agent tasks simultaneously? Will they interfere with each other?**

A: All `ego_*` tools are serialized via an in-process mutex lock, so multiple agents within the same plugin instance won't grab the same browser; multi-tasking is isolated by "task spaces"—`ego_space_open` creates an independent space, and the `space` parameter routes to the corresponding space. Cookies are isolated between spaces; please log in within the target space.

**Q: What are the requirements for the observation window's live feed?**

A: The CDP backend (default) has minimal overhead; any machine that can run a modern browser on DSH Web can view it; the FFmpeg backend requires the Chrome window to be correctly captured—Windows must use FFmpeg with `gfxcapture` filter, macOS first-time use needs "Screen Recording" permission, Linux needs Chromium and FFmpeg to share `DISPLAY`, Wayland lacking Portal/PipeWire input will prompt switching to CDP.

**Q: What if the agent reaches a captcha page?**

A: `ego_captcha` and `ego_page_info` detect reCAPTCHA / hCaptcha / Turnstile / Cloudflare controls and return `{ detected, kind }`; when detected, a "Human Verification" guidance bar pops up at the top of the observation window—you can directly enter the observation window to complete verification, the agent's same browser is still waiting, and continues after completion.

**Q: Am I viewing "DSH's own interface that the agent is operating"? Or is it in another browser?**

A: The agent uses this one browser, there's no second one. When it operates on DSH's session management, task board, settings interface, the observation window is equally visible throughout and can be taken over—the plugin documentation calls this "self-observation".

**Q: How do I uninstall? Will cookies or profile be left behind?**

A: `dsh plugin remove` follows cordis's standard uninstall flow (fire-and-forget, doesn't block host exit). Chrome profile and vendored runtime are in user-space directories (macOS/Linux: `$XDG_STATE_HOME/ego-lite-linux`, Windows: `%LOCALAPPDATA%\ego-lite-linux`); they persist as long as the plugin isn't deleted; to clean completely, manually delete the corresponding directory.

**Q: The "32 tools" seem like a lot. How do I find the one I need?**

A: Call `ego_help` once (supports `topic` parameter for categorized viewing, such as `tools` / `navigate` / `input` / `form` / `wait` / `network` / `login` / `script`, etc.), it returns a complete index with categories and one-sentence descriptions.

## Learning Curve

Advanced — 32 tools have wide coverage, agent can use directly; regular users only need to install Chrome once and confirm `ego_status` passes. Advanced users wanting to squeeze quality/frame rate need to understand CDP vs. FFmpeg dual-backend differences, and manually install managed FFmpeg as needed.

## Known Issues and Limitations

- Underlying ego-lite host is officially supported on macOS; Linux/Windows use community port; stability of complex multi-step processes is weaker than macOS; tab/space state may be lost between cross-CLI calls
- FFmpeg backend cross-platform capture: Windows needs new build with `gfxcapture`; PATH's old version is skipped with prompt to download compatible version; Linux uses `x11grab`, macOS uses `avfoundation` display crop; macOS ScreenCaptureKit and Wayland Portal helper are future enhancements
- Login state only persists when Chrome shuts down gracefully;强杀 or power loss requires re-login; to immediately固化, call `ego_auth_flush` or click "Logged in, save" in the observation window login guidance bar
- Tool output schema uses loose `additionalProperties: true`; clients should use actual return value as authoritative, not rely on schema-listed fields
- During installation, not all DSH peer packages are in public npm registry; regular `pnpm install` may fail when resolving `@deepseek-ai/*` peers; need to install via DSH profile to provide peers; CDP path doesn't depend on FFmpeg, plugin installation phase won't download binaries
- Snapshot quality: Linux uses CDP `DOMSnapshot` to rebuild semantic tree (not macOS kernel-level), complex iframe / canvas scenarios may degrade
- `ego_http`'s `mode: server` (Node-side fetch.server) may trigger libuv crash in vendored Windows runtime; defaults to browser context (`fetch.browser`)

---

This document is auto-generated by [deepseek-plugin.org](https://deepseek-plugin.org). HTML page: [dsh-ego-browser](https://deepseek-plugin.org/plugins/Fisfzy/ego-browser)
Wiki generated by AI (model: `MiniMax-M2.5`)