Skip to main content

How to use dsh-ego-browser

Integrate ego-lite (Chromium for AI Agents) into DSH, providing 32 structured browser tools and a real-time observation window. Users can watch AI operate web pages and take over at any time.

This article is auto-derived from indexed fields (wiki / faq / compatibility_json), not freshly AI-generated.

This article is derived from the plugin's already-indexed fields (wiki / faq / compatibility_json / readme), not freshly generated by AI. Source field is noted at the end of each section.

Quick start

ego-browser

— source: plugin_wiki.wiki_content

Install & verify

dsh plugin --profile web add dsh-ego-browser

Run the command above in your DSH Web Profile. Then enable the plugin in the plugin list.

— source: plugins.install

Key points

  • 看得到:别家是"跑完告诉你结果"的黑盒;我们实时推流,你看着 agent 操作,卡在验证码/走岔立刻发现。
  • 控得住:别家只读;我们监控窗直接驱动同一个 agent 浏览器,需要时你亲手接管(缩放/拖拽/点击),不必打断 agent 重来。
  • 文献 / 数据抓取:让 agent 登录知网 / 谷歌学术翻页收集,你在观察窗看着它滚动、点下一页、下载 PDF,中途卡住立刻能发现。
  • 表单与登录:agent 填表到一半,观察窗弹出验证码——你直接接管把验证码点了,再交还给 agent 继续。
  • QA / 冒烟测试:让 agent 在自己产品上点一圈,观察窗等于一台"会说话的录屏",顺手还能回看历史轨迹。

— source: plugin_wiki.readme_en (fallback readme_raw)

FAQ

What should I do if I can't see the browser ball in the bottom right corner after installation?

The observation window depends on the host providing webServer service (DSH Web has it by default). If running on headless or non-Web host, ego_* tools still work normally, but there will be no observation window panel; the 'Show' switch in the top right won't show a floating window either.

What should I do when the tool call prompts 'CDP channel is not open' or similar errors?

Usually Chrome hasn't warmed up during first cold start. The plugin has built-in withWarmupRetry that automatically retries once when encountering such transient errors; if it recurs repeatedly, please run ego_status first to confirm ego-browser CLI is available, and use EGO_LINUX_CHROME to specify Chrome path if needed.

What's the difference between the 32 tools and the 'Webpage Operation' tool?

All 32 are fine-grained actions on the same browser (click, fill form, scroll, wait, etc.), not 32 independent browsers; they share the same Chromium and are executed serially by an internal mutex in the plugin to avoid contention.

Will the login state be preserved?

They will be preserved within the corresponding task space, but Chrome only saves cookies to disk during graceful shutdown; after force kill or power loss, you need to log in again; to save immediately, you can call ego_auth_flush or click 'Logged in, save' in the observation window login bar.

Can I directly take control of the agent by clicking in the observation window?

Yes. The floating window supports mouse click, drag, scroll, Ctrl+scroll zoom, Ctrl+drag pan; after clicking the画面, keyboard input (Chinese IME, paste, Tab/Enter, arrow keys, Ctrl/Cmd shortcuts) is directly passed back to the same browser.

What's the FFmpeg backend for? Do I have to install it?

Not required. CDP is the default backend with no extra dependencies; the FFmpeg backend on Windows can directly capture D3D11 frames from Chrome windows, providing better quality and resolution. You need to manually switch to FFmpeg in settings and get the build with gfxcapture through managed download.

Can I run it on a Linux server without display / root environment?

Yes. resolveEgoEnv automatically detects root/no-display environment and injects --no-sandbox wrapper scripts with headless fallback; FFmpeg backend on Linux uses x11grab, and will prompt to switch back to CDP when Wayland lacks Portal/PipeWire input.

The schema returned by the tool looks very loose, will fields be missing?

The tool output schema uses additionalProperties: true, and the documentation explicitly states that clients should use actual return values rather than schema fields as the standard; this is to accommodate fields returned by runtime extensions.

— source: plugin_wiki.faq_json

Compatibility

  • DSH: 0.1.0-rc.8
  • Node: >=22

— source: plugin_wiki.compatibility_json

Pitfalls

Review the upstream repo before installing. This guide is auto-derived from indexed fields and may lag the latest release. If anything contradicts the official docs, treat the upstream source as authoritative.

— source: general rule

How to use dsh-ego-browser