dsh-vision

One-Line Description

Give pure text DeepSeek models visual capability: convert images into "seeable" placeholders and expose a vision tool, allowing the model to proactively call image recognition during conversations. Defaults to Douban Web channel, works with browser login, no API key required.

Core Capabilities

• Image Placeholder: Convert image blocks in user messages into text placeholders, feeding them to DeepSeek which originally only accepts text, avoiding "image paste rejected"
• Register Vision Tool: Enable proactive model calls for single/multi-image/OCR/region detail/multi-image comparison
• Auto Detail Upgrade: When detail=auto, first run standard check, automatically run high detail check if judged as complex image
• Visual Evidence Memory: Recognition results written to session timeline, cross-turn reuse, auto-restore after session compression
• Content Hash Cache: Same image + same question + same channel + same detail level = recognize only once per process, avoid duplicate charges or re-runs
• Multi-channel Auto-fallback: Douban Web / Antigravity IDE quota / Gemini API / Cockpit reverse proxy / aicode direct / generic IDE CLI (Claude Code, Gemini CLI, etc.), auto-select per config

Technical Implementation

• Language: TypeScript (server + React client)
• Key Dependencies: @deepseek-ai/dsh-tools (register tools), @deepseek-ai/schemastery (Config validation), @deepseek-ai/dsh-llm (message construction)
• Architecture Pattern: Dual-end plugin — server src/index.ts registers LLM adapter and vision tool and exposes /api/vision, client src/client/plugin.tsx mounts "Vision" panel in session header; Douban Web channel additionally requires running bridge.mjs on Windows side (puppeteer-core drives logged-in Chrome)
• Entry Files: Server src/index.ts (exports apply, Config), client src/client/plugin.tsx (exports apply), queue bridge bridge.mjs

Use Cases

DSH users paste images in conversations wanting DeepSeek to help analyze, but the model itself only accepts text. For example: "Explain this error screenshot", "Compare these two UI screenshots which is better", "OCR this chat record". If you don't want to configure API key but want image analysis, just log in to Douban once on the default channel; if you have Antigravity/Gemini/Claude Code subscription, you can switch channels to use your quota.

Prerequisites and Compatibility

Installation

dsh plugin --profile web add dsh-vision-web

Configuration Options

Local Override: All configs can be overridden in ~/.dsh/dsh-vision.json as JSON (loaded by loadOverrides, path uses $DSH_HOME or ~/.dsh).

FAQ

Q: Can it be used immediately after installation?

A: The default channel is Douban Web (zero cost, no API key). In WSL environment, you need to run node bridge.mjs once on the Windows side, and log in to Douban in that Chrome. After that, you can paste images.

Q: Can it work without a Windows browser?

A: Yes, but you need to switch channels. Configure genlangKey (Gemini API) or cockpitKey (Cockpit reverse proxy), then change defaultChannel to the corresponding channel; Antigravity channel is only available when that IDE is running.

Q: Will the same image with repeated questions cause duplicate charges or re-runs?

A: No. The plugin calculates cache key by image byte hash + prompt + detail level + mode + region + model + channel + prompt version; same process + same request = recognize only once (src/vision-core.ts:124).

Q: What's the maximum number of images per request?

A: Maximum 8 images, single image max 8MB, total max 32MB, HTTP request body max 48MB; OCR/region/comparison modes have additional validation (src/index.ts:731,902-904,1004).

Q: Why doesn't the web channel connect directly to Chrome on WSL?

A: WSL cannot actively connect to Windows ports by default, so the plugin runs a queue service on WSL side (default 127.0.0.1:9340), polled actively by bridge.mjs on Windows side to drive the logged-in Chrome (src/web-channels.ts:43, README.md:113-127).

Q: How to uninstall?

A: Config file is ~/.dsh/dsh-vision.json (loaded by loadOverrides); delete this file to clear local key/path overrides. Plugin itself is uninstalled via dsh plugin --profile web remove dsh-vision-web (src/index.ts:109-122).

Learning Curve

Beginner — works out of the box with zero config on WSL + Windows browser combination; switching to API/Antigravity/IDE CLI channels requires writing some YAML.

Known Issues and Limitations

• Single image 8MB / single request total 32MB / HTTP request body 48MB limits, exceeding returns error (src/index.ts:902-904,1004)
• Maximum 8 images per request; comparison mode requires at least 2 images; region detail requires region coordinates or natural language (src/index.ts:731-733)
• Default Douban Web channel requires bridge.mjs continuously running on Windows side, and that Chrome must be logged into Douban; unlogged bridge reports "Douban input box not found"
• Antigravity channel depends on running Antigravity IDE and language_server.exe; PID/port/CSRF auto-discovered per call, but if IDE is completely closed it errors directly (src/index.ts:320-323,509)
• API channel depends on WSL calling Windows curl.exe (default /mnt/c/Windows/System32/curl.exe), ensure path exists in network-restricted environments
• Gemini API overload (503) triggers auto-retry 2 times, only then returns error to model if still failing (src/index.ts:211-229)
• aicode channel (oauthAccount) returns 500 when Antigravity IDE workspace binding is needed; plugin suggests switching to cockpit channel (src/index.ts:279-281)
• Client panel doesn't provide advanced editing (crop/rotate) besides "delete single image" shortcut; pre-process externally
• Error messages auto-truncate (e.g., stderr limited to 500 chars, agentapi errors limited to 1200 chars) for log readability, but full logs may be needed for troubleshooting (src/index.ts:141-148)

给 DeepSeek Harness 的纯文本 DeepSeek 一双眼睛——默认走豆包 Web，零成本、免 API key。

English | 简体中文

一句话：不用 API key、不用付费——浏览器里登录一次豆包，DeepSeek 就能在每轮对话里看图。粘贴、识图、回答。

✨ 为什么值得用

🎯 真实效果（2026-08 实测，全链路真实调用）

输入：一张橘猫照片 + 这是什么动物？它在做什么？请用中文回答。

输出（豆包 Web 通道——默认）：

这是一只橘猫（家猫），它四仰八叉仰躺在床上睡觉，肚皮露在外面，四肢舒展，睡得十分放松惬意。猫咪把肚子露出来，说明它对周围环境很有安全感。

输出（反重力通道 · flash 档）：

这是一只橘猫（橘色虎斑家猫）。它正仰面熟睡/惬意放松：四脚朝天、露出圆滚滚毛茸茸的肚子，正舒适地躺在深色床垫/毯子上睡觉。

你粘贴图片 + 提问
  → dsh-vision 把图片转成占位符，DeepSeek（大脑）看到
  → DeepSeek 调用 vision 工具 → 豆包 Web（默认）/ 其他通道识图
  → 文字证据回填 → DeepSeek 继续回答

🚀 快速开始

方式零：豆包 Web（默认，零成本）

1. 装插件：

   dsh plugin --profile web add dsh-vision-web
   

2. 启动 Windows 桥接（驱动你已登录的 Chrome）：

   # 在 Windows 侧：连接一个开了调试端口的 Chrome，然后运行
   node bridge.mjs    # 详见下方「豆包桥接」章节
   

3. 在那个 Chrome 里登录一次 doubao.com。
4. 重启 dsh web，粘贴图片——搞定。全程不需要任何 API key。

方式一：反重力额度（如果你用 Antigravity）

配置好工作区后，插件自动优先走反重力（按模型名选 flash/pro 档）：

- insert:
    - id: vision
      name: dsh-vision
      config:
        antigravityWorkspace: /path/to/workspace
        antigravityProjectId: your-project-id
        antigravityLsExe: /path/to/language_server.exe
        antigravityWindowsHome: /mnt/c/Users/you
        antigravityBrainDir: /mnt/c/Users/you/.gemini/antigravity/brain

端口/CSRF 每次调用自动发现，IDE 重启也不用改配置。

方式二：Gemini API（降级通道）

      config:
        genlangKey: AIza...   # https://aistudio.google.com/apikey

方式三：任意 IDE CLI（Claude Code / Gemini CLI / Qwen Code / MiMo…）

如果你有编程软件的会员，用它的本地 CLI 来识图（配置驱动，无需改代码）：

      config:
        ideCli:
          enabled: true
          exe: claude                      # 或 gemini / qwen / 任意 CLI
          argsTemplate: "-p {prompt}"      # {prompt} 替换为提问 + 图片引用
          imageRefTemplate: "{path}"       # Gemini CLI 用 "@{path}"
          timeoutMs: 120000

通道把提问 + 图片文件路径传给 CLI，stdout 即回复——Claude Code、Gemini CLI、Qwen Code、MiMo 等都用同一套配置接入。

使用

1. 面板：会话头部点「识图」→ 添加/粘贴图片 → 提示词 → 模式/档位/通道 → 识别。
2. 对话流：模型选择器选 deepseek-vision 路由，粘贴图片发送——模型自动调用识图。

🌉 豆包桥接（零成本通道怎么跑）

豆包 Web 通道自动化的是你已登录的浏览器，而不是调 API：

DSH 插件 (WSL) ──submit──▶ 队列服务 (127.0.0.1:9340)
                                ▲ 轮询
Windows 桥接 (node + puppeteer-core) ──┘
        │ CDP 连接 Chrome（调试端口）
        ▼
豆包侧边栏：上传图片 → 输入提问 → 回车 → 等回复
        │
        ▼ 回复文本 → POST /result → DSH 插件

• 一次登录：在桥接用的 Chrome profile 里登录一次豆包，之后一直复用。
• Windows→WSL localhost 转发承载队列通信，无需改防火墙。
• 桥接以 bridge.mjs 随仓库分发——在 DSH 旁边跑一次（node bridge.mjs），永久轮询。
• WSL 无法直连 Windows 端口，所以队列放在 WSL 侧、桥接从 Windows 侧主动轮询。

🧠 识图引擎——复杂识图到底怎么工作

天真的"看图→描述"循环在密集截图、表格、UI 原型和多图对比面前会崩。dsh-vision 把识图做成了结构化、自动升级、带记忆的流水线：

1. 档位自动升级——它知道一遍不够

detail: auto 走两遍策略：

第一遍（standard + 预判）──▶ complexity == "simple" ──▶ 完成
                            └─▶ complexity == "complex" ──▶ 第二遍（deep）──▶ escalated 结果

视觉模型自己判断复杂度。以下情况都判为 complex 并自动触发深度检查：

• 多主体关系 · 密集小字 · OCR 密集内容
• 表格 / 图表 / 代码 / 界面 · 计数 · 对比 / 找差异
• 专业画面 · 多步空间推理

2. 四种任务模式，一个工具

3. 结构化证据，不是流水账

每次识图都要求输出严格 JSON 证据对象：

{
  "complexity": "simple|complex",
  "base_evidence": {
    "summary": "中性概述",
    "ocr": "可见文字（没有则为空）",
    "layout": ["布局观察"],
    "entities": ["实体"],
    "relations": ["关系"],
    "uncertainty": ["明确的不确定项"]
  },
  "query_answer": "直接回答用户"
}

观察与推断分离、不确定性显式写出（绝不脑补）、每项列表限长——上下文保持紧凑。

4. 长上下文视觉记忆——模型永远不会"忘"它看过什么

• 每次结果作为持久化 <dsh-vision-evidence> 记录写入会话时间线。
• 跨轮复用：同一张图 + 同一个问题命中已有记录——识别一次，永远记住。
• 视觉记忆清单：每次请求流自动附带近期证据目录，模型可以主动追问——放大区域、重新 OCR、跟新截图对比——你什么都不用重新粘贴。
• compaction 恢复：长会话压缩后，近期视觉记录自动恢复。

5. 内容哈希缓存——同样的像素绝不付两次钱

缓存键 = SHA-256(图片字节) + prompt + detail + mode + region + model + channel + prompt版本，进程内 LRU。

⚙️ 完整配置

🔧 Troubleshooting

🔒 隐私

• 豆包 Web：图片发给你自己登录的豆包会话——和手动用网页一样。
• API key：只存本地配置；错误信息自动脱敏，绝不写日志。
• OAuth 凭据：不内置——aicode 通道要求用户自带，存本地配置。

🏗️ 架构

src/
├── index.ts        # 服务端：适配器、vision 工具、/api/vision、compaction 恢复
├── vision-core.ts  # 提示词构建、响应归一、证据记录、流修复
├── web-channels.ts # 豆包 Web 队列服务（WSL 侧）
└── client/
    └── plugin.tsx  # 客户端面板（多图/粘贴/模式/档位/通道）

bridge.mjs          # Windows 桥接：轮询队列，CDP 驱动豆包

🛠️ 开发

git clone https://github.com/54xkeee/dsh-vision
cd dsh-vision
npm install
npm run build     # esbuild → lib/index.js + lib/client.js
npm test          # node --test

📄 License

MIT

🙏 Credits

• vision-toolkit 架构（占位符 + vision 工具 + 包装适配器 + 证据记忆）与 dsh-youreyes 共享
• 通道与错误处理约定参考 dsh-vision-proxy 等社区插件