Local image-to-text OCR plugin based on PP-OCRv5 and ONNX Runtime for offline recognition. Automatically invokes when pasting images in TUI or Web.
- Language
- JavaScript
- License
- NOASSERTION
- Branch
- main
Install
$ dsh plugin --profile web add dsh-ocr-localRun the command above in your terminal to install this plugin via the dsh CLI. You can switch Profile in the top-right corner. New to dsh? Read the beginner tutorial
Install via your agent
Install the DeepSeek Harness plugin balcoz/dsh-ocr-local for me: review the repository at https://github.com/balcoz/dsh-ocr-local first, then run the install command and verify the plugin loads successfully.
Paste this instruction to the DSH Web GUI assistant — it will install and verify for you.
One-Line Description
Add local image-to-text capability to DeepSeek Harness: paste screenshots, error dialogs, chat records, or document photos into the conversation, and the plugin automatically invokes the local PP-OCRv5 model to read the text. The entire recognition runs locally, images stay on the machine, and no vision LLM is required.
Core Features
- Paste images in TUI terminal client and Web client, path automatically inserted into input box, text model automatically calls
ocr_imageto recognize text in images - Register
ocr_imagetool, pass in absolute image path to return recognition result (output by line, each line with confidence score) - Register
ocr_setuptool, one-click to create Python virtual environment, install ONNX Runtime and other dependencies, download recognition model (~20MB), entire process is idempotent - Images saved to local cache
~/.dsh/ocr/cache, deduplicated by content hash, auto-cleaned by file count and days - Dark theme screenshots automatically processed with color inversion, lines with too small text or low confidence are marked with ⚠
Technical Implementation
- Languages: Node.js (plugin host) + Python (OCR inference script), cordis plugin protocol
- Key Dependencies:
@deepseek-ai/cordis(host integration),@deepseek-ai/dsh-tools(tool registration),onnxruntime+opencv-python-headless+numpy(Python-side OCR inference), PaddleOCR PP-OCRv5 mobile model (ONNX format) - Architecture Pattern: Plugin provides both Node-side cordis plugin (dsh/index.js registers
ocr_image/ocr_setuptools and/ocr/pasteHTTP route) and browser-side lazy-CJS module (dsh/client.js listens for paste event and uploads image bytes to host route via fetch), automatically injected to Web side viadsh.client.injectfield; Python script invoked by Node side viachild_process.execFile, standard output returned as JSON - Entry Files:
dsh/index.js(cordis plugin main entry),dsh/client.js(browser-side paste-to-path),ocr/ocr.py(PP-OCRv5 inference),ocr/setup.py(one-click bootstrap)
Use Cases
When the model cannot directly view images (text model, tight context window, vision bridge disabled), but you want the agent to read text from screenshots, error dialogs, chat records, or document photos, installing this plugin enables seamless "paste image" + "read text from image" workflow. It's especially suitable for sensitive screenshots—the image never leaves the local machine and doesn't depend on any cloud vision API.
Prerequisites and Compatibility
| Dependency | Min Version | Description |
|---|---|---|
| Host DSH | Integrated via @deepseek-ai/cordis ^4.0.1 + @deepseek-ai/dsh-tools ^0.0.1-rc.1 | No separate DSH version field declared in package, provided by host at runtime |
| Node.js | Not declared | No engines field declared in package, host dsh provides Node runtime |
| Python | Only calls python3/python, no minimum version limit | On first use, will automatically create venv to install onnxruntime/numpy/opencv-python-headless |
| Platform | macOS / Windows / Linux | TUI paste patch adapted for all three platforms, Linux Wayland requires wl-clipboard, X11 requires xclip |
| Native Modules | None | Only uses Node built-in modules (fs/crypto/path/child_process) and Python wheels (onnxruntime etc. installed via pip) |
Installation
dsh plugin --profile web add dsh-ocr-local
Tip: DSH profiles are isolated from each other. For TUI client, install again and run installation script to configure paste key (
./install.sh --profile <profile>orpowershell -ExecutionPolicy Bypass -File install.ps1 -Profile <profile>).
Configuration Options
Config file located at ~/.dsh/profiles/<profile>/cordis.patch.yml, all options have default values, works without configuration.
| Config | Type | Description | Default |
|---|---|---|---|
autoOcr | Boolean | Monitor image attachments in conversation, auto-save and prompt model to call ocr_image for local recognition | true |
pasteToPath | Boolean | Web client "paste image auto-convert to path inserted into input box" | true |
pythonPath | String | Specify Python interpreter path for OCR; empty = auto-find via venv → DSH_OCR_PYTHON → python3/python | Not set |
modelDir | String | Model storage directory | ~/.dsh-ocr/models |
maxCacheFiles | Number | Max number of cached pasted images to retain; exceed = evict oldest by modification time | 300 |
maxCacheAgeDays | Number | Days to retain pasted image cache; expired auto-cleaned | 30 |
Common environment variables:
| Variable | Purpose |
|---|---|
DSH_OCR_MODELS_MIRROR | Model download mirror prefix (set when downloads are slow in China, e.g., https://ghproxy.com/) |
DSH_OCR_PYTHON | Specify which Python to use for OCR (default auto-find) |
DSH_OCR_MODELS | Model storage directory (default ~/.dsh-ocr/models) |
DSH_OCR_VENV | Custom venv directory (default ~/.dsh-ocr/venv) |
FAQ
Q: First use error "environment not ready / missing dependencies" what to do?
A: Tell the agent "use ocr_setup tool to install OCR environment", the plugin will automatically create Python virtual environment, install onnxruntime/numpy/opencv-python-headless, download PP-OCRv5 model. Can also manually run python .../ocr/setup.py, entire process is idempotent and can be re-executed.
Q: Model download very slow or fails?
A: Set mirror prefix and re-run: DSH_OCR_MODELS_MIRROR=https://ghproxy.com/ python .../ocr/setup.py. Download includes sha256 verification, corrupted files will auto re-download.
Q: TUI client paste image no response?
A: First confirm terminal can read clipboard image: Windows/macOS works out of box; Linux Wayland needs wl-clipboard installed, X11 needs xclip, otherwise terminal cannot read clipboard image and paste will silently fail. After installing, paste should work.
Q: Web client paste image no response?
A: Confirm plugin is installed to web profile, config pasteToPath not changed to false, and dsh web restarted. Browser console will have [dsh-ocr] error messages.
Q: Recognition results have typos?
A: Look at ⚠ markers in output, lines with too small text or low confidence are marked. Try zooming in the original image, or ask agent to reconfirm the corresponding lines. PP-OCRv5 mobile has limited effect on long lines, artistic fonts, and blurry images.
Q: After plugin upgrade, functionality unchanged?
A: Restart dsh to reload plugin. After TUI upgrade if paste fails (plugin files overwritten), re-run installation script install.sh or install.ps1 (patch is idempotent).
Q: System prompts pip externally-managed-environment (PEP 668)?
A: Don't add --break-system-packages. Just use ocr/setup.py, it will automatically create a virtual environment, bypassing system Python restrictions.
Learning Curve
Beginner — After installation, just tell the agent "use ocr_setup to install environment" or "recognize this image" and it runs, no configuration needed; only edit cordis.patch.yml if you want to change paste key or cache strategy.
Known Issues and Limitations
- Very small text (e.g., below 4px) or lines with low detection confidence are marked as low confidence, recognition results may be incorrect (see
ocr/ocr.py:59-60) - TUI paste key only supports
ctrl+v/ctrl+shift+v/alt+vthree options, other combinations not recognized (seepatch/apply-cc-tui-patch.mjs:23) - Web client single paste image size limit 16MB (
PASTE_MAX_BYTES), returns 413 if exceeded - Linux Wayland paste image requires
wl-clipboard, X11 requiresxclip, otherwise terminal cannot read clipboard image and paste will silently fail (seedocs/usage.md:108-114) - Model is only PP-OCRv5 mobile, effect is average for long paragraphs, artistic fonts, handwritten text, severely blurred or compressed images
autoOcrenabled by default, after disabling model will no longer auto-recognize pasted images (can still manually ask model to callocr_image)
给 DeepSeek Harness 装一个「本地文字识别」:把截图、报错弹窗、聊天记录、文档照片 变成文字。完全离线、免费,图片不会离开你的电脑,也不需要视觉大模型。
多端支持(TUI + Web)
| 端 | 怎么粘贴图片 | 识别方式 |
|---|---|---|
| TUI(终端客户端) | 终端里 Ctrl+V / 粘图键 / 终端菜单粘贴 | 自动:图片进会话 → 插件存到本地缓存 → 文本模型调 ocr_image 识别 |
| Web | 浏览器里直接 Ctrl+V / Cmd+V 粘贴图片 | 自动:粘贴即转成路径插入输入框 → ocr_image 识别 |
两条路最终都汇合到同一个流程:图片到达会话 → ocr_image 本地读出文字。
如果你的模型支持识图(或配置了视觉桥),图片会原样送达模型,本地 OCR 提示与
视觉链路互不干扰、并存生效。
快速开始(约 5 分钟)
第 1 步:安装插件
DSH 的 profile 互相隔离,插件要装到每个你想用的 profile:
# Web 端
npx -y @deepseek-ai/dsh plugin --profile web add dsh-ocr-local
# TUI 端(把 <profile> 换成你的终端 profile 名)
npx -y @deepseek-ai/dsh plugin --profile <profile> add dsh-ocr-local
TUI 端装完后,再运行一次安装脚本来配置终端粘图键 (Windows 还会自动改写 Windows Terminal 键绑定,备份在 settings.json.bak):
# Windows
powershell -ExecutionPolicy Bypass -File install.ps1 -Profile <profile>
# macOS / Linux
./install.sh --profile <profile>
装完重启 dsh,插件才会生效。
第 2 步:准备识别引擎(只需一次)
把任意一张图片发给 agent,说:
识别这张图片
如果引擎还没装好,工具会告诉你缺什么。这时再对 agent 说:
用 ocr_setup 工具安装 OCR 环境
插件会自动完成三件事:建虚拟环境 → 装 Python 依赖 → 下载识别模型 (约 20MB),之后每次识别都在本地秒级完成。
想手动装也可以(和上面等价,把
<profile>换成你的 profile 名,如web):python ~/.dsh/profiles/<profile>/node_modules/dsh-ocr-local/ocr/setup.py
第 3 步:开始使用
方式 A:粘贴截图(最常用)
- Web:在输入框里按 Ctrl+V / Cmd+V
- TUI:按粘图键(见下表)
图片会自动保存成路径插入输入框,agent 会自动调 ocr_image 读出里面的文字。
方式 B:告诉 agent 图片路径
把图片文件的绝对路径发给 agent,说「识别这张图片」。
粘图键(TUI 端)
安装时指定的快捷键,默认 ctrl+v,可选 ctrl+shift+v / alt+v。
不想占用你习惯的 Ctrl+V 文本粘贴,就选 alt+v 或 ctrl+shift+v:
| 粘图键 | 粘图(发图片) | 文本粘贴 |
|---|---|---|
ctrl+v(默认) | Ctrl+V | Ctrl+Shift+V |
ctrl+shift+v | Ctrl+Shift+V | Ctrl+V |
alt+v | Alt+V | Ctrl+V |
换粘图键:重跑安装脚本 install.ps1 -PasteKey <新键> / install.sh --key <新键>,
会自动切换并清理旧绑定。补丁幂等,重复运行自动跳过。
能识别什么 / 有什么限制
| ✅ 擅长 | ⚠️ 效果一般 |
|---|---|
| 截图、报错弹窗、聊天记录 | 极小的字(如 4px)可能有个别错字 |
| 中文 + 英文混排、长段落 | 复杂背景、艺术字、手写体 |
| 暗色主题截图(自动反色处理) | 模糊或严重压缩的图片 |
识别结果里,字太小或置信度低的行会标注 ⚠,方便你判断哪些字不能全信。
配置(可选,默认不用动)
识别方式与开关
| 场景 | 行为 | 开关 |
|---|---|---|
| 文本模型(不支持识图) | 粘贴/附带的图片自动保存到本地缓存,并向模型注入路径提示 → 模型调 ocr_image 本地识别 | 本插件 autoOcr(默认 true;设 false 关闭自动提示,仍可手动让模型调 ocr_image) |
| 视觉模型(支持识图) | 图片原样送达模型,由模型直接看图;本地 OCR 提示并存、不干扰 | 由你的模型/客户端配置决定(模型声明识图能力或启用视觉桥),本插件不干预 |
autoOcr 只管"本地 OCR 提示"这一半:图片进会话后插件保存并提示模型识别。
"图片要不要发给视觉模型"由模型/客户端决定,与 autoOcr 无关——两件事可以同时开。
配置文件:~/.dsh/profiles/web/cordis.patch.yml
- insert:
- id: ocr
name: 'dsh-ocr-local'
config:
autoOcr: true # 可选:false 关闭「粘贴图片自动提示识别」
pythonPath: ~/miniconda3/envs/ocr/bin/python # 可选:指定 Python
modelDir: ~/.dsh-ocr/models # 可选:模型目录
pasteToPath: true # 可选:false 关闭 web「粘贴图片转路径」
maxCacheFiles: 300 # 可选:粘贴缓存最多文件数
maxCacheAgeDays: 30 # 可选:粘贴缓存保留天数
常用环境变量:
| 变量 | 作用 |
|---|---|
DSH_OCR_MODELS_MIRROR | 模型下载镜像前缀(国内下载慢时设,如 https://ghproxy.com/) |
DSH_OCR_PYTHON | 指定 OCR 用哪个 Python(默认自动找) |
DSH_OCR_MODELS | 模型存放目录(默认 ~/.dsh-ocr/models) |
常见问题
Q:提示「环境未就绪」/「缺少依赖」?
对 agent 说「用 ocr_setup 安装 OCR 环境」即可自动修复;或手动运行
python ~/.dsh/profiles/web/node_modules/dsh-ocr-local/ocr/setup.py。
Q:模型下载很慢或失败?
设镜像后重试(幂等,可反复跑):
DSH_OCR_MODELS_MIRROR=https://ghproxy.com/ python .../ocr/setup.py
Q:系统提示 pip externally-managed-environment(PEP 668)?
不要加 --break-system-packages。直接用 ocr/setup.py——它会自动创建虚拟环境,
绕开系统 Python 的限制。
Q:TUI 端粘贴图片没反应?
先确认终端客户端能读到系统剪贴板图片:Windows/macOS 开箱即用;
Linux(Wayland)需要安装 wl-clipboard(X11 装 xclip),否则终端读不到剪贴板
图片、粘贴会静默失败。装好后重新粘贴即可——图片进入会话后,本插件会自动保存
并提示模型用 ocr_image 识别。
Q:Web 端粘贴图片没反应?
确认插件装到了 web profile、pasteToPath 没被改成 false、且重启过 dsh web。
Q:识别结果有错字? 看输出里的 ⚠ 标注。字太小时模型确实会看走眼:把原图放大一点再试, 或让 agent 把对应行再确认一遍。
工作原理(一句话)
任何端粘贴的图片(TUI 粘贴 / web 粘贴 / 附件)进入会话后,插件把它保存到
~/.dsh/ocr/cache 并提示模型;模型调 ocr_image → 本地 PP-OCRv5 模型
(ONNX Runtime,纯 CPU)→ 文字。模型第一次使用时下载到 ~/.dsh-ocr/models,
之后完全离线。更多细节见 docs/usage.md。
升级
npx -y @deepseek-ai/dsh plugin --profile web update dsh-ocr-local
TUI 端升级后如果粘图失效(插件文件被覆盖),重跑一次安装脚本即可(补丁幂等)。
许可
MIT(代码)。识别模型 Apache-2.0(PaddleOCR),安装时自动下载。见 LICENSE。
Read the usage guide →
Install steps, key points, FAQ and compatibility for this plugin — auto-derived from indexed fields.
Listing badge
[](https://deepseek-plugin.org/plugins/balcoz/dsh-ocr-local)Paste this markdown into your GitHub README to link back to this listing. The badge only states the listing — not a security endorsement.