Skip to main content

picturereader

31Stars4Forks0Issues0Watchers

Adds visual document reading capability to the text-only DeepSeek Harness model: visual twin receives native thumbnails, local pixel tool with three-mode routing and optional external vision endpoint, image data can remain entirely local.

Evidence5/5methodologySourceInstallMaintenanceDSH versionSecurity scan
Machine-auditedInstall commandRepo verifieddsh-plugin topicLicenseREADMEAI wiki
Language
JavaScript
License
MIT
Branch
main
deepseek-harnessdsh-pluginimage-readingocrpixel-artvision

Install

cmdweb profile
$ dsh plugin --profile web add picturereader

Run the command above in your terminal to install this plugin via the dsh CLI. You can switch Profile in the top-right corner. New to dsh? Read the beginner tutorial

Install via your agent

Install the DeepSeek Harness plugin jing-hy/picturereader for me: review the repository at https://github.com/jing-hy/picturereader first, then run the install command and verify the plugin loads successfully.

Paste this instruction to the DSH Web GUI assistant — it will install and verify for you.

One-Line Description

Adds "image viewing" capability to pure text models in DeepSeek Harness: visual twin renders native thumbnails, local pixel toolchain translates images into model-readable text evidence, external visual endpoints called on demand, and in privacy mode image bytes never leave the machine.

Core Features

  • Visual Twin Adapter: Declares checked text models as image-capable, making "deepseek-v4-flash (Vision)" variants appear in the model selector; pasting images triggers native thumbnails, with image blocks transparently replaced by local pixel analysis text (src/picturereader-vision.mjs:108-163)
  • Local Pixel Tools: image_scan (color/zone/texture scanning), image_sample (N×N precise sampling), image_crop (region-based PNG export), image_palette (dominant colors + hue families), image_compare (two-image/two-region pixel diff)
  • Multi-Engine OCR: image_ocr supports three engines—windows (built-in), paddle (PaddleOCR, better for glowing/bent/game text), rapid (RapidOCR, lightweight fast)—with automatic fallback on failure (src/tool.js / src/more-tools.js)
  • Unified Entry vision_analyze: Routes based on current usage mode—privacy hard-paths to local, smart first judges locally then decides whether to call externally, rigorous mode performs multi-source evidence cross-validation; includes low-information interception to skip blank/solid-color images (src/vision-analyze.js / src/guard.js)
  • Batch & Context: image_batch performs "auto-scan + type detection + is-worth-deeper-look" evaluation on a batch of images; image_batch_ocr auto-runs full OCR then truncates by character
  • Document to Image: document_to_image renders PDF/DOCX/PPTX/XLSX page-by-page to PNG (dpi and max_pages configurable), for visual twin/toolchain to read by page
  • Optional External VLM Bridge: Model decides via routing strategy whether to call sendVisionRequest, OpenAI-compatible endpoint baseURL auto-completes /v1/chat/completions (src/vlm.js)
  • Privacy/Smart/Rigorous Three Modes: Toggle at top of settings card, runtime src/runtime.js takes snapshots, privacy mode hard-blocks with isVlmConfigured()=false

Technical Implementation

  • Language: JavaScript (ESM, "type": "module")
  • Key Dependencies: jpeg-js (pure JS JPEG decoding), pngjs (PNG decoding), omggif (GIF decoding), @deepseek-ai/dsh-settings + dsh-llm (host service injection), @deepseek-ai/schemastery (settings schema declaration and runtime validation)
  • Architecture Pattern: Single cordis bundle + dual half-zone patch. Host side src/index.js writes namespace to settings.yaml, wraps checked adapter registration for visual twin via Proxy, positions sequentially by ctx.inject phase (tools/llm/attachments/webServer/settings); client side client.js registers "Image Reading" card on Web settings page. cordis.patch.yml only inserts plugin line, doesn't touch host code
  • Entry Files: src/index.js (host half-zone entry), client.js (web settings page entry, dsh.client.platform=web)

Use Cases

Running pure text models like DeepSeek in DSH that lack visual encoders, but frequently handling screenshots, scans, UI visual drafts, contract photos, cross-page long PPTs; wanting the model to read image information without sending raw bytes externally. Daily light image reading uses "Smart Mode" to save costs, switches to "Privacy Mode" for ID photos and private materials to stay fully local, and uses "Rigorous Mode" for image review/batch proofreading to do cross-validation.

Prerequisites & Compatibility

DependencyMin VersionDescription
DSH0.1.0-rc.6+peerDependencies declares @deepseek-ai/dsh-settings ^0.1.0-rc.6, @deepseek-ai/dsh-llm ^0.1.0-rc.6
Node^22.19 or >=24engines.node field
PlatformCross-platformCore image decoding uses pure JS libraries, no native modules; OCR/document-to-image dependencies installed on-demand
Native ModulesNonejpeg-js / pngjs / omggif are pure JavaScript, no node-gyp needed
Optional Python venvOptionalTo use PaddleOCR / RapidOCR run scripts/setup-ocr.mjs, setup-rapid.mjs; to convert documents run scripts/setup-doc-venv.mjs

Installation

dsh plugin --profile web add picturereader

Configuration Options

ConfigTypeDescriptionDefault
Mode (mode)EnumPrivacy / Smart / Rigorous, decides whether to allow external visual endpoint callssmart
Enable External Visual API (vlm_enabled)BooleanChecked to show in settings card and allow calling external visual endpoints; unchecked always uses localfalse
Visual API Base URL (vlm_base)StringOpenAI-compatible endpoint address, empty string = disable external VLMEmpty
Visual Model (vlm_model)StringVisual model name for the endpointgpt-4o-mini
Visual API Key (vlm_key)Password fieldrole:'secret', write-only, save-overwrites, no echoEmpty
Key Environment Variable (vlm_key_env)StringFallback reads this env var name when vlm_key is emptyEmpty
Visual Bridge Model List (vision_models)ArrayCheck which text models generate "(Vision)" variants; changes require DSH restart[]
Default OCR Engine (ocr_engine)Enumwindows / paddle / rapidwindows
OCR Language (ocr_language)StringBCP-47 tag, e.g., zh-Hans / en-USEmpty
Single Image Size Limit (max_image_bytes)NumberByte count, reject if exceeded (default 50 MB)52428800
Scan Default Grid Size (scan_default_size)Number 8..64Default size parameter for image_scan32
Scan Palette (scan_palette)Enumauto / full / basic / grayauto
Scan Mode (scan_mode)Enumauto / ascii / colorauto
Multimodal Whitelist (multimodal_models)StringComma-separated, models in whitelist receive images directly without fallbackEmpty
Request Guard (request_guard)BooleanLast line of defense for streaming requests degrades image block to avoid UNSUPPORTED_CONTENTtrue
Document Render DPI (doc_dpi)Number 72..300document_to_image render resolution150
Document Max Pages (doc_max_pages)Number 1..500document_to_image truncation threshold50
Batch Probe First N (batch_probe_first)Numberimage_batch determines if text-dense3
Per-Image OCR Truncate Chars (batch_ocr_limit_chars)Numberimage_batch single image OCR output character limit800
External Visual Timeout (vlm_timeout_ms)NumberMilliseconds300000
External Visual max_tokens (vlm_max_tokens)NumberEndpoint output token limit8192
Image Bridge Export Directory (bridge_export_dir)StringEmpty = use system temp directoryEmpty
Debug Logging (debug)BooleanOutput more detailed diagnostic logsfalse

FAQ

Q: Must I install Python or LibreOffice to use it?

A: Not required. Windows OCR is a built-in engine, works out of the box; unlocking PaddleOCR and RapidOCR requires running scripts/setup-ocr.mjs / scripts/setup-rapid.mjs to create Python venv; using document_to_image for PDF/Office conversion requires installing LibreOffice and running scripts/setup-doc-venv.mjs.

Q: Does Privacy Mode really never call external APIs? How is this guaranteed?

A: Privacy Mode is a hard gate on the host side, not a model promise. The runtime config in src/runtime.js makes isVlmConfigured() always return false in this mode, and the vision_analyze entry forces include_vlm to false; visual twin stream also only exports local evidence text. Even if you fill in baseURL and vlm_key in the settings card, image bytes still won't go to external endpoints.

Q: If I want to cleanly uninstall the plugin, what needs to be done?

A: Just run dsh plugin remove picturereader. The plugin persists two things to the host: exported images in ~/.dsh/picturereader-vision/images/, and settings written to ~/.dsh/settings.yaml under the picturereader section; thorough cleanup requires manually deleting these two locations.

Q: No reaction after checking visual twin, how to debug?

A: Changes to the vision_models list don't hot-reload; you must restart the DSH desktop client for them to take effect. If your current tools.mode is code, the plugin still works, just call it as await tools.image_scan({...}) instead of directly invoking the tool name.

Q: WebP images throw errors when passed in, what to do?

A: Errors are expected; convert the file to PNG or JPEG first before passing. src/core.js explicitly lists .webp in UNSUPPORTED_EXTENSIONS and throws directly; image_scan / image_ocr / image_batch / document_to_image all follow the same rule.

Q: Is vlm_key in the config file safe?

A: Field declares role:'secret', code only writes and never reads, won't echo; if you want to completely keep credentials out of the config file, write the key in an environment variable and tell the plugin the variable name via the vlm_key_env field.

Q: Does visual twin affect all provider models?

A: No. Visual twin only affects models checked in the vision_models list, and modifies their provider's adapter to "twin" via Proxy; unchecked models remain unchanged.

Q: What's the maximum single image size that can be processed?

A: Tool layer hard limit is 50 MB bytes and 24,000,000 decoded pixels; however, DSH attachment upload side defaults to ~5 MB, oversized images may be blocked by the host during upload. Use image_crop to slice or reduce resolution before uploading.

Difficulty Level

Advanced — requires at least specifying "Usage Mode" in the DSH settings card, optionally filling in visual endpoint / enabling visual twin; to use enhanced OCR also needs to run setup scripts to create Python venv. Daily image viewing works with default mode; advanced play requires understanding the privacy vs. external call and visual twin activation conditions.

Known Issues & Limitations

  • WebP format not supported: image_scan / image_ocr / image_batch / document_to_image all reject .webp input, need to convert to PNG/JPEG first (src/core.js:316)
  • DSH attachment upload single image ~5 MB: Oversized images may be blocked at host layer (README.md:269-276)
  • Visual twin requires manual DSH restart: Changes to vision_models list don't hot-load (README.md:218-222)
  • BMP limitations: 16-bit BMP, RLE compressed BMP not supported (src/core.js:200-209)
  • Cross-package compatibility sensitive: v3.0.5 fixes missing scope.load() scenarios; v3.0.6 fixes jpeg-js/omggif/pngjs not properly installed in plugin package causing top-level import failures in core.js (README.md:301-315)
  • dsh-file-drop conflicts with visual twin: If both enabled in same profile, may cause "drag-in image injects text" and "visual twin auto-analyze" dual injection, recommend disabling dsh-file-drop (README.md:275)
  • External VLM bridge needs network: Automatically skips with prompt when not configured/offline, privacy mode never calls (README.md:276)
  • image_batch output character truncation: Single image OCR text truncated by batch_ocr_limit_chars (default 800 characters), long text loses tail (src/image-batch.js:175-264)

Read the usage guide →

Install steps, key points, FAQ and compatibility for this plugin — auto-derived from indexed fields.

Listing badge

Listed on deepseek-plugin.org
[![Listed on deepseek-plugin.org](https://img.shields.io/badge/listed_on-deepseek--plugin.org-007EC6)](https://deepseek-plugin.org/plugins/jing-hy/picturereader)

Paste this markdown into your GitHub README to link back to this listing. The badge only states the listing — not a security endorsement.

← Back to plugin directory