Skip to main content

deepseek-harness-desktop/packages/dsh-tool-describe-image

156Stars5Forks6Issues0Watchers

Adds image understanding to text-only models: sends images to OpenAI-compatible vision endpoints for parsing, with only text results returned to the conversation and image bytes excluded from logs.

Evidence5/5methodologySourceInstallMaintenanceDSH versionSecurity scan
Machine-auditedInstall commandRepo verifieddsh-plugin topicLicenseREADMEAI wiki

ⓘ This plugin is a sub-package of the ningbainb/deepseek-harness-desktop monorepo — stars and activity count the whole repository.

Language
TypeScript
License
BSD-3-Clause
Branch
main
ai-agentai-coding-assistantcodexdeepseekdeepseek-harnessdesktop-appdshdsh-plugin

Install

cmdweb profile
$ dsh plugin --profile web add @linxin666/dsh-tool-describe-image

Run the command above in your terminal to install this plugin via the dsh CLI. You can switch Profile in the top-right corner. New to dsh? Read the beginner tutorial

Install via your agent

Install the DeepSeek Harness plugin ningbainb/deepseek-harness-desktop/packages/dsh-tool-describe-image for me: review the repository at https://github.com/ningbainb/deepseek-harness-desktop first, then run the install command and verify the plugin loads successfully.

Paste this instruction to the DSH Web GUI assistant — it will install and verify for you.

One-Line Description

Adds "image viewing" capability to text-only models in DSH: forwards images to OpenAI-compatible vision endpoints during tool calls, passing only the model's text response back to the conversation—image bytes never enter the chat log.

Core Features

  • Enables image understanding for text-only models: Models can actively call the describe_image tool to analyze an image and obtain a text description
  • Accepts three image sources: Local absolute paths, http(s) links (redirects rejected), session attachment references (drag-and-drop or paste-generated markdown references)
  • Send images directly in text-only conversations: Dragged or pasted images are automatically rewritten as references, images render normally, and the model analyzes them via tools
  • Supports custom instructions: Pass prompt during calls (OCR, chart interpretation, UI diagnostics, translation, etc.); falls back to defaultPrompt if not provided
  • Dual protocol support: chat-completions uses /chat/completions, responses uses /responses, covering Qwen-VL, GLM-4V, GPT-4o, Ollama, etc.
  • Real-time configuration card: Settings → Plugin Config → Web UI Plugin Group → Image Understanding, endpoint/model/key/limit changes take effect immediately without restart

Technical Implementation

  • Language: TypeScript
  • Key dependencies: @deepseek-ai/cordis (cordis plugin runtime), @deepseek-ai/dsh-tools (model tool registration), @deepseek-ai/dsh-credentials (credential resolution service), schemastery (settings card schema)
  • Architecture pattern: host + client dual-half cordis bundle; the host half registers describe_image tool and /describe-image/attach / /describe-image/raw routes in the DSH process, the client half rewrites image sending on the browser side and renders the settings card; injected via cordis.patch.yml, zero DSH source code modification
  • Entry files: src/index.ts (host half), src/client/index.ts (browser half)

Use Cases

When using text models in DSH like DeepSeek V4 that don't support native image input, but you want them to "see" images: upload screenshots for the model to describe the interface, convert chart images to CSV, perform OCR recognition, diagnose UI screenshots, etc. The plugin translates "images" into "descriptive text" for the text model—throughout the process image bytes are not written to conversation history, avoiding the risk of sensitive screenshots leaking into logs.

Prerequisites & Compatibility

DependencyMin VersionDescription
DSH0.1.0-rc.7+All @deepseek-ai/dsh-* packages in devDependencies declare ^0.1.0-rc.7
Node^22.19.0 or >=24.0.0Declared in engines.node field
PlatformCross-platformHost half runs on Node, client half runs in browser, no OS-specific dependencies
Native modulesNoneOnly depends on official SDK and schemastery, no native extension compilation required

Installation

dsh plugin --profile web add github:ningbainb/deepseek-harness-desktop/packages/dsh-tool-describe-image

Configuration Options

ConfigTypeDescriptionDefault
baseURLstringRoot address of OpenAI-compatible vision endpoint (e.g., https://dashscope.aliyuncs.com/compatible-mode/v1), trailing slashes auto-removedRequired, no default
modelstringVision model ID to callRequired, no default
apiKeystringInline API key, for local debugging; recommend injecting via environment variable instead of plaintextNone (falls back to apiKeyEnv when unset)
apiKeyEnvstringEnvironment variable name for credential reference; plugin falls back hierarchically: inline key → credential service → startup envVISION_PROMPT
defaultPromptstringFallback instruction when model doesn't receive prompt (make model "describe factually, transcribe readable text, highlight anomalies")See DEFAULT_PROMPT in source code
apiStyleenumEndpoint protocol shape: chat-completions uses /chat/completions, responses uses /responseschat-completions
maxBytesintegerImage byte limit (shared by local files and download links), rejects if exceeded10485760 (10 MiB)
maxOutputTokensintegerPer-request output token limit, mapped to max_tokens or max_output_tokens respectively1024
timeoutMsintegerPer vision request timeout (ms)60000

FAQ

Q: First call fails with baseURL error, how to fix?

A: Open "Settings → Plugin Config → Web UI Plugin Group → Image Understanding" card, fill in the endpoint root address (e.g., https://dashscope.aliyuncs.com/compatible-mode/v1) and model ID, save and it takes effect immediately without restart.

Q: Where should the API key be placed for security?

A: Not recommended to write key directly in config file. Defaults to reading from environment variable VISION_API_KEY (can change via apiKeyEnv), or paste inline apiKey in settings card for local debugging; keys are not written to logs.

Q: When dragging images into a text-only conversation, can the model "see" them?

A: Not directly. The plugin automatically rewrites image blocks as references (like image) when sending; image bytes are uploaded to attachment storage via the attach route, only the reference text enters the conversation, and the model "sees" and analyzes indirectly via the describe_image tool.

Q: Should I choose chat-completions or responses?

A: Default is chat-completions, accessing baseURL/chat/completions; if the endpoint only provides OpenAI Responses API, set apiStyle to responses, which will use baseURL/responses and input/max_output_tokens/output_text shape.

Q: One image not enough, can I give multiple at once?

A: Currently can only process one image per call, no multi-image input, no follow-up questions about the previous image, no structured results like coordinate boxes—this is a declared functional boundary.

Q: How to cleanly uninstall the plugin?

A: Remove the describe-image composite entry from dsh profile (the line corresponding to cordis.patch.yml), and optionally clean up local .npmrc and symlinks; the plugin itself doesn't persist any state to the host.

Difficulty Level

Beginner — configuration is集中 in a single form in the settings card; just fill in endpoint, model, and API key to use; no code writing or DSH source code modification required.

Known Issues & Limitations

  • Only performs magic-byte header validation, doesn't decode image content: if the extension is correct but the file is actually corrupted, the error won't surface until the request reaches the vision endpoint
  • Single image, single response: each describe_image call can only process one image, no multi-image input, no follow-up questions about the previous image, no structured results like coordinates/bounding boxes
  • OCR also counts as a VLM call: even in pure text extraction scenarios, vision model quota is consumed; point baseURL to a cheaper OCR-specific endpoint to reduce costs
  • OpenAI-compatible protocol only: only supports Chat Completions and Responses request/response shapes; vendors using other proprietary protocols need separate adaptation
  • Response body is truncated at maxOutputTokens * 8 + 64 KiB before parsing; risk of truncation when model answers very long is controlled by maxOutputTokens

Read the usage guide →

Install steps, key points, FAQ and compatibility for this plugin — auto-derived from indexed fields.

Listing badge

Listed on deepseek-plugin.org
[![Listed on deepseek-plugin.org](https://img.shields.io/badge/listed_on-deepseek--plugin.org-007EC6)](https://deepseek-plugin.org/plugins/ningbainb/deepseek-harness-desktop/packages/dsh-tool-describe-image)

Paste this markdown into your GitHub README to link back to this listing. The badge only states the listing — not a security endorsement.

← Back to plugin directory