Skip to main content

dsh-ocr1-memory/dsh-plugins/dsh-ocr1-memory

7Stars1Forks0Issues0Watchers

DeepSeek-OCR based optical compression memory plugin that renders memories as images for storage, degrades resolution by age, and automatically restores them in HD for retrieval.

Evidence5/5methodologySourceInstallMaintenanceDSH versionSecurity scan
Machine-auditedInstall commandRepo verifieddsh-plugin topicLicenseREADMEAI wiki
Language
JavaScript
License
MIT
Branch
main
agent-skillsai-agentdeepseek-harnessdsh-pluginprompt-engineeringpythonskillstranslation

Install

cmdweb profile
$ dsh plugin --profile web add github:DDDFXYqiming/Agent_Extensions#path:dsh-plugins/dsh-ocr1-memory

Run the command above in your terminal to install this plugin via the dsh CLI. You can switch Profile in the top-right corner. New to dsh? Read the beginner tutorial

Install via your agent

Install the DeepSeek Harness plugin DDDFXYqiming/Agent_Extensions/dsh-plugins/dsh-ocr1-memory for me: review the repository at https://github.com/DDDFXYqiming/Agent_Extensions first, then run the install command and verify the plugin loads successfully.

Paste this instruction to the DSH Web GUI assistant — it will install and verify for you.

One-Line Positioning

Store DSH Agent memories as "images" instead of plain text: each memory is rendered as a numbered PNG, older memories automatically downscale in resolution by age, and when retrieval hits a memory, DeepSeek-OCR reads back the original text fragment.

Core Capabilities

  • Text entry automatically segmented and rendered as images (with SoM numbering), stored in local memory store
  • Automatic resolution reduction by age: vivid(1280px, within 24h) → normal(1024px, within 7 days) → fuzzy(640px, older)
  • Active recall temporarily restores HD when retrieving low-res memories
  • Use OCR to read back images as retrieval evidence, returning original text fragments instead of generative summaries
  • Store 1280-dimensional visual vectors (DeepSeek-OCR embeddings), using cosine similarity as the primary retrieval signal
  • Provide 9 tools directly callable by Agents: status/store/update/retrieve/list/compression ratio metric/baseline calibration/delete/render and embedding self-test

Technical Implementation

  • Language: JavaScript (Node ESM, zero runtime dependencies)
  • Key Dependencies: @deepseek-ai/dsh-tools (register Agent tools), @deepseek-ai/schemastery (configure schema), Python + Pillow (image rendering); optional DeepSeek-OCR backend (vLLM or llama.cpp GGUF) for reading back and embedding
  • Architecture Pattern: Mount 9 ocr1_mem_* tools to host tools service via inject=['tools']; configuration injected via cordis.patch.yml; apply(ctx, config) builds renderer + OCR client + embedding client at startup, and can automatically start llama-server on demand
  • Entry Files: lib/index.js (apply entry registers all tools), lib/core.js (memory engine, retrieval algorithm, rendering pipeline), lib/ocr-server.js (llama-server lifecycle)

Use Cases

Suitable for scenarios requiring DSH Agent to have "persistent + long-term + visualizable" memory, such as letting Agents remember document key points, project knowledge, user preferences, and retrieve original text fragments on demand in multi-turn conversations. If only lightweight text memory is needed, the official dsh-memory is simpler; if you want memories to naturally "fade" over time and be able to OCR-read back image evidence, this plugin provides capabilities that the official memory lacks.

Prerequisites and Compatibility

DependencyMinimum VersionDescription
Node.js>=22.19.0engines.node declared
@deepseek-ai/cordis>=4.0.0-rc <5peerDependencies
@deepseek-ai/dsh-tools>=0.0.1-rc <2peerDependencies
@deepseek-ai/schemastery>=3.18.0 <4peerDependencies
Python + PillowAny available versionscripts/render_memory.py dependency, Pillow needed locally
PlatformCross-platformOCR auto-start script defaults to Windows path, can override serverPath/modelDir
Native modulesNonePure Node + Python subprocess, no node-gyp dependency

Installation

dsh plugin --profile web add github:DDDFXYqiming/Agent_Extensions#path:dsh-plugins/dsh-ocr1-memory

Configuration Options

ConfigTypeDescriptionDefault
storeDirstringMemory store root directory, stores images and memories.json<home>/.dsh/ocr1-memory
ocrBaseUrlstringDeepSeek-OCR OpenAI-compatible endpoint (e.g. vLLM http://127.0.0.1:8000/v1), empty skips OCR readbackempty
ocrApiKeystringOCR endpoint API key, usually empty for local servicesempty
ocrModelstringOCR model namedeepseek-ai/DeepSeek-OCR
pythonPathstringPython interpreter for rendering scriptpython
renderScriptstringAbsolute path to rendering script<plugin dir>/scripts/render_memory.py
requireOcrbooleanWhen true, OCR unavailability throws error instead of degradingfalse
useMockRendererbooleanWhen true, skip Python, use pure text placeholder (self-test only)false
autoStartOcrServerbooleanWhen true, plugin automatically ensures llama-server is online after loadingfalse
ocrServerPathstringllama-server executable pathWindows default or OCR_SERVER_PATH env var
ocrModelDirstringDeepSeek-OCR GGUF directoryWindows default or OCR_MODEL_DIR env var
ocrServerPortnumberOCR service port18080
ocrEmbeddingBaseUrlstringVisual embedding endpoint, empty falls back to ocrBaseUrl (combined mode)empty
ocrEmbeddingApiKeystringEmbedding endpoint API keyempty
ocrEmbeddingModelstringEmbedding model name, empty uses ocrModelempty
ocrEmbeddingTimeoutMsnumberEmbedding request timeout120000
ocrEmbeddingAutoStartbooleanWhether to auto-start for independent embedding servicefalse
ocrEmbeddingPortnumberIndependent embedding service port (not used in combined mode)18084
ocrEmbeddingUbatchSizenumberllama-server -ub parameter, must be ≥ visual tokens per image2048
ocrEmbeddingOnDemandbooleanNot enabled in combined mode (only effective for independent service)true
ocrEmbeddingIdleTimeoutMsnumberEmbedding service auto-shutdown after idle milliseconds300000
sharedStorebooleanWhen true, reread memories.json before each operation, supports multi-Agent sharingfalse
embeddingRetrievalbooleanWhen true, use 1280-dim visual embedding similarity as primary retrieval signaltrue
ocrMaxEntriesPerRetrievenumberMax memories to OCR-readback when text retrieval insufficient, prevents large库 freezing5
ocrRepeatPenaltynumberOCR readback repeat penalty coefficient1.2
ocrNoRepeatNgramSizenumberOCR readback n-gram repeat suppression window30
ocrTextOnlyPromptTokensnumberOCR text baseline (for estimating visual token count)5

FAQ

Q: How is this plugin different from official dsh-memory?

A: dsh-memory stores memories as plain text; dsh-ocr1-memory renders each memory as an image with SoM numbering, downscales to 640px fuzzy layer over time, and automatically active-recalls to restore HD when hitting low-res memories. It uses OCR readback instead of pure text retrieval, returning original text fragments instead of generative summaries, avoiding LLM hallucinations.

Q: Can I use it without DeepSeek-OCR service?

A: Yes, you can install and use ocr1_mem_status / ocr1_mem_store / ocr1_mem_list and other basic functions. But ocr1_mem_retrieve's image readback and visual embedding require an OpenAI-compatible OCR endpoint (vLLM running deepseek-ai/DeepSeek-OCR, or llama.cpp running GGUF version), otherwise retrieval can only use pure text scoring.

Q: What local environment is needed for rendering?

A: Python 3 plus Pillow library required. The plugin uses scripts/render_memory.py to call Python to render memory segments as PNGs (with SoM number boxes). The script has built-in CJK font candidate paths (Microsoft YaHei/PingFang/WenQuanYi), covering Windows, macOS, and Linux.

Q: Where are memories stored? How to delete individual memories?

A: Default location is <home>/.dsh/ocr1-memory/, can be changed to custom directory via storeDir config. Use ocr1_mem_forget tool to delete by id; uninstalling the plugin doesn't auto-clean that directory, manual deletion required.

Q: Why do older memories become blurry? Is this a bug?

A: Not a bug, it's by design: each memory falls into vivid(within 24h 1280px) / normal(within 7 days 1024px) / fuzzy(older 640px) tiers based on createdAt. When retrieval hits low-res tier, it automatically active-recalls to restore HD and grants temporary immunity from further decay within the window, simulating "human memory becomes clearer when recalled."

Q: Can multiple Agents share the same memory store?

A: Yes, set sharedStore to true. This mode rereads memories.json before each operation and uses unique temporary files + rename for serialized writes to avoid concurrent overwrites; test reports M1-M6 include multi-Agent sharing scenarios.

Q: How is visual token count measured?

A: The plugin measures visualTokensDirect via embedding endpoint's marker-only request (only sends media, no text), using text baseline for approximation. This is a measurement of llama.cpp's public interface, not DeepEncoder's internal tensor output; README clearly states this is an engineering approximation of the paper's idea.

Q: Can DSH Agent directly call these tools after installing the plugin?

A: Yes. The plugin registers 9 ocr1_mem_* tools to the host via inject=['tools'], Agents can call them directly in conversations without additional enable flags; tool output schema is strict, won't report invalid output under headless profile.

Difficulty Level

Advanced — requires local Python+Pillow installation, and OCR endpoint (vLLM or llama.cpp GGUF) must be truly online for retrieval and visual embedding to work; 27 configuration items total, relatively high barrier for users who just want "install and use."

Known Issues and Limitations

  • OCR backend is engineering approximation, not paper reproduction: README lines 138-170 clearly state that DeepEncoder internal tensor output and "model outputs SoM number" from Locate-and-Transcribe cannot be implemented under current llama.cpp public interface, requires LoRA fine-tuning for further alignment
  • Default path binds to Windows: lib/ocr-server.js:8 and lib/index.js:80-93 hardcode llama-server path as D:\AI_Projects\models\llama.cpp\llama-server.exe, macOS/Linux users must override via ocrServerPath / OCR_SERVER_PATH env var
  • ocrEmbeddingUbatchSize must be ≥ visual tokens per image: Default 2048 covers it, but shrinking to default 512 will cause llama-server to reject large images (README lines 128-131)
  • Embedding service auto-shuts down after idle: Defaults to stop after 5 minutes idle (lib/index.js:153-160), high-frequency multi-Agent scenarios may repeatedly start/stop
  • Retrieval text scoring is character-level token overlap: Chinese splits by single character, English by space, punctuation and emoji stripped; long queries not automatically rewritten
  • Locate-and-Transcribe uses text scoring + OCR evidence instead of model output numbers: Returns verbatim fragments to avoid generation hallucination, but differs from OCR-Memory paper's original Locate path (README lines 145-147)
  • Multi-modal embedding depends on llama.cpp extension: Cannot switch to official DeepEncoder output in AMD without NVIDIA/vLLM environment (README line 166)
  • Python rendering 30s timeout: lib/core.js:267 sets 30s limit for single render, extra long text gets hard-wrapped to avoid single image timeout

Read the usage guide →

Install steps, key points, FAQ and compatibility for this plugin — auto-derived from indexed fields.

Listing badge

Listed on deepseek-plugin.org
[![Listed on deepseek-plugin.org](https://img.shields.io/badge/listed_on-deepseek--plugin.org-007EC6)](https://deepseek-plugin.org/plugins/DDDFXYqiming/Agent_Extensions/dsh-plugins/dsh-ocr1-memory)

Paste this markdown into your GitHub README to link back to this listing. The badge only states the listing — not a security endorsement.

← Back to plugin directory