Skip to main content

How to use dsh-computer-use

Enables DeepSeek Harness Agent to read, write, and control other desktop applications on macOS through accessibility APIs without moving the system cursor or changing the foreground window.

This article is auto-derived from indexed fields (wiki / faq / compatibility_json), not freshly AI-generated.

This article is derived from the plugin's already-indexed fields (wiki / faq / compatibility_json / readme), not freshly generated by AI. Source field is noted at the end of each section.

Quick start

dsh-computer-use

— source: plugin_wiki.wiki_content

Install & verify

dsh plugin --profile web add @anionex/dsh-computer-use

Run the command above in your DSH Web Profile. Then enable the plugin in the plugin list.

— source: plugins.install

Key points

  • No system-cursor movement: the helper contains no cursor-warp path.
  • No global pointer injection: click, scroll, and drag fallback use a pid/window-targeted SkyLight route, not the global HID event stream.
  • No blind replay: every action is tied to an exact, unexpired observation and returns fresh state.
  • Observe before acting. Return a bounded Accessibility tree, indexed elements, exact app/process/window metadata, permission state, and an optional screenshot Artifact.
  • Prefer semantic input. Use AXPress, editable values, selected-text assignment, and advertised Accessibility actions before pointer fallback.

— source: plugin_wiki.readme_en (fallback readme_raw)

FAQ

Does this plugin support Windows or Linux?

The current version only provides a macOS provider; the README clearly states that Windows UI Automation and Linux providers are not yet implemented. Windows and Linux users will need to wait for future versions.

What macOS system permissions are required to use this?

Calling read-only operations like computer_observe requires macOS Accessibility permission; Screen Recording permission is only needed when taking screenshots. Both must be manually granted in macOS System Preferences - the plugin itself has no privilege escalation capabilities.

Can the Agent control my computer immediately after installation?

No. After installation, the plugin only registers the computer_use_activate引导工具 (boot tool) by default. The 11 computer_* tools that can actually perform read/write operations only appear after loading the /computer-use Skill in the current Agent session.

Will this plugin steal focus from the app I'm currently using?

No, not by default. The focus policy focusPolicy defaults to preserve - it only uses a separate small cursor indicator for clicking, scrolling, and dragging, without switching the foreground. Only when you change keyboardPolicy or focusPolicy to activate will the target App be brought to the foreground before keyboard input.

How do I completely uninstall this plugin?

Execute dsh plugin --profile web remove @anionex/dsh-computer-use (same for Headless Profile). Uninstalling will deregister the Skill and tools, release process-level observation and confirmation tokens, and close storage domain handles; previously generated screenshot files and computer_use_state sidecar files need to be manually cleaned up.

— source: plugin_wiki.faq_json

Compatibility

  • DSH: >=0.1.0-rc.6
  • Node: ^22.19.0 || >=24.0.0
  • Native modules: native/macos/bin/dsh-computer-use-helper (ad-hoc-signed universal Swift binary)

— source: plugin_wiki.compatibility_json

Pitfalls

Review the upstream repo before installing. This guide is auto-derived from indexed fields and may lag the latest release. If anything contradicts the official docs, treat the upstream source as authoritative.

— source: general rule