realhands - Real Desktop Computer Use MCP server

<!-- mcp-name: io.github.kanishka089/realhands -->

computer-use-mcp ("realhands")

An MCP server that lets Claude operate your real computer the way a human does — moving the actual mouse, clicking, typing, and reading the actual screen.

Unlike OpenAI Operator, browser-use, or Playwright agents (which spin up a separate, isolated, logged-out Chrome), this drives the physical OS cursor and keyboard. So it works in your own Chrome with your own logged-in sessions — and in every other app — because it's just a human at the keyboard, as far as any website can tell.

Status: LIVE and battle-tested. Registered with Claude Code as the user-scope MCP realhands (tool mcprealhandscomputer) and ✓Connected since 2026-06-02. On 2026-06-09 it drove the user's real, logged-in Chrome through a complete Google Play Console deployment (app upload, release notes, submission) end-to-end.

The server is registered as realhands rather than computer-use because the name

"computer-use" is reserved in Claude Code.

How it works

Claude (Desktop or Code) is the agent loop. You type a task; Claude calls the single computer tool in a see → think → act cycle:

See — screenshot returns the real screen (downscaled to ~1280px for grounding accuracy)

→ Think — Claude picks the next action + pixel coordinates

→ Act — the server moves the real mouse / types on the real keyboard

→ a fresh screenshot comes back automatically after every action, and it repeats.

Two Windows-specific details make clicks land accurately (src/screen.py):

pixels == pyautogui cursor coordinates even under display scaling (125% / 150% / …).

• DPI awareness — SetProcessDpiAwareness(2) is set at import time so screenshot

COMPUTERUSEMAXDIM on the longest side before sending; incoming click coordinates are scaled back up to real pixels. The scale factor is a pure function of monitor geometry + MAXDIM, so mapping never depends on which screenshot ran last. Coordinates are clamped inside the target monitor so a stray click can't fly off-screen. Each axis is rounded down to a whole 28px vision patch, so the two axes can scale very slightly differently (<2.2%); to_real() maps each with its own factor and stays exact.

• Stateless coordinate scaling — screenshots are downscaled (LANCZOS) to at most

Multi-monitor: every call takes an optional monitor index (1 = primary, 2.. = others, 0 = the whole virtual desktop). action="monitors" enumerates the setup. Origins may be negative for screens left/above the primary — toreal() handles the offset. Use the same** monitor for a click as for the screenshot you're clicking on.

Architecture

src/realhands/
  server.py   FastMCP server "computer-use"; the single `computer` tool (action enum
              modeled on Anthropic's reference computer_20250124 tool); runs one
              action or a batch of `steps`, then returns status text + at most one
              screenshot
  screen.py   DPI awareness, mss capture, patch-aligned downscale, model-space ->
              real-pixel mapping, unchanged-screen detection
  input.py    pyautogui mouse/keyboard execution; xdotool-style key-name translation
              (Return, Page_Down, ctrl+a, super, ...); clipboard-paste fast path for
              long/Unicode/multiline typing (preserves your existing clipboard);
              activate_window via win32 AttachThreadInput
  safety.py   kill switches + lazy arm / stand-down lifecycle
  config.py   .env-driven configuration (all defaults are sensible; .env is optional)
install.py    one-shot installer: venv, deps, .env, Claude Desktop registration

Stack: Python 3.10/3.11 · mcp (FastMCP, stdio) · pyautogui · mss · pillow · pynput · keyboard · pyperclip · python-dotenv — plus pygetwindow and pywin32 for activate_window.

The computer tool

A single tool with an action parameter:

Shortened. The full README is on GitHub.