Rocuronium MCP server

rocuronium

Drive this Mac without taking the cursor.

A signed, notarized disk image · free and open source (MIT)

A menu bar daemon that lets an AI agent see and operate macOS — read text, click buttons, type, scroll, drag, draw — while the human keeps their cursor, their focus, and a kill switch. Every action returns evidence of what observably happened, because accessibility APIs routinely lie about success.

A cursor-taking action while someone is at the machine: the jellyfish marks where the agent is, the prompt waits for a one-second hold on Y or N, and the bezel narrates the action with the shortcut that halts everything.

What makes it different

Ghost input. Most actions are delivered through accessibility and per-process posted events — no cursor movement, no focus change, invisible to the person at the keyboard. The agent works beside you, not instead of you.

Evidence, not return codes. Every action is verified: text is read back after writing, pixels are compared before and after, window counts are checked, process exits are detected. The reply says confirmed, noEffect, or unverifiable — and noEffect (the API said "success" but nothing changed) is the most important one.

The human is never locked out. Cursor-taking actions show a presence overlay — a jellyfish escorting the cursor, a bezel narrating what's happening. Touch the mouse and the gesture yields. Press Ctrl+Option+Shift+Escape and everything stops within one sample (~8 ms). Resume is a button in the menu bar and nothing else — the agent cannot un-halt itself.

A virtual display for isolation. Park a window onto an invisible headless display, work on it there with full hardware input (no occlusion, no screen real estate), put it back. The display exists only while a lease holds it and sweeps every window home on teardown.

Sequence plans. Execute multi-step flows as one daemon-side operation — click, wait for the dialog, type, verify the field — with postcondition guards between steps and failure policies (abort, continue, pause-for-human, fallback). No round-trips between steps means the world can't change mid-sequence.

Diff perception. Read an app's text for a fraction of a screenshot's cost; pass the observation token back and get only what changed — elements appeared, vanished, values moved. Screenshots diff the same way: changed-region crops instead of the whole frame, scroll detection with revealed-edge strips.

Vision when the tree lies. About a sixth of Mac apps expose no usable accessibility tree. A three-tier cascade covers them: accessibility first, then a local UI-element detector — a 5.4 MB CoreML YOLOv11n, ~8 ms per window — plus OCR to turn an icon toolbar into addressable boxes, then a local VLM for the rest. read --ocr and find --ocr return the parsed rows, and groundedBy on every row says which tier answered.

Install

brew install kageroumado/tap/rocuronium

Or download the DMG of the latest release.

Requires macOS 26.5 or later on Apple silicon; tested on macOS 26 and 27.

Grant Accessibility (required) and Screen Recording (for screenshots and scroll --until-text OCR) in System Settings > Privacy & Security.

Quick start

Shortened. The full README is on GitHub.