Umbriel MCP server

Umbriel · A set of hands for your AI agent — and Playwright for the Windows desktop

Drive any Windows app through five layers — fall back to OCR and pixel-matching when there's no other way in, see and manage windows even when they're hidden, use cursor-free semantic or posted input where Windows permits it, target controls by name and role, and reach past the GUI to drive the OS itself: processes, services, the registry, scheduled tasks, the firewall, network, power, displays, disks, environment, and the event log. Built for ChatGPT/Codex, Claude, and any AI agent that speaks MCP.

What is it?

Umbriel drives the Windows desktop the way a person would — through whatever channel actually works for the app in front of it:

1. Pixels & OCR — when an app exposes no tree at all (canvas, custom-draw, games), fall back to full-screen capture, template matching, and text recognition.
2. Semantic control — read the UI Automation tree an app exposes and target controls by name and role, not coordinates. Survives the DPI, layout, and theme changes that shatter pixel scripts.
3. Sight & window control — capture the live pixels of any window (even fully GPU-composited or occluded), inspect raw HWND hierarchies, and move, raise, or size windows.
4. Input routing — semantic UIA and posted-message input for background-capable controls, with a verified SendInput fallback only when an HWND-less editor or real pointer gesture actually needs the foreground.
5. Drive the OS, not just GUIs — when the task is the machine, act on it directly: read and write the registry (values and keys), list / kill / suspend / reprioritize processes, query / start / stop / configure services, create / delete / enumerate scheduled tasks, read the firewall rules, enumerate network adapters and live connections, drive session power (lock / restart / sleep / hibernate), list volumes and free space, get and set environment variables, read the event log, and read or change the display mode — all native, no PowerShell or shelling out.

Underneath it's a few kilobytes of TypeScript on Bun's built-in FFI — no Appium server, no .NET, no node-gyp, no prebuilt binaries.

Install

bun add umbriel

That's the entire install story. No build step, nothing to compile.

Built for AI agents

This is what Umbriel is for. Register the published server with Codex (ChatGPT desktop, CLI, and the IDE share this configuration):

codex mcp add umbriel -- bunx umbriel

Or register it with Claude:

claude mcp add umbriel -- bunx umbriel

Any MCP-speaking agent can then ground itself in the accessibility tree in ~15 ms per step and use semantic, posted-message, synthetic-input, or pixel fallback as the target requires. Most classic controls remain driveable backgrounded; a no-own-HWND Chromium/WPF/WinUI editor needs a brief unlocked foreground lease for real input. Umbriel verifies that destination before injecting, can verify the resulting editor state, and restores the previous foreground window by default.

Shortened. The full README is on GitHub.