Glovebox MCP server

glovebox-mcp

A sandboxed computer-use MCP server — let an AI agent drive a real browser and desktop apps (mouse, keyboard, screenshots, vision grounding), confined to a nested X11 window so it can never touch your real screen, files, or other apps.

Like a lab glovebox: the agent reaches in and manipulates real applications, sealed off from everything else. Bring the sandbox up, log into whatever sites or apps you want to automate inside that window, and the agent operates only there — you can watch it live and close it instantly.

Speaks the Model Context Protocol, so it works with MCP clients

like Claude Code. Your host can run Wayland; the sandbox gives the agent a real X server to drive.

An agent driving a real browser in the sandbox — gliding the cursor, inserting a unicode name (Nadja Kovačič), typing, and submitting. All confined to a nested X11 window.

Why a nested X11 sandbox?

does — clicks, typing, screenshots — is confined to that window, not your real desktop.

• Most desktop automation (xdotool, PyAutoGUI) is X11-only, but many modern desktops run Wayland.
• Xephyr provides a real X server inside a single window (DISPLAY :1). Everything the agent

• You stay in control: watch it live, pkill Xephyr to close everything.

Requirements

(via Xwayland). Not macOS/Windows. Developed on Ubuntu; any modern Linux with the packages below.

• Linux — the sandbox nests a real X server (Xephyr), so it works even on Wayland hosts

wmctrl, xclip (+ tesseract-ocr for basic). On Debian/Ubuntu the installer auto-installs them via apt (sudo); on Fedora/Arch it prints the matching dnf/pacman command. The MCP server itself is distro-agnostic — any Linux with these tools works.

• Python 3.10+ and uv (used for the virtualenv).
• System packages — xserver-xephyr (Xephyr), openbox, scrot, x11-utils, xdotool,

• A browser in the sandbox (Chromium or Chrome).
• NVIDIA GPU (≥6 GB VRAM) — only for the local vision mode.

Install

Pick a vision backend and run its one-liner (clone → install). Each one installs the system packages (auto via apt on Debian/Ubuntu) and the Python deps for that mode, and writes a ready-to-paste mcp-config.json with your paths.

none — no local models; your agent reads screenshots itself (lightest, instant):

git clone https://github.com/segentic-lab/glovebox-mcp && cd glovebox-mcp && ./install.sh none

basic — Tesseract OCR grounding (parse_screen → text + coordinates, CPU-only):

git clone https://github.com/segentic-lab/glovebox-mcp && cd glovebox-mcp && ./install.sh basic

local — OmniParser on an NVIDIA GPU (parsescreen → text + icons**, pixel-precise; ~4 GB weights, ≥6 GB VRAM):

git clone https://github.com/segentic-lab/glovebox-mcp && cd glovebox-mcp && ./install.sh local

Your choice is written to .vision-mode (override per run with the GLOVEBOX_VISION env var).

Works with any MCP client / harness

Claude Code, Cursor, Codex, or your own agent — it's a standard MCP server, not tied to any one host. Two compatibility notes: basic/local return element coordinates as text, so they work even with text-only agents; none relies on the client passing the tool's screenshots to a multimodal model (fine for Claude Code, Cursor, and other image-capable MCP clients).

Quickstart

Shortened. The full README is on GitHub.