glass MCP server

glass

Give your coding agent hands for the app it's building — launch it, drive it, and verify the result, without burning a screenshot on every step.

A Rust MCP server that gives an AI coding agent a closed build → see → interact → debug loop over external native GUI applications.

glass lets an agent launch a GUI app, capture what is on screen, inject mouse and keyboard input, read the app's logs, and detect visual changes — so a coding agent can build and debug UI applications independently instead of asking the user "does this look right?".

glass drives apps as an external black box, so it works with any native GUI app regardless of toolkit or language. It has two Linux backends (X11 and Wayland), a Windows backend, an Android backend (an AVD emulator, driven over adb from any host), an iOS backend (native apps in the Simulator over xcrun simctl, with input and the accessibility tree via idbcompanion, including a two-finger pinch), and a macOS** backend, behind a platform-agnostic core.

See it

An agent building a GUI app runs it under glass, reproduces a bug from the accessibility tree (no screenshots), fixes the code, and re-verifies — the loop it otherwise can't close on its own. Try it yourself.

Try it in 60 seconds

and connect it to your agent — glass works with any MCP host (see which are verified).

1. Download glass for your platform from the Releases page

examples/tasks-demo/tasks_demo.py (on Linux it needs sudo apt install python3-gi gir1.2-gtk-4.0).

1. Get the example app — clone this repo, or download

1. Paste this to your agent:

Use glass to run examples/tasks-demo/tasks_demo.py with accessibility on. There's a bug: clicking

Add doesn't add the typed task. Reproduce it by driving the UI and checking the

accessibility tree (don't just screenshot), then find and fix the bug in the code and verify a

task actually appears.

Your agent launches the app, reproduces the bug from the accessibility tree, fixes the one-line wiring bug, and confirms the task appears — the whole build → see → interact → debug loop, start to finish. (glass-mcp doctor checks your environment if anything's off.)

The loop in practice

An agent can fill in a form, click Save, and check that the app reports success. When the app exposes an accessibility tree, the agent works with named controls and verifies changes from text:

glass_start { "run": ["python3", "app.py"] }
glass_do { "actions": [
  { "action": "set_value",
    "target": { "query": "account", "role": "TextField", "states": ["enabled"] },
    "text": "hello" },
  { "action": "click_element",
    "target": { "query": "Save", "role": "Button", "states": ["enabled"] },
    "mode": "auto" },
  { "action": "wait_for_element", "name": "Saved" }
] }
glass_logs

Use glassfindelements to inspect candidates when the intended target is not unique or known well enough to act on directly. Use glassa11ysnapshot to explore the app's controls. See the tool reference for click modes and platform behavior.

Shortened. The full README is on GitHub.