Screen Control MCP server

🖥️ Screen Control

A local remote-control system for AI agents: watch your computer's screen

live and send mouse/keyboard commands to it. Everything runs on your

own machine — no data ever leaves it, no cloud middleman.

Every keystroke and every screenshot in this GIF went through the API — the agent never touched a physical keyboard.

Why Screen Control?

AI agents today can write code and call APIs — but they can't see or touch your desktop. Screen Control gives any agent general-purpose computer use over a clean, safety-gated HTTP/MCP interface:

vision-capable models, and a text-only diff endpoint for models that can't consume images at all.

• Perceive — OCR for text, single frames or a live MJPEG stream for

management, background (focus-free) control, virtual desktops.

• Act — absolute and relative mouse, Unicode-safe keyboard, window

stuck-input watchdog and an emergency failsafe are all enforced server-side, no matter how confused the agent gets.

• Stay safe — token auth, blocked deadly shortcuts, focus guard, a

One process, zero configuration, works with any language that can speak HTTP — or natively through MCP in Claude Desktop, Cursor, VS Code and cloud agents.

Performance Is Agent-Bound

Screen Control is the perception and actuation layer — the eyes and hands. The effective speed and capability of any agent using it are bounded by that agent itself and by the environment it runs in:

act → verify loop lives in the agent, so model inference latency and reasoning depth directly set the pace. The API itself adds only milliseconds per call.

• Thinking speed — one action per agent "turn": the perceive → plan →

the agent's context window; a larger window means more situational awareness before verification degrades.

• Context capacity — screen readings (OCR text, frames, diffs) consume

tool-call limits and hosting constraints all stack on top of the loop.

• Runtime environment — network latency, MCP/HTTP round-trip overhead,

In practice this means: the same repo makes a fast reasoning model fast and capable, and makes a slow model slow — the toolchain is not the bottleneck. Real-time or action-heavy tasks need an agent with fast inference and tight tool-loop latency; slower agents should prefer deliberate, verification-heavy tasks.

Table of Contents

Shortened. The full README is on GitHub.