Win Mind MCP server

WinMind

<!-- mcp-name: io.github.jhonpork1233-beep/win-mind -->

A Windows MCP server that lets an AI see and drive any app through the accessibility tree - no screenshots, no mouse, no pixel guessing - plus 200+ native Windows tools.

Most computer-use agents look at the screen: take a screenshot, send thousands of image tokens to a model, guess coordinates, click, take another screenshot. WinMind reads what Windows already knows. Every app publishes its buttons, text boxes, lists and text to UI Automation, the accessibility layer screen readers like Narrator and NVDA use. WinMind turns that into a compact, numbered tree an AI can read in one call and act on in the next - even when the app is behind other windows.

WhatsApp  (WhatsApp.Root.exe, window)
├─ [6] banner
│  ├─ [7] button "Chats" (42) checked [toggle]
│  ├─ [8] button "Calls" (4) unchecked [toggle]
├─ [20] textbox "Search or start a new chat" [set_value]
├─ [21] tabs "chat-list-filters" (4)
│  ├─ [22] tab "All" selected [select]
│  ├─ [23] tab "Unread 42" [select]
├─ [26] grid "Chat list" (9)
│  ├─ [29] row "Alex Yesterday Photo" [click]
│  ├─ [30] row "Study Group 3 unread messages Yesterday" [click]
└─ [37] button "Send document" [click]

uiact(ref=30) opens that chat. uiact(name="Play", title_contains="spotify") plays music. No screenshot was taken.

Why it's different

Things WinMind handles that trip up simpler UI Automation tools (details in docs/ARCHITECTURE.md):

covered - Chromium's occlusion tracking pauses them. WinMind wakes them properly once; the tree then stays readable in the background. It never pulls a window over a full-screen game.

• Chromium-based apps behind other windows (Discord, Spotify, Steam, Teams, VS Code) expose no content while

elements thousands of times. WinMind splits and de-duplicates: 7.3 s -> 0.25 s.

• WinUI 3 apps that host WebView2 (WhatsApp) loop back into their own web content - naive scans fetch the same

performs the default action of the element that really owns the click - without moving the mouse.

• Web clicks that silently do nothing: pressing a list item whose click handler sits on an inner link. WinMind

saying what was cut; errors are one actionable line, not tracebacks.

• Buttons and text in one tree: editor/document content comes through next to the controls.
• Finds apps by process, not just title (Spotify's title is the song playing), including apps hidden in the tray.
• Token budget: 222 tool definitions cost ~32k tokens (143 per tool); long outputs are trimmed with a note

What's in the box (222 tools)

Shortened. The full README is on GitHub.