Vidin MCP server

vidin

Turn a video into something a coding agent can actually read.

Claude Code, Codex and friends accept images, not video. So when a bug report arrives as a screen recording, the video is dead weight — someone has to watch it and retype what happened. vidin watches it instead: it converts the recording into a small set of annotated keyframes plus a written timeline, and serves them to agents over MCP.

A 20-second recording in, six frames out: the refund panel, the 500 dialog, the retry toast, the recovery — each boxed and timestamped. (The clip is vidin's synthetic test fixture.)

changes and then comes to rest. Three minutes of someone reading costs almost nothing; every real UI event is captured.

• Event-driven, not one-frame-per-second. A frame is kept when the screen

An hour-long one costs time, not RAM.

• Fast. A 3-minute 1080p recording takes ~4 seconds on an M-series Mac.

is ffmpeg; no sharp, no headless browser, no native image library.

• One command to run. npx @teunlao/vidin — all it asks of the machine

contact sheet first; full-detail frames only for the moments that turn out to matter.

• Shaped for agent context. The MCP tools return a written report and one

00:41.2" and you jump straight there in the original file.

• Timestamps you can scrub to. An agent can say "the error appears at

Quickstart

brew install ffmpeg        # + yt-dlp, only if you'll pass URLs

Hook it into Claude Code:

claude mcp add vidin -- npx -y @teunlao/vidin mcp

Or in .mcp.json / ~/.codex/config.toml equivalents:

{
  "mcpServers": {
    "vidin": { "command": "npx", "args": ["-y", "@teunlao/vidin", "mcp"] }
  }
}

Or straight from the shell:

npx @teunlao/vidin ~/Desktop/bug.mov       # → ./.vidin/bug/
npm i -g @teunlao/vidin                    # or put `vidin` on PATH for good
vidin https://example.com/screencast.mp4   # downloaded with yt-dlp
vidin zoom bug.mov --from 00:41 --to 00:45 --fps 6
vidin probe bug.mov

What comes out

out/
  REPORT.md        the entry point: timeline table, what changed where, what to open
  manifest.json    the same, machine readable
  sheets/          the whole video as labelled 3x3 grids (~1.3k tokens each)
  frames/          full-detail annotated frames

Each frame carries a red dashed box around the regions that changed since the previous kept frame, and a bottom bar with the frame number, the timestamp in the original video, the gap since the previous frame, why the frame was kept, and how much of the screen changed.

The MCP tools

Four tools, shaped so an agent spends context in the right order:

The intended loop: analyse → look at the sheet → read the timeline → open two or three frames → zoom in if the moment between them is still unclear. Opening everything up front would burn the context before knowing which second matters.

Profiles and tuning

Shortened. The full README is on GitHub.