verifyfirst MCP server

verifyfirst

A registry of software failures that report success.

Live at verifyfirst.dev · machine-readable at /registry.json · CC0

A failure that crashes is cheap. It announces itself, and you fix it.

A failure that reports success is expensive. The screenshot looks fine. The command exits zero. The endpoint returns 200. You move on, and the bug ships.

This is a reference for the moment before you claim work is done. It is organised by the instrument you verified with — not by the bug, because the bug is the thing you are trying to find. You know how you looked. Look up what that method cannot see.

$ curl verifyfirst.dev/screenshot.txt

VERIFYFIRST // screenshot
A rendered image — You captured the page and looked at it.

WHAT IT CANNOT SEE
  - Time. A still frame cannot distinguish 'renders one frame then stops'
    from 'renders one frame correctly'.
  - Whether scripts ran at all, as opposed to running and producing this.
  - Why an element is absent: never drawn, drawn transparent, drawn
    offscreen, or covered.
  ...

Three ways in

The six instruments

Entry format

Every entry names the one observation that separates the two hypotheses:

{
  "id": "NS-005",
  "instrument": "exit-code",
  "title": "enable --now does not restart an already-running unit",
  "false_reading": "The command exits zero and the service is active. Conclusion drawn: the new code is live.",
  "true_state": "systemctl enable --now starts a stopped unit. On a running one it is a no-op. The old process, with the old ExecStart, survives.",
  "why_blind": "Exit code zero and `active (running)` are true statements about the wrong process.",
  "discriminating_check": "systemctl show -p ExecStart NAME and ps -p $MAINPID -o args=",
  "cost_of_missing": "A deploy is reported as complete twice while the previous binary keeps serving.",
  "generalises_to": "Any idempotent-looking command whose semantics differ by current state."
}

A check qualifies only if it returns different output under the two hypotheses. An observation that comes out the same either way has confirmed nothing, however much work it took to produce.

MCP server

Query the registry from your own tooling instead of fetching web pages:

claude mcp add verifyfirst -- uvx verifyfirst-mcp

On PyPI and in the official MCP registry as io.github.simulacra/verifyfirst.

Python 3.12 stdlib only — no pip install, no dependencies, works offline from the bundled registry copy. Tools: listinstruments, getinstrument, blindspots, search, getentry, get_protocol.

Shortened. The full README is on GitHub.