Cidiff MCP server

cidiff

Which of these CI failures are actually yours?

A red PR with 12 failing checks usually has one real problem. The rest were already failing on the base branch, belong to a deploy preview service, or are waiting on a maintainer to approve the workflow run. Today you find that out by clicking through every check twice, once on the PR and once on the base, and comparing by eye.

cidiff does that comparison for you. It reads the check runs and commit statuses on the PR head commit and on its base commit, and classifies every check:

Exit code 1 when introduced failures exist, 0 otherwise, so it drops straight into a CI gate or a maintainer's triage script.

The honesty rules

cidiff was extracted from the CI triage inside Greenlight, where it classified 89 failing checks across 46 open PRs in one sweep and found zero introduced. Getting that number right depends on refusing to guess:

• No base evidence means indeterminate, never a verdict. A failing head check with no base run is reported as unproven, in both directions.
• Base failures are only "fixed" when the head actually ran checks. On day one, dogfooding against a real fork PR (usebruno/bruno #9569) showed the trap: the PR's check suites were stuck at action_required, waiting for a maintainer to approve the workflow, so the head commit had zero check runs. A naive differ reported every base failure as fixed and every base pass as passing. cidiff detects the empty head, reads the check suites, and says in plain warnings: CI has not run on this head, these failures cannot be classified yet.
• Legacy commit statuses count as evidence. Snyk, CodeRabbit, and older CI systems report through the Statuses API, not check runs. cidiff merges both, with real check runs winning on name collisions.
• External blockers classify first. A failing Vercel preview is not a test failure, and a check that needs maintainer approval is not a code failure. Both are reported as external before any base comparison happens.

Usage

CLI

# Classify a PR's checks against its base
npx @reneturcios/cidiff pr usebruno/bruno 9569

# Machine readable output
npx @reneturcios/cidiff pr owner/repo 123 --json

# Compare two commits directly
npx @reneturcios/cidiff runs owner/repo --head <sha> --base <sha>

Example output (the fork PR case above, real output):

cidiff usebruno/bruno PR #9569: 0 introduced, 0 pre-existing, 0 external, 0 fixed, 6 indeterminate, 0 pending, 2 passing
  [WARNING] No check runs on head commit 885c6fafed1e (only 2 commit status(es)). The workflow system has produced no evidence to diff against; base check failures below are indeterminate, not fixed.
  [WARNING] 3 check suite(s) on the head commit are action_required: CI is waiting on maintainer approval to run (typical for fork PRs). The failures cannot be classified until CI actually runs.
  [INDETERMINATE] Unit Tests (Linux): Failing on base, and the head commit has no check runs, so there is nothing to compare. CI may not have run on the head yet
  [PASSING] CodeRabbit, security/snyk (Bruno)
Verdict: no introduced failures. Any failing checks are pre-existing, external, or unproven either way (indeterminate).

Shortened. The full README is on GitHub.