LLM Preflight MCP server

LLM Preflight

<!-- mcp-name: io.github.feronovak/llm-preflight -->

Last reviewed: 2026-09-08 · As of: v2.12.0

Catch LLM integration regressions before they ship. LLM Preflight is a local contract preflight for model, prompt, structured-output, and provider-call changes. It runs a small cross-provider preflight and compares validated output, response speed, tokens, and estimated cost.

Try it in 60 seconds

Create and run a deterministic local benchmark—no API key or network request:

python3 -m pip install llm-preflight
llm-preflight init
llm-preflight benchmark.json --no-save

From a source checkout:

python3 -m llm_preflight init
python3 -m llm_preflight benchmark.json --no-save

init never overwrites an existing config. It creates a mock benchmark so you can see the report and exit behavior before making a paid request. Its result is intentionally inconclusive (exit code 3): a local mock validates configuration and output handling, but cannot approve a live model.

Choose your path

route with a candidate using the model-change guide.

• Validate a change. Compare an approved model, prompt, schema, or provider

prepare a bounded candidate smoke with the model-catalogue guide.

• Review a new model. Discover metadata, deliberately probe a route, then

GitHub Marketplace Action or use CI and JSON output.

• Automate an established contract. Add the no-spend

repository with llm-preflight-mcp --workspace "$PWD", then use the MCP server guide for your client configuration.

• Configure a coding agent. Start the local MCP server from a trusted

Safety boundary

flowchart LR
    A[Integration change] --> B[No-spend validation\ndoctor, pricing, dry run]
    B --> C{Human reviews\nevidence and cost bound}
    C -->|Explicit approval| D[Bounded paid smoke]
    C -->|No approval or missing evidence| E[Inconclusive: fix or stop]
    D --> F[Local evidence for\nproduction approval]

LLM Preflight is local evidence, not production approval. It is not a hosted evaluation platform, tracing system, RAG framework, or public leaderboard. Its results apply to your account, network, prompts, and validation rules.

[!WARNING]

Live benchmarks make paid API requests. Start with the no-key demo, preview

the plan before a live run, and keep limits and repetitions small.

CLI, CI, and MCP

Works as a CLI, GitHub Action, and local MCP server. Every path starts with no-spend validation and planning; a live provider run remains an explicit, bounded human-approved step. See the GitHub Action guide or the MCP server guide.

For earlier releases, see the changelog.

Purpose

Mission: help engineers catch LLM integration regressions before shipping a change.

Shortened. The full README is on GitHub.