MCP.Market

Evalview MCP server

by hidai25·io.github.hidai25/evalview-mcp·v0.6.0·135 stars
No reviews yet

Regression testing for AI agents. Golden baselines, CI/CD, LangGraph, CrewAI, OpenAI, Claude.

B75/100grade B
Write a reviewInstall
What users say
No reviews yet
Be the first
Safety scan
B75/100

full report

Adoption
Established

135 stars282 downloads/wk

Reviews

Write one

Nobody has reviewed Evalview yet.

If you have run it, two minutes of your experience saves the next person an afternoon.

Evalview tools

No tool declarations could be read from the package source. They show once the server is installed.

Public scan report

scanner v0.1.9 · 2026-09-27 · same rubric, same numbers if you re-run it

1 medium
  • Code scan174 source files scanned20/25
  • –Live reliabilityno gateway calls yet and no remote to proben/a
  • –Tool poisoningtools not inspected (local package is not executed); not countedn/a
  • Auth qualitystatic API keys via environment variables6/15
  • Maintenancelast push 23 days ago15/15
  • Maintainer identityregistry namespace matches repository owner; GitHub account older than a year8/10

Findings (1)

  • mediumsubprocess with shell=Trueexec.shell-true
    evalview-0.6.0/evalview/chat.py: … cmd, shell=True, cwd=os.get…
Overall 75/100. Components that don't apply are left out of the denominator. Any critical finding is an F.RubricAppeal a findingJSON

How Evalview compares

Against every graded Developer tools MCP server, from our own scans and uptime checks. Updated after each scan run.

Safety score
75/100
higher than 40% of 1,119 Developer tools servers
GitHub stars
135
more than 95% of 854 Developer tools servers
Downloads a week
282
more than 76% of 430 Developer tools servers with a package

Install Evalview in Claude Code, Cursor or VS Code

claude mcp add evalview-mcp -- uvx evalview
Add to Cursor

What the publisher says

From the Evalview repository's README, as published. We do not edit it. Read it on GitHub

<!-- mcp-name: io.github.hidai25/evalview-mcp --> <!-- keywords: AI agent testing, regression detection, golden baselines -->

Snapshot testing for AI agents. Record what your agent does today. Get told when it silently changes.

Your agent returns 200 and looks fine. But a model update, a provider change, or a one-line prompt edit just made it skip a clarification, call the wrong tool, or quietly drop output quality. Your tests still pass. Your users notice before you do.

EvalView snapshots your agent's behavior — the tools it calls, in what order, with what output — and tells you the moment that behavior changes. Like Jest snapshots, but for tool-calling, multi-turn agents.

↑ 30-second live demo — no API key needed

Quick Start

OpenAI adapter migration: OpenAI shut down the Assistants API on August 26, 2026.

The latest published EvalView release, 0.8.1, still uses that API; the Responses API

migration is currently unreleased source. If you use openai-assistants, follow the

migration guide before running your tests. An assistant_id

alone cannot preserve your agent's configuration. Other adapters are unaffected.

pip install evalview
evalview snapshot    # Record your agent's current behavior as the baseline
evalview check       # After any change, diff against the baseline

That's the whole loop. check returns one of:

✓ login-flow        PASSED          behavior matches baseline
  ⚠ refund-request    TOOLS_CHANGED   called a different tool, or in a different order
  ✗ billing-dispute   REGRESSION      score dropped — output quality fell

It diffs the whole trajectory — tool names, parameters, and order — not just the final string. The deterministic tool + sequence diff runs offline, with no API key. Add an LLM judge only when you want output-quality scoring.

Executing your agent can still incur backend API charges: --no-judge skips the judge, not those calls. Embedding-based semantic comparison is opt-in.

No agent yet? See it work in 30 seconds:

evalview demo

Why snapshot testing (and not assertions)?

Most eval tools ask you to write down what "good" looks like — assertions, metrics, rubrics. That's a lot of upfront work, and you can only catch the failures you thought to assert.

Shortened. The full README is on GitHub.

Nothing above is checked by us. What we check is on the safety report.

Evalview: common questions

Is Evalview MCP server safe?
Mostly: it is graded B (75/100). Read the Evalview safety report
How do I install Evalview?
It runs on your machine. Copy the Claude Code, Cursor, VS Code or Claude Desktop config from the install section.
Does Evalview need an API key?
Yes. The registry entry asks for OPENAI_API_KEY.
Is Evalview maintained?
The last commit was 58 days ago (2026-09-05). The latest release is v0.6.0.
What can I use instead of Evalview?
Servers from other publishers that do the same job: Observatory MCP server.
How does Evalview compare with other Developer tools MCP servers?
safety score 75/100 (higher than 40% of 1,119 Developer tools servers); github stars 135 (more than 95% of 854 Developer tools servers); downloads a week 282 (more than 76% of 430 Developer tools servers with a package). Full comparison.

Alternatives to Evalview

Same job from other publishers: the closest match first, then the best rated.

  • Observatory
    MCP security scanner. CI-native testing, attack simulation, health scoring, and SARIF.
    B