MCP.Market

Evals MCP server

by cyanheads·io.github.cyanheads/evals-mcp-server·v0.1.6·1 stars
No reviews yet

Author verifiable eval records through a draft→review→revise→submit loop with enforced graders.

B77/100grade B
Write a reviewInstall
What users say
No reviews yet
Be the first
Safety scan
B77/100

full report

Adoption
Growing

1 stars270 downloads/wk

Reviews

Write one

Nobody has reviewed Evals yet.

If you have run it, two minutes of your experience saves the next person an afternoon.

Evals tools

No tool declarations could be read from the package source. They show once the server is installed.

Public scan report

scanner v0.1.10 · 2026-11-01 · same rubric, same numbers if you re-run it

2 medium
  • Code scan38 source files scanned; 38 source files scanned15/25
  • –Live reliabilityno gateway calls yet and no remote to proben/a
  • –Tool poisoningtools not inspected (local package is not executed); not countedn/a
  • Auth qualitylocal package, no credentials required12/15
  • Maintenancelast push 15 days ago15/15
  • Maintainer identityregistry namespace matches repository owner; GitHub account older than a year8/10

Findings (2)

  • mediumeval / new Function usedexec.eval
    dist/mcp-server/tools/definitions/export-records.tool.js: … lossy), * inspect (Inspect AI), or lm-eval (lm-evaluation-harness). Writes the artif…
  • mediumeval / new Function usedexec.eval
    dist/mcp-server/tools/definitions/export-records.tool.js: … lossy), * inspect (Inspect AI), or lm-eval (lm-evaluation-harness). Writes the artif…
Overall 77/100. Components that don't apply are left out of the denominator. Any critical finding is an F.RubricAppeal a findingJSON

How Evals compares

Against every graded MCP server, from our own scans and uptime checks. Updated after each scan run.

Safety score
77/100
higher than 50% of 48,255 servers
GitHub stars
1
more than 65% of 28,688 servers
Downloads a week
270
more than 80% of 12,191 servers with a package

Release history: 4 versions in the registry since 2026-08-22, latest 2026-10-13, typically one every 15 days.

Install Evals in Claude Code, Cursor or VS Code

Runs npx -y @cyanheads/evals-mcp-server on your machine. Read the scan report first; the gateway never runs local packages.

claude mcp add evals-mcp-server -- npx -y @cyanheads/evals-mcp-server
Add to Cursor

What the publisher says

From the Evals repository's README, as published. We do not edit it. Read it on GitHub

@cyanheads/evals-mcp-server Author verifiable eval records through a draft → review → revise → submit loop with server-enforced graders; compile to JSONL/CSV/Inspect/lm-eval via MCP. STDIO or Streamable HTTP. 9 Tools • 1 Resource

Overview

Verifiable eval records, authored through a draft → review → surgical-revise → submit loop with server-enforced graders. Create a draft carrying its own executable grader, patch it surgically field by field, and submit through a committability gate that requires the gold to pass, a declared negative case to fail, and an independent verification to agree — then compile submitted records to JSONL, CSV, Inspect AI, or lm-evaluation-harness. Runs as a stdio process or a local Streamable HTTP server.

Tools

Resources

Shortened. The full README is on GitHub.

Nothing above is checked by us. What we check is on the safety report.

Evals: common questions

Is Evals MCP server safe?
Mostly: it is graded B (77/100). Read the Evals safety report
How do I install Evals?
It runs on your machine. Copy the Claude Code, Cursor, VS Code or Claude Desktop config from the install section.
Does Evals need an API key?
No secret keys are declared. It reads 15 settings from the environment.
Is Evals maintained?
The last commit was 17 days ago (2026-10-17). The latest release is v0.1.6.
How does Evals compare with other MCP servers?
safety score 77/100 (higher than 50% of 48,255 servers); github stars 1 (more than 65% of 28,688 servers); downloads a week 270 (more than 80% of 12,191 servers with a package). Full comparison.

More from cyanheads →