MCP.Market

LLM Eval Harness MCP server

by razilzul·io.github.razilzul/llm-eval-harness·v1.0.0
No reviews yet

Run a golden set and gate LLM model swaps on accuracy, with latency and cost per request.

B73/100grade B
Write a reviewInstall
What users say
No reviews yet
Be the first
Safety scan
B73/100

full report

Adoption
New

0 stars

Reviews

Write one

Nobody has reviewed LLM Eval Harness yet.

If you have run it, two minutes of your experience saves the next person an afternoon.

LLM Eval Harness tools

No tool declarations could be read from the package source. They show once the server is installed.

Public scan report

scanner v0.1.10 · 2026-10-28 · same rubric, same numbers if you re-run it

no findings
  • –Code scanpackage could not be scannedn/a
  • –Live reliabilityno gateway calls yet and no remote to proben/a
  • –Tool poisoningtools not inspected (local package is not executed); not countedn/a
  • Auth qualitystatic API keys via environment variables6/15
  • Maintenancelast push 10 days ago15/15
  • Maintainer identityregistry namespace matches repository owner; GitHub account older than a year8/10
Overall 73/100. Components that don't apply are left out of the denominator. Any critical finding is an F.RubricAppeal a findingJSON

How LLM Eval Harness compares

Against every graded AI and agents MCP server, from our own scans and uptime checks. Updated after each scan run.

Safety score
73/100
higher than 42% of 4,460 AI and agents servers

Install LLM Eval Harness in Claude Code, Cursor or VS Code

claude mcp add llm-eval-harness -- docker run -i --rm ghcr.io/razilzul/llm-eval-harness:1.0.0
Add to Cursor

LLM Eval Harness: common questions

Is LLM Eval Harness MCP server safe?
Mostly: it is graded B (73/100). Read the LLM Eval Harness safety report
How do I install LLM Eval Harness?
It runs on your machine. Copy the Claude Code, Cursor, VS Code or Claude Desktop config from the install section.
Does LLM Eval Harness need an API key?
Yes. The registry entry asks for ANTHROPIC_API_KEY, GEMINI_API_KEY.
Is LLM Eval Harness maintained?
The last commit was 17 days ago (2026-10-18). The latest release is v1.0.0.
What can I use instead of LLM Eval Harness?
Servers from other publishers that do the same job: Emilia Protocol MCP server, Decionis Commerce Gate MCP server and llmtrim MCP server.

Alternatives to LLM Eval Harness

Same job from other publishers: the closest match first, then the best rated.

  • Emilia Protocol
    Exact-action approval for consequential agent actions: request, track, and verify signed receipts.
    B
  • Decionis Commerce Gate MCP
    Commerce preflights, D365 authorization; opt-in agent writes require authorized backend connectors.
    A
  • llmtrim
    MCP server and proxy that compresses LLM prompts, tool output, and replies to cut token cost.
    A
  • Ausca
    Pay-per-call APIs and MCP services for agents, no accounts or keys, with verifiable receipts.
    A
  • Claude Prompts
    Reusable prompt templates, multi-step workflow chains, and quality gates for any MCP client.
    A