MCP.Market

Calibration Explorer MCP server

by m-sanchez·io.github.m-sanchez/calibration-explorer·v0.2.0
No reviews yet

Check classifier confidence against accuracy, fit temperature scaling and save local reports.

A92/100grade A
Write a reviewInstall
What users say
No reviews yet
Be the first
Safety scan
A92/100

full report

Adoption
Growing

0 stars81 downloads/wk

Reviews

Write one

Nobody has reviewed Calibration Explorer yet.

If you have run it, two minutes of your experience saves the next person an afternoon.

Calibration Explorer tools (6, 1 write)

write = sends, deletes, buys or posts

Read from the package source without running it. The installed server may list more.

  • assess_predictions

    Measure an explicit non-test split using the recorded fit. Returns aggregates, never individual observations.

  • create_reportwrite action

    Write matching HTML and JSON into a new folder under a configured output root. Optional CSV contains selected-split row IDs; optional source data contains all splits/provenance. Raw exports require client elicitation. Never overwrites.

  • fit_temperature

    Fit temperature using calibration logits only. Changing a fit invalidates any locked policy. Test results are not returned.

  • load_predictions

    Load one explicit local CSV/JSON path or the bundled digits reference. Returns metadata only; test outcomes stay unopened. Maximum 16 loaded runs per process.

  • review_test

    Explicitly reveal test outcomes after a user decision through the client's elicitation interface. Without a lock the assessment becomes exploratory. Clients without elicitation cannot reveal test data.

  • set_policy

    Evaluate an explicit threshold on full policy_validation, optionally locking the recorded temperature and threshold. Does not optimise thresholds.

Public scan report

scanner v0.1.10 · 2026-10-14 · same rubric, same numbers if you re-run it

no findings
  • Code scan2 source files scanned25/25
  • –Live reliabilityno gateway calls yet and no remote to proben/a
  • –Tool poisoningtools not inspected (local package is not executed); not countedn/a
  • Auth qualitylocal package, no credentials required12/15
  • Maintenancelast push 9 days ago15/15
  • Maintainer identityregistry namespace matches repository owner; GitHub account older than a year8/10
Overall 92/100. Components that don't apply are left out of the denominator. Any critical finding is an F.RubricAppeal a findingJSON

How Calibration Explorer compares

Against every graded MCP server, from our own scans and uptime checks. Updated after each scan run.

Safety score
92/100
higher than 95% of 48,255 servers
Tools
6
the typical server has 7
Tools that change things
17%
typical: 0%. Fewer write tools means less an agent can break
Downloads a week
81
more than 58% of 12,191 servers with a package

What its tools let an agent do: 4 read · 1 create · 1 change

Install Calibration Explorer in Claude Code, Cursor or VS Code

Runs npx -y @m-sanchez/calibration-explorer-mcp on your machine. Read the scan report first; the gateway never runs local packages.

claude mcp add calibration-explorer -- npx -y @m-sanchez/calibration-explorer-mcp
Add to Cursor

What the publisher says

From the Calibration Explorer repository's README, as published. We do not edit it. Read it on GitHub

Calibration Explorer

Calibration Explorer is for anyone with saved classifier predictions and known outcomes who wants to check whether the model's confidence matches how often it is correct, fit temperature scaling and export a report.

Try it in the browser: open the workbench. It opens with a handwritten-digits example loaded, and your own CSV or JSON is processed in the browser.

Try it from Claude Code: add the local MCP server for the directory you run this in, using two existing absolute directories. It needs Node.js 20 or newer.

claude mcp add calibration -- npx -y @m-sanchez/calibration-explorer-mcp@0.2.0 --input-root ABSOLUTE_PREDICTION_DIRECTORY --output-root ABSOLUTE_REPORT_DIRECTORY

Website copy · Worked example · Numerical library

The MCP server reads selected local files and saves reports directly into your project. The first client target is Claude Code interactive terminal 2.1.283. Raw inputs stay local by default; returned summaries can reach the client's model provider. claude mcp add stores the server in Claude Code's local scope for that directory, and claude mcp remove calibration -s local removes it. Install from npm has the session-only JSON configuration and the Claude Code plugin.

Workflow

  • Inspect: compare confidence and accuracy with reliability bins, ECE, and sample counts. Four controlled examples illustrate sampling variation, binning, temperature scaling, and different group distortions.
  • Calibrate: fit one temperature using calibration logits. Confidence-only inputs support measurement, not temperature fitting or NLL.
  • Decide: choose a threshold on separate policy-validation rows, lock temperature and threshold, then inspect test results.
  • Export: save an HTML report, experiment JSON, or prediction CSV. Configuration links share settings without imported observations.

The app evaluates saved predictions; it does not train or run a classifier. It retains failed fits and preserves original results if scaled evaluation fails.

Follow the short reference walkthrough for the chart, calibration, policy and export sequence. Its linked audit records distinguish actual client/browser observations from scripted checks.

Inputs

IDs must be unique across splits. Logit vectors must use a consistent class order and dimension. JSON accepts an array of rows or the versioned dataset object shown in the template. Missing CSV splits become exploration; the other partitions are calibration, policy_validation, and test. See the CSV template.

The Python preparation recipes write both formats from explicit experiment arrays. Test-only files stay unopened until an explicit review choice. Browser report previews and downloads share one snapshot. Default JSON records require the matching input for reproduction; raw observations remain an explicit option. See the browser/MCP exposure matrix.

Shortened. The full README is on GitHub.

Nothing above is checked by us. What we check is on the safety report.

Calibration Explorer: common questions

Is Calibration Explorer MCP server safe?
Yes, by our scan: it is graded A (92/100). Read the Calibration Explorer safety report
How do I install Calibration Explorer?
It runs on your machine. Copy the Claude Code, Cursor, VS Code or Claude Desktop config from the install section.
Does Calibration Explorer need an API key?
Not as far as the registry entry and our scan can tell: no credentials are declared or required.
Is Calibration Explorer maintained?
The last commit was 30 days ago (2026-10-05). The latest release is v0.2.0.
How does Calibration Explorer compare with other MCP servers?
safety score 92/100 (higher than 95% of 48,255 servers); tools 6 (the typical server has 7); tools that change things 17% (typical: 0%. Fewer write tools means less an agent can break). Full comparison.