CompletionKit MCP server

• agreements_createwrite actionFree

  Upsert an agreement for (run, response, metric, created_by). Verdict is one of agree, disagree, borderline. corrected_score (1..5) is required when verdict is 'disagree'.

• agreements_listFree

  List agreements. Filter by run_id, response_id, metric_id, or created_by.

• datasets_createwrite actionFree

  Create a dataset with CSV data. First row is the header. Two column names are recognized specially: "expected_output" is each row's answer key (ground truth) given to the judge and to checks that compare against the row's expected value, and "actual_output" is a pre-made output to score in a prompt-less run. Both are overridable per run (expected_column / output_column). Every column is also available to the prompt as a variable.

• datasets_create_from_urlwrite actionFree

  Create a dataset by downloading CSV from a URL instead of inlining it. Use this for large datasets: pass a public http(s) URL and the server fetches the CSV directly, so the data never has to pass through the tool-call arguments. The URL is SSRF-checked and the download is capped at 10MB. First row is the header; the "expected_output" (answer key) and "actual_output" (pre-made output) columns are recognized specially, overridable per run.

• datasets_deletewrite actionFree

  Delete a dataset

• datasets_getFree

  Get a dataset by ID

• datasets_listFree

  List all datasets

• datasets_updatewrite actionFree

  Update a dataset

• judges_compareFree

  Compare two versions of one metric's agreement stats side by side. Requires metric_id, metric_version_a_id, and metric_version_b_id (both versions must belong to that metric). Unavailable for check metrics.

• judges_replaywrite actionFree

  Create a scoring run for the current judge over a dataset's existing outputs (wraps runs_create with prompt_id omitted and output_column supplied). This only sets up the run; call runs_generate to actually re-judge the outputs so you can compare against human verdicts.

• metric_groups_createwrite actionFree

  Create a metric group

• metric_groups_deletewrite actionFree

  Delete a metric group

• metric_groups_getFree

  Get a metric group by ID

• metric_groups_listFree

  List all metric groups

• metric_groups_updatewrite actionFree

  Update a metric group

• metric_versions_dismisswrite actionFree

  Destroy a draft MetricVersion (use for either source: 'edit' or source: 'suggestion'). Published versions are refused — to demote a published version, publish a different one as current instead.

• metric_versions_listFree

  List every MetricVersion (drafts + published) for a metric, newest first. Each row carries version_number, state, source, current flag, and timestamps.

• metric_versions_publishwrite actionFree

  Publish a MetricVersion as the live version of its metric. Works for both 'draft → published' and 'revert to an older published version → current'. Transactionally flips current, demotes peers, and writes the version's instruction + rubric_bands back onto the metric so the judge grades against it.

• metrics_createwrite actionFree

  Create a metric with evaluation criteria. For a deterministic check set metric_type:"check" and check_config. Per-kind required keys: value (contains/not_contains/equals), pattern (regex), json_path+expected (json_path_equals), min and/or max (length_bounds); valid_json takes no extra keys. target_path is required when target is json_path. For contains, not_contains, and equals, set compare_to:"expected" to grade against each row's own expected_output (ground truth) instead of a constant value (drop value); add expected_path to dig into the expected value when it is JSON.

• metrics_deletewrite actionFree

  Delete a metric

• metrics_getFree

  Get a metric by ID

• metrics_listFree

  List all metrics

• metrics_suggest_variantsFree

  Ask the model to rewrite the metric's judge instruction in N variants targeted at the recent disagreements. Each variant is saved as a draft MetricVersion with source="suggestion". Returns the persisted drafts. Stripe-metering hooks fire via ActiveSupport::Notifications under completion_kit.judge_suggestion.generated.

• metrics_updatewrite actionFree

  Update a metric. For a deterministic check set metric_type:"check" and check_config. Per-kind required keys: value (contains/not_contains/equals), pattern (regex), json_path+expected (json_path_equals), min and/or max (length_bounds); valid_json takes no extra keys. target_path is required when target is json_path. For contains, not_contains, and equals, set compare_to:"expected" to grade against each row's own expected_output (ground truth) instead of a constant value (drop value); add expected_path to dig into the expected value when it is JSON.

• promptfoo_importFree

  Import a promptfooconfig.yaml. Creates a prompt, a dataset from the test vars, and metrics from the assert blocks (llm-rubric/g-eval become judge metrics; contains/equals/regex/is-json become deterministic check metrics). Returns a summary of what mapped and what was skipped and why; nothing is dropped silently.

• prompts_createwrite actionFree

  Create a prompt

• prompts_deletewrite actionFree

  Delete a prompt

• prompts_getFree

  Get a prompt by ID

• prompts_listFree

  List all prompts

• prompts_publishwrite actionFree

  Publish a prompt version, making it the current version

• prompts_suggest_improvementFree

  Suggest an improved version of a prompt, grounded in a run's test results and judge feedback. Analyzes the run's responses, scores, and reviews, then returns reasoning plus a rewritten template (preserving {{variables}}) and persists it as a Suggestion. Requires a run that has a prompt (not a scoring-only run).

• prompts_updatewrite actionFree

  Update a prompt. If the prompt already has runs, this creates a new DRAFT version (current=false) rather than editing in place or publishing — promote it with prompts_publish — so an agent's edits don't go live without a gate. If it has no runs, it is updated in place.

• provider_credentials_createwrite actionFree

  Create a provider credential

• provider_credentials_deletewrite actionFree

  Delete a provider credential

• provider_credentials_getFree

  Get a provider credential by ID (API key is not exposed)

• provider_credentials_listFree

  List all provider credentials (API keys are not exposed)

• provider_credentials_updatewrite actionFree

  Update a provider credential

• responses_getFree

  Get a specific response

• responses_listFree

  List responses for a run, in row order. Returns {total, limit, offset, returned, responses}. Defaults to 50 rows because full payloads are large: use "fields" to drop the bodies, "min_score"/"max_score" to isolate low scorers, and sort "score_asc" to read the worst rows first. For per-metric averages of the whole run use runs_get instead of aggregating here.

• runs_createwrite actionFree

  Create a run. Omit prompt_id and provide output_column to score existing outputs by grading a pre-existing dataset column instead of generating new ones.

• runs_deletewrite actionFree

  Delete a run

• runs_generateFree

  Start a run. Required for every run, including score-only runs (no prompt): generates responses with the prompt when there is one, otherwise copies the graded dataset column and grades it.

• runs_getFree

  Get a run by ID, including "metric_averages": a per-metric breakdown with each metric's average score (or pass rate for checks), how many rows it graded, and how many scored low. Use this to find the metric dragging a prompt down without listing responses.

• runs_listFree

  List all runs

• runs_regradeFree

  Re-grade a run's existing responses with its currently attached metrics, without regenerating. Use after attaching or editing metrics on an already-generated run.

• runs_rerunwrite actionFree

  Create and start a fresh copy of a run with the same prompt, dataset, metrics, and settings. Use when the judge changed and you want a clean run instead of mixing versions.

• runs_retry_failuresFree

  Re-run only the failed responses of a run, optionally limited to specific response ids via "only".

• runs_updatewrite actionFree

  Update a run

• tags_createwrite actionFree

  Create a tag. Color is auto-assigned.

• tags_deletewrite actionFree

  Delete a tag. Removes the tag from every linked metric, prompt, run, and dataset.

• tags_getFree

  Get a tag by ID

• tags_listFree

  List all tags

• tags_updatewrite actionFree

  Rename a tag.

• usage_getFree

  Get this organization's plan usage and limits for the current billing period: runs and prompt fetches used, their limits, how many remain, and when the period resets. Call this to pre-check quota before starting runs. Runs are hard-blocked once the run limit is reached (with a small grace band), so a run over the limit will fail with run_limit_reached.

• –Code scanremote-only server, no package to scann/a
• Live reliabilityremote reachable in 820ms20/20
• Tool poisoning54 tool descriptions checked15/15
• Auth qualityAPI key sent as a header8/15
• Maintenancelast push 9 days ago15/15
• Maintainer identitynamespace and repository owner differ; GitHub account older than a year; website matches verified namespace8/10

Safety score

88/100

higher than 80% of 4,460 AI and agents servers

Tools

54

the typical AI and agents server has 6

Tools that change things

52%

typical: 0%. Fewer write tools means less an agent can break

Uptime

100%

of our recent checks; the typical remote AI and agents server answers 100%

Response time

697 ms

faster than 17% of 2,331 AI and agents servers

GitHub stars

3

more than 75% of 2,711 AI and agents servers

CompletionKit: common questions

Is CompletionKit MCP server safe?

Yes, by our scan: it is graded A (88/100). Read the CompletionKit safety report

How do I install CompletionKit?

It runs remotely at completionkit.com. Add it to Claude Code, Cursor, VS Code or Claude Desktop with the snippets above, or call it through the mcp.market gateway without installing anything.

Does CompletionKit need an API key?

Yes. The registry entry asks for Authorization.

Is CompletionKit maintained?

The last commit was 49 days ago (2026-09-16). The latest release is v1.0.0.

Is CompletionKit up?

100% of our last 28 checks got an answer. We check remote servers about four times a day.

What can I use instead of CompletionKit?

Servers from other publishers that do the same job: llmtrim MCP server, agent-device MCP server and Pm Claude Skills MCP server.

How does CompletionKit compare with other AI and agents MCP servers?

safety score 88/100 (higher than 80% of 4,460 AI and agents servers); tools 54 (the typical AI and agents server has 6); tools that change things 52% (typical: 0%. Fewer write tools means less an agent can break). Full comparison.