Grounded Support Agent MCP server

Grounded Support Agent

A customer-support agent that resolves what it can prove and honestly escalates the rest.

AI support agents are strong on common questions and dangerous on the edges: asked something the knowledge base does not cover, most will still produce a fluent, confident, wrong answer. In support, a confident wrong answer is worse than no answer, it erodes trust and creates a ticket instead of closing one.

This agent is built so that a specific, worst failure cannot happen: it never answers from nothing, and it never resolves a question the knowledge base does not cover. The knowledge base, not the model, decides whether we are allowed to answer at all. Every answer is grounded in a cited passage. Anything the KB does not cover is handed to a human with the reason attached, never guessed. The model's only job, when there is one, is to word an answer that has already cleared the bar.

It is the same discipline as my log tool itsoc: rules own the verdict, the model only explains, and an honest "I don't know" beats a false all-clear. Here the verdict is resolve or escalate.

The one idea

Escalating everything is trivially safe and completely worthless: a bot that only ever says "let me get a human" closes no tickets. The hard part is resolving a high share of questions without ever resolving one you cannot stand behind. Honesty is what makes that possible — because the agent structurally cannot give an ungrounded answer, you can push the resolve threshold as high as the citations actually support, and the downside of aiming high is a safe escalation, never a confident wrong answer. Honesty is not the tax on the resolution rate; it is what lets you raise it.

Three outcomes, and only three:

The decision is made by deterministic retrieval and term coverage, with explicit, auditable thresholds (core/resolver.py), not by a prompt asking a model to be careful.

Quick start

Python 3.9+, standard library only. No pip install to run the core, no API key, nothing leaves your machine.

python3 ask.py "how do I reset my password?"
python3 ask.py "do you integrate with Salesforce and migrate my Zendesk tickets?"
python3 ask.py --json "can I get a refund after 30 days?"

The first resolves with a citation. The second escalates honestly (nomatch). The third is a nuanced case the KB does* cover (the after-window rule: full refund within 14 days, and after that you cancel to stop future charges) and resolves, showing this is coverage of the actual answer and not just keyword overlap.

The eval that matters

Accuracy on easy questions is table stakes. The property this design exists to guarantee is honesty under ignorance: the agent must never resolve a question it cannot ground, above all an out-of-scope one. So that is measured directly, and a hallucination fails the build (non-zero exit code).

python3 eval/run_eval.py

Shortened. The full README is on GitHub.