Neuronscope MCP server

NeuronScope

<!-- mcp-name: io.github.RudrenduPaul/neuronscope --> <!-- Ownership-proof string for registry.modelcontextprotocol.io publishing. Do not remove. -->

Ask a language model "why did you say that" and get back the actual attention heads and neurons responsible, as JSON, from the command line or from an agent over MCP.

Install

pip install neuronscope-cli

That gets you the neuronscope command. To install from source instead (for development or to track main):

git clone https://github.com/RudrenduPaul/NeuronScope
cd NeuronScope
pip install -e .

[!NOTE]

The first run of any command downloads the requested model from the HuggingFace Hub

(gpt2 is about 500MB) and prints two lines to stderr that are expected, not errors: a

CPU-fallback notice if you don't have a CUDA GPU, and an unauthenticated-HF-Hub

rate-limit notice. Neither one means anything broke.

Quickstart

neuronscope trace gpt2 "The capital of France is Paris. The capital of Japan is" --top-k 5

Real output from this exact command (stderr trimmed to the two expected warnings mentioned above):

Prompt: The capital of France is Paris. The capital of Japan is
Predicted next token: ' Tokyo'
Top attention heads (by direct logit
            attribution)
┏━━━━━━━┳━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ Layer ┃ Head ┃ Logit attribution ┃
┡━━━━━━━╇━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│     9 │    8 │            4.0679 │
│     8 │   11 │            2.9028 │
│    10 │    7 │           -1.4782 │
│     8 │   10 │           -1.3999 │
│    10 │    0 │            1.1424 │
└───────┴──────┴───────────────────┘
Top MLP neurons (by activation
          magnitude)
┏━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━┓
┃ Layer ┃ Neuron ┃ Activation ┃
┡━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━┩
│    10 │     97 │     7.8394 │
│    11 │    611 │     4.6954 │
│    11 │   2997 │     4.6468 │
│    10 │   1793 │     4.5443 │
│     9 │   1460 │     4.4196 │
└───────┴────────┴────────────┘

gpt2 predicts Tokyo correctly, and head L9H8 is the single biggest contributor to that prediction. Add --json to get the machine-readable version of the same result:

neuronscope trace gpt2 "The capital of France is Paris. The capital of Japan is" --top-k 3 --json

Shortened. The full README is on GitHub.