Inferbench MCP server

InferBench

<!-- mcp-name: io.github.RudrenduPaul/inferbench -->

Every "best local LLM engine" article benchmarks someone else's machine. InferBench benchmarks yours.

Local-inference engines all publish their own benchmarks, on their own hardware, in their own README. None of them tell you which one is actually fastest on the machine sitting in front of you. InferBench runs a fixed, varied prompt set against whichever supported engines are installed on your own hardware and reports real, measured tokens/second -- not a number copied from someone else's blog post.

Install, first run, and a real omlx benchmark against a cached model:

npx inferbench-cli run --engines llama.cpp --model "bartowski/Qwen2.5-1.5B-Instruct-GGUF:Q4_K_M"

Table of contents

• Install
• Features
• Quickstart
• CLI command reference
• Library API reference
• How the measurement works
• Comparison
• Why this exists
• Documentation
• FAQ
• Contributing
• Security

Install

InferBench ships two independent, equally first-class packages -- pick whichever fits your toolchain, or install both. Neither is deprecated in favor of the other; both run the same measurement architecture against the same two supported engines.

# npm -- JavaScript/TypeScript CLI
npm install -g inferbench-cli
# or, no install:
npx inferbench-cli run --engines llama.cpp --model "<repo>:<quant>"

# PyPI -- Python CLI + library (genuine port, not a wrapper around the Node binary)
pip install inferbench-cli

Both packages are published and installable today. npm install -g inferbench-cli and pip install inferbench-cli both work -- see npmjs.com/package/inferbench-cli and pypi.org/project/inferbench-cli, or python/README.md and docs/getting-started.md for the Python-specific walkthrough, and CHANGELOG.md for each distribution's version history.

Requires Node.js >=18 for the npm package, Python >=3.9 for the PyPI package. At least one supported engine must already be installed either way (InferBench does not install engines for you):

• llama.cpp: brew install llama.cpp (macOS) or build from ggml-org/llama.cpp
• omlx: brew tap jundot/omlx https://github.com/jundot/omlx && brew install omlx (Apple Silicon only)

Features

Shortened. The full README is on GitHub.