Quant Research MCP server

quant-integrity

Statistical tests for whether a backtest means anything.

**This library will not find you a profitable strategy. Its function is to reject

them.** Everything here exists to answer one question: given how many

configurations you tried before reporting this one, does the result survive?

pip install quant-integrity

The problem, in one picture

A moving-average crossover on BTC-USD hourly bars. After sweeping parameters, the best configuration returned an annualised Sharpe of 0.95 — and a Probabilistic Sharpe of 0.86, which reads as "probably beats zero".

That is where most backtests stop. Here is the same result next to the bar that 28 attempts actually sets:

The best of 28 tries is large even when every underlying edge is exactly zero. Against that bar the deflated Sharpe is 0.157. The observed result is not marginal; it is below half of what luck alone produces.

A high PSR with a low DSR is the signature of an overfit search, not a borderline edge.

What the search was actually selecting on

The cause was visible in the trials themselves. Sorting the 28 recorded runs by trade count against Sharpe:

Every configuration that traded less looked better, at ρ = −0.96. That is not a signal being discovered — it is cost drag being measured, plus a slow moving average approximating buy-and-hold in a rising market. The search was not finding an edge. It was finding the configuration that traded least.

And the interval was never tight to begin with

Reported alone, Sharpe = 0.95 implies a precision the data does not support.

Every number above is reproducible

git clone https://github.com/abhayjnayakk/quant-integrity
cd quant-integrity && pip install -e .
python docs/make_figures.py        # regenerates all three figures and prints the values

The trial count is taken from the recorded runs rather than typed in. An earlier version of this page hardcoded 29 while the file recorded 28, and quoted a deflated Sharpe from one and a minimum backtest length from the other.

The figures and this table are generated by the same script from examples/btcmaruns.npz (24 KB, the actual return series and per-trial results). No server and no account. There is also a longer walkthrough in examples/btcmapostmortem.ipynb.

Shortened. The full README is on GitHub.