Research Repo Doctor MCP server

Research Repo Doctor

<!-- mcp-name: io.github.Tom409114/rrdoctor -->

Get your research artifact ready for Artifact Evaluation before the deadline: scan the repo, scaffold the easy fixes, verify the run path, and generate the appendix.

Web demo for public repos: If the hosted demo is waking up or reinstalling dependencies, the local zero-install command below is the reliable fallback:

uvx rrdoctor scan .

Maintainers can check anonymous demo access with python scripts/checklivedemo.py https://research-repo-doctor-bckncrcwwmg6jrbsrd6btj.streamlit.app/.

rrdoctor is a local CLI and GitHub Action for research artifact preparation. It audits whether a repo is reviewable, citable, and close to runnable; scaffolds safe mechanical fixes; maps findings to an AE-style readiness level; and turns the rest into a checklist any coding agent or human can finish.

Make it a pull-request gate

The GitHub Action is the main adoption path: run the CLI while preparing the artifact, then keep the same deterministic preflight on every pull request.

name: Reproducibility preflight

on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read

jobs:
  rrdoctor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: Tom409114/research-repo-doctor@v0.2.24
        with:
          profile: standard
          fail-on: error

The full GitHub Action guide covers sticky PR comments, new-finding baselines, agent plans, appendices, and complete AE prep packets.

AE deadline loop

uvx rrdoctor prepare . --profile acm --out-dir rrdoctor-prep

# Or run the pieces explicitly:
uvx rrdoctor scan . --profile acm
uvx rrdoctor fix . --write
uvx rrdoctor appendix . --profile acm --output ARTIFACT_APPENDIX.md
uvx rrdoctor verify . --profile acm
uvx rrdoctor verify . --profile acm --run --timeout 600 --fail-on error  # trusted repos only
# Or pin the official quickstart command as the dynamic gate:
uvx rrdoctor verify . --profile acm --command "python train.py config/default.py" --run --timeout 600 --fail-on error

For trusted repositories, rrdoctor verify --run can go beyond static checks. For supported Python repositories it creates a temporary isolated environment, installs declared dependencies, and executes the declared entrypoint there under a timeout. Other ecosystems retain an explicit resolver preflight. With the default gate (--fail-on error), failed or blocked dynamic L2/L3 steps return a nonzero exit code. Use --command when the artifact has a specific smoke-test or quickstart command that reviewers should run. Dynamic mode may run dependency build/install hooks as well as the entrypoint, so never use it on untrusted code. rrdoctor prepare writes the report, agent plan, artifact appendix, and verification ladder into one local evidence directory.

Artifact Evaluation chairs and lab maintainers can use the AE chair guide for optional pre-submission wording and CI examples.

For public calibration evidence, see the 80-repository data brief and the nanoGPT first-run regression case study.

What it catches

Shortened. The full README is on GitHub.