Mmcp.market

autoresearch skill

by github·github/awesome-copilot·39k stars·MIT

Autonomous iterative experimentation loop for any programming task. Guides the user through defining goals, measurable metrics, and scope constraints, then runs an autonomous loop of code changes, testing, measuring, and keeping/discarding results. Inspired by Karpathy''s autoresearch. USE FOR: autonomous improvement, iterative optimization, experiment loop, auto research, performance tuning, automated experimentation, hill climbing, try things automatically, optimize code, run experiments, autonomous coding loop. DO NOT USE FOR: one-shot tasks, simple bug fixes, code review, or tasks without a measurable metric.

A100/100content scan

Is the autoresearch skill safe?

Clean: nothing in its files matched our rules. We read 1 file in the folder on 2026-09-28.

No findings.

Install the autoresearch skill

A skill is a folder. Copy it into your agent's skills folder and the agent loads it when the task matches its description. 

git clone --depth 1 https://github.com/github/awesome-copilot.git /tmp/awesome-copilot
mkdir -p ~/.claude/skills
cp -r /tmp/awesome-copilot/skills/autoresearch ~/.claude/skills/autoresearch
available in every project

In the Claude apps, zip the folder and upload it from the Skills settings. The folder on GitHub

The instructions your agent would load

SKILL.md as published, without the frontmatter. Read it on GitHub

Autoresearch: Autonomous Iterative Experimentation

An autonomous experimentation loop for any programming task. You define the goal and how to measure it; the agent iterates autonomously -- modifying code, running experiments, measuring results, and keeping or discarding changes -- until interrupted.

This skill is inspired by Karpathy's autoresearch, generalized from ML training to any programming task with a measurable outcome.

Agent Behavior Rules

  1. DO guide the user through the Setup phase interactively before starting the loop.
  2. DO establish a baseline measurement before making any changes.
  3. DO commit every experiment attempt before running it (so it can be reverted cleanly).
  4. DO keep a results log (TSV) tracking every experiment.
  5. DO revert changes that do not improve the metric (git reset to last known good).
  6. DO run autonomously once the loop starts -- never pause to ask "should I continue?".
  7. DO NOT modify files the user marked as out-of-scope.
  8. DO NOT skip the measurement step -- every experiment must be measured.
  9. DO NOT keep changes that regress the metric unless the user explicitly allowed trade-offs.
  10. DO NOT install new dependencies or make environment changes unless the user approved it.

Phase 1: Setup (Interactive)

Before any experimentation begins, work with the user to establish these parameters. Ask the user directly for each item. Do not assume or skip any.

1.1 Define the Goal

Ask the user:

What are you trying to improve or optimize?

Examples: execution time, memory usage, binary size, test pass rate, code coverage,

API response latency, throughput, error rate, benchmark score, build time, bundle size,

lines of code, cyclomatic complexity, etc.

Record the user's answer as the goal.

1.2 Define the Metric

Ask the user:

How do we measure success? What exact command produces the metric?

I need:

1. The command to run (e.g., dotnet test, npm run benchmark, time ./build.sh, pytest --tb=short)

2. How to extract the metric from the output (e.g., a regex pattern, a specific line, a JSON field)

3. Direction: Is lower better or higher better?

Example: "Run dotnet test --logger trx, count passing tests. Higher is better."

Example: "Run hyperfine './my-program', extract mean time. Lower is better."

Record:

  • METRIC_COMMAND: the command to run
  • METRIC_EXTRACTION: how to extract the numeric metric from output
  • METRICDIRECTION: lowerisbetter or higheris_better

1.3 Define the Scope

Ask the user:

Which files or directories am I allowed to modify?

And which files are OFF LIMITS (read-only)?

Record:

  • INSCOPEFILES: files/dirs the agent may edit
  • OUTOFSCOPE_FILES: files/dirs that must not be modified

1.4 Define Constraints

Ask the user:

Are there any constraints I should respect?

Examples:

- Time budget per experiment (e.g., "each run should take < 2 minutes")

- No new dependencies

- Must keep all existing tests passing

- Must not change the public API

- Must maintain backward compatibility

- VRAM/memory limit

- Code complexity limits (prefer simpler solutions)

Record as CONSTRAINTS.

1.5 Define the Experiment Budget (Optional)

Ask the user:

How many experiments should I run, or should I just keep going until you stop me?

You can say a number (e.g., "try 20 experiments") or "unlimited" (I'll run until you interrupt).

Record as MAX_EXPERIMENTS (number or unlimited).

1.6 Simplicity Criterion

Inform the user of the default simplicity policy:

Simplicity policy (default): All else being equal, simpler is better. A small improvement

that adds ugly complexity is not worth it. Removing code while maintaining or improving

the metric is a great outcome. I'll weigh the complexity cost against the improvement

magnitude. Does this policy work for you, or do you want to adjust it?

Record any adjustments as SIMPLICITY_POLICY.

1.7 Confirm Setup

Summarize all parameters back to the user in a clear table:

Ask the user to confirm. Do not proceed until confirmed.

Phase 2: Branch & Baseline

Once the user confirms:

More skills from github/awesome-copilot

  • Aacquire-codebase-knowledgeUse this skill when the user explicitly asks to map, document, or onboard into an existing codebase. Trigger for prompts like "map this codebase", "document this architecture", "onboard me to this repo", or "create codebase docs". Do not trigger for routine feature implementation, bug fixes, or narrow code edits unless the user asks for repository-level discovery.
  • Aacreadiness-assessRun the AgentRC readiness assessment on the current repository and produce a static HTML dashboard at reports/index.html. Wraps `npx github:microsoft/agentrc readiness` and hands off rendering to the @ai-readiness-reporter custom agent. Supports policies (--policy) for org-specific scoring. Use when asked to assess, audit, or score the AI readiness of a repo.
  • Aacreadiness-generate-instructionsGenerate tailored AI agent instruction files via AgentRC instructions command. Produces .github/copilot-instructions.md (default, recommended for Copilot in VS Code) plus optional per-area .instructions.md files with applyTo globs for monorepos. Use after running /acreadiness-assess to close gaps in the AI Tooling pillar.
  • Aacreadiness-policyHelp the user pick, write, or apply an AgentRC policy. Policies customise readiness scoring by disabling irrelevant checks, overriding impact/level, setting pass-rate thresholds, or chaining org baselines with team overrides. Use when the user asks about strict mode, AI-only scoring, custom weights, CI gating, or wants org-wide standardisation.
  • Aad-campaign-analyzerUse this skill when the user shares ad campaign performance data and asks what to cut, scale, or test. Trigger for prompts like "analyze my ad campaigns", "where am I wasting ad spend", "reallocate my ad budget", "which ads are actually working", or "ROAS analysis". Do not trigger for campaign planning or creative generation without performance data.
  • Aadd-educational-commentsAdd educational comments to the file specified, or prompt asking for file to comment if one is not provided.
  • Aadobe-illustrator-scriptingWrite, debug, and optimize Adobe Illustrator automation scripts using ExtendScript (JavaScript/JSX). Use when creating or modifying scripts that manipulate documents, layers, paths, text frames, colors, symbols, artboards, or any Illustrator DOM objects. Covers the complete JavaScript object model, coordinate system, measurement units, export workflows, and scripting best practices.
  • Aagent-architectureDesign AI agent architectures through requirements discovery, or audit and diagnose architectural flaws in existing agents. Architecture only; excludes implementation and general code review.
  • Aagent-governancePatterns and techniques for adding governance, safety, and trust controls to AI agent systems. Use this skill when: - Building AI agents that call external tools (APIs, databases, file systems) - Implementing policy-based access controls for agent tool usage - Adding semantic intent classification to detect dangerous prompts - Creating trust scoring systems for multi-agent workflows - Building audit trails for agent actions and decisions - Enforcing rate limits, content filters, or tool restrictions on agents - Working with any agent framework (PydanticAI, CrewAI, OpenAI Agents, LangChain, AutoGen)
  • Aagent-owasp-complianceCheck any AI agent codebase against the OWASP Agentic Security Initiative (ASI) Top 10 risks. Use this skill when: - Evaluating an agent system's security posture before production deployment - Running a compliance check against OWASP ASI 2026 standards - Mapping existing security controls to the 10 agentic risks - Generating a compliance report for security review or audit - Comparing agent framework security features against the standard - Any request like "is my agent OWASP compliant?", "check ASI compliance", or "agentic security audit"
  • Aagent-skill-stackFind, evaluate, and assemble the smallest compatible set of AI Agent Skills for an end-to-end natural-language goal. Use when a user wants Skills for a multi-step workflow, asks which Skills fit a project, needs an installed-Skill audit or conflict check, has low Skill recall, wants indirect helpers such as humanizers or compliance checks, or wants a project-specific Skill Stack with controlled installation. Search local Skills, registries, GitHub, and OpenCLI; compare adoption, verified fit, safety, and overlap. Do not use for locating one known or common Skill; use the generic find-skills workflow.
  • Aagent-supply-chainVerify supply chain integrity for AI agent plugins, tools, and dependencies. Use this skill when: - Generating SHA-256 integrity manifests for agent plugins or tool packages - Verifying that installed plugins match their published manifests - Detecting tampered, modified, or untracked files in agent tool directories - Auditing dependency pinning and version policies for agent components - Building provenance chains for agent plugin promotion (dev → staging → production) - Any request like "verify plugin integrity", "generate manifest", "check supply chain", or "sign this plugin"

All agent skills → · MCP servers