openclaw-ci-limits skill

OpenClaw CI Limits

Use this skill for CI capacity changes, not ordinary test failure triage. The goal is to keep OpenClaw fast while distinguishing runner registration, runner availability, provider control-plane health, and downstream queue drains.

Resolve the active backend

Read the configured OPENCLAWCIRUNNER_BACKEND and the affected job's actual runner before applying provider-specific guidance. The runner routing guide owns Boat's rollout, physical labels, and hosted fallbacks. Boat uses the hybrid logical plan; a logical Blacksmith label alone does not identify the actual VM.

The owner rollout admits only the documented human source authors and starters on same-repository first attempts. Boat's public-user allowlist is a separate provider check; allowing a bot there does not admit bot-started ordinary CI through the workflow. Fork PRs use GitHub-hosted runners, including first attempts without repository variables. Preserve their coverage. The Blacksmith measurements below apply when a job actually selects Blacksmith.

Core Facts

A green push is not evidence that the full suite passed. Ordinary PRs defer Windows, macOS Node/Swift, iOS build/simulator, real-Gateway UI, published-driver update, Knip, and most architecture proof. Import-cycle and Kysely guards remain blocking. Use docs/ci/scope-and-routing/selection.md and the current manifest for exact event, owner, and release-fallback selection; do not restore deferred lanes while tuning capacity. Android screenshots remain release-only.

• Full main-tier CI runs hourly; ordinary main pushes run security-fast.

registrations, eligible runner availability, Blacksmith capacity/control plane, workflow dependencies, test runtime, or a downstream queue writer.

• Do not assume the scarce resource. Prove whether pressure is runner

5-minute bucket in actionsrunnerregistration. Verify the live bucket before each tuning pass because GitHub can change it. The openclaw organization shares one bucket.

• GitHub runner registrations for openclaw currently report a 10,000 per

actionsrunnerregistration separately; core quota can be healthy while runner registration is throttled.

• Core REST quota does not draw down this bucket. Check

10,000-registration bucket, keep planned Blacksmith burst load under 6,000 registrations per 5 minutes and leave the rest for other repos, retries, and burst overlap.

• Use about 60% of the live bucket as the operating target. With the current

scans should stay on GitHub-hosted runners unless measured evidence says Blacksmith is required.

• Jobs that route, notify, summarize, choose shards, or run short CodeQL quality

Runner Cost Policy

short control jobs hosted and ordinary trusted development proof local.

• Use the smallest runner that completes the required workload reliably. Keep

command and measured peak memory, a smaller-runner OOM, or a controlled comparison showing lower total billed cost. Low CPU use alone does not prove a memory-heavy job can move down. Preserve resource-based worker limits.

• Treat physical 8-vCPU and larger runners as exceptions, not a default or generic retry. Record the

default, including hydration. Keep the 15-minute idle ceiling. Shorter deadlines are welcome for known short commands; do not request four hours automatically or upsize only to obtain more time. Select .github/workflows/ci-check-high-memory-testbox.yml explicitly only for a named memory-heavy command; see docs/reference/test/remote-proof.md. That workflow has at most four concurrent leases inside the shared 32-slot Testbox pool. The high-memory profile uses 8 vCPUs; both profiles preserve their pre-November 3 physical CPU and RAM. All Testbox profiles cap idle time at 15 minutes.

• Routine OpenClaw Testboxes use the 4-class with a 60-minute total-job

the source capsule preserves the checkout being tested. Do not dispatch an old workflow ref to bypass spending limits. Queue age is checked before checkout; an admitted lease keeps its job and idle deadlines.

• Allocate through the current OpenClaw wrapper with workflow ref main;

Keep proven high-memory CI rows scoped to their owning planner and evidence; remeasure before changing their allocation. A 32-class label is not proof of 32 available CPUs. Compare observed resources and total billed job cost.

• Do not promote an entire workflow family because one command needs more RAM.

Rejected Experiments

app trees removed 48.31 MiB of logical tracked content, but did not qualify as a runner-cost improvement. In run 38866595993, the Blacksmith UI E2E checkout took 56s and the hosted browser checkout 58s; same-run full UI checkouts took 22–38s. These are small, unpaired samples, not a measured billing delta. Keep the full source checkout. Reconsider only with lower complete checkout time and billed cost, including filtered fetch, lazy materialization, and trusted-harness export; fewer logical bytes alone do not establish savings.

• UI blobless sparse checkout (2026-10-16): Excluding native mobile/macOS

test/scripts/tsdown-declaration-fixture.ts derives real source copies from its runtime entrypoint closures, so product PRs that edit one do not select the four declaration suites built on it. PR #162614 broke write-unified-entry-dts that way; hourly scheduled CI (run 36979748808) caught it about 35 minutes after merge and 0a188c4f038f0 repaired the fixture within the hour. Declaring the owners as new URL(..., import.meta.url) literals reaches the suites only below aggressive selection's 20-importer hub cutoff: the incident owner and seven others are hubs, so the replayed #162614 diff still selected none. Replaying all 92 main commits from the prior 30 days that touched an owner through the canonical PR planner options, URL literals newly covered 12 (+35 rows, +1,689 class-vCPU-minutes). Full coverage, such as a policy watch over the same list, newly selects the suites for 81 and starts build-artifacts (median 330s on the 16-class across 12 PR runs) for 69, because the suites are build owners in ci-build-manifest.mjs: +242 rows and +11,628 class-vCPU-minutes, a median +161 (+8%) per affected PR. That buys one escaped owner break in seven weeks, so keep the plain paths; hourly scheduled CI, tooling-owner PRs and release validation own this proof. Reconsider with a materially higher escape rate or a selection that does not also start build-artifacts.

• Declaration-fixture owner selection (2026-10-02):

CompilerInputSnapshot.prepare() calls at the three declaration/boundary callers increased full cold validation from 464.21s to 544.68s on the same four-CPU/15.42-GiB Testbox; warm validation increased from 12.62s to 14.63s. All 125 plugin checks and the canary passed, but CPU use also increased. The six caller additions were removed. Do not repeat this as an assumed speedup; any different use needs measured end-to-end benefit.

• Boundary asynchronous input preparation (2026-09-26): Adding the existing

run 35547255790 measured median complete setup at 45.295s cold versus 52.738s restored (+16.4%), despite reusing all 1,453 packages with zero downloads. The 763.8-MiB archive took 23.601–26.980s to restore; producer setup/save added 62.527s. All seven native jobs passed, but the qualification reducer failed on a 24-KiB reported-RAM difference. No assertions were relaxed or jobs rerun. The exactly RAM-matched subset was still 11.3% slower; that is descriptive evidence, not a replacement passing qualification. Keep normal Windows setup uncached. Reconsider only with net complete-setup savings, including restore, extraction, frozen reconciliation, and producer work.

• Windows pnpm store (2026-09-20): Original receipts from

consumer in run 33458856298 successfully saved and restored a 1.66-GB dependency archive, but setup took 142s versus 86s with the ordinary store cache. Extraction took 82s versus 27s; install improved only from 43s to 35s. Keep hosted Mac jobs on the ordinary store cache. Reconsider only with measured total setup savings, including transfer, extraction and frozen reconciliation, not a successful cache hit.

• Hosted Mac exact dependencies (2026-09-01): The same-head publisher and

shared Blacksmith Git fetch with a preflight-produced workspace or .git artifact. PR #124818 measured a 16s Blacksmith checkout baseline versus 7s hosted. The best direct artifact variant cost 1s to pack, 3s to upload, and 11s median to restore; including the serial prefix left only about 1s median improvement and regressed Blacksmith's fast-fetch runs. The official artifact client was worse: run 31971531521 measured 22s median download plus 2s materialization. Blacksmith's fast Actions-cache path does not imply fast Actions-artifact downloads. Reconsider only with measured end-to-end proof for a different transport, including its producer cost and fast-fetch regressions.

• Actions-artifact checkout (2026-08-16): Do not recommend replacing the

First Checks

Before changing CI, collect current pressure:

gh api rate_limit --jq '{core:.resources.core,graphql:.resources.graphql,search:.resources.search,actions_runner_registration:.resources.actions_runner_registration}'
gh run list -R openclaw/openclaw --limit 20 --json databaseId,status,conclusion,workflowName,event,headBranch,createdAt,updatedAt,url
gh run list -R openclaw/clawsweeper --limit 20 --json databaseId,status,conclusion,workflowName,event,headBranch,createdAt,updatedAt,url
gh api repos/openclaw/clawsweeper/actions/runs/<run-id>/jobs --paginate --jq '.jobs[] | {id,name,status,conclusion,labels,created_at,started_at,completed_at,runner_name,runner_group_name}'
blacksmith testbox list --all
curl -fsS https://clawsweeper.openclaw.ai/api/status | jq '{generated_at,fleet,diagnostics:{errors:.diagnostics.errors}}'
curl -fsS https://clawsweeper.openclaw.ai/api/exact-review-queue | jq '{generated_at,review:.lanes.review,publication:.lanes.publication,state_writer,state_append}'
node scripts/ci-run-timings.mjs --latest-main
node scripts/ci-run-timings.mjs --recent 10

For a suspicious queued run, inspect its jobs. A run-level queued status does not reveal whether the job is waiting on dependencies or has no eligible runner. Compare createdat, startedat, labels, and runner_name. Recheck stale queued runs live before canceling them; cancel only runs proven obsolete.

Testbox warmup confirms lease creation, not runner readiness. An initial queued snapshot without providerMetadata.runURL does not establish a capacity shortage. Use that URL to inspect the exact workflow once associated. An assigned check runner performing setup is hydration; a confirmed admission or capacity queue follows the existing stop-owned-allocation policy. Do not infer association from the newest workflow run or wait on a confirmed capacity queue.

scripts/ci-run-timings.mjs start delay can include workflow dependency wait plus runner queue time. It is trend evidence, not runner-pressure proof alone.

Read:

scripts/ci-changed-scope.mjs

• .github/workflows/ci.yml
• .github/workflows/codeql-critical-quality.yml
• docs/ci.md
• test/scripts/ci-workflow-guards.test.ts
• test/scripts/ci-workflow-planning.test.ts
• test/scripts/ci-workflow-evidence.test.ts
• touched planner files under scripts/lib/ci, scripts/lib/test-plan, or

Diagnose The Bottleneck

Classify the issue before changing caps:

Blacksmith/GitHub reports 403/429 or spam-style 422 responses from generate-jitconfig, and API core quota is still healthy. Treat 422 as this signal only when the request payload is otherwise valid. Fix burstiness and Blacksmith job count.

• Runner-registration throttle: many jobs queued before runner assignment,

unavailable capacity. Do not solve this with GitHub workflow fanout alone.

• Blacksmith capacity: Blacksmith dashboard shows actual concurrency caps or

before a lease is returned. This is separate from Actions runner registration and Actions job capacity. Trusted source may use the documented local fallback; untrusted source stays blocked.

• Blacksmith Testbox control plane: list, warm, status, or run calls time out

startedat and runnername remain empty, and no eligible runner exists. Restore an available hosted or registered label; fanout cannot fix it.