release-openclaw-plugin-testing skill

OpenClaw Pre-Release Plugin Testing

Use this skill when the user asks for plugin release confidence, plugin lifecycle sweeps, package-artifact plugin proof, or "what else should we test before release?" It complements openclaw-testing; use that skill too when choosing the cheapest safe runner or debugging a failing lane.

Goal

Prove the plugin system as a product surface, not just as source tests:

• bundled plugin lifecycle: install, inspect, enable, disable, uninstall
• package artifact behavior from a clean HOME
• doctor/fix/config validation and idempotence
• config discovery and config round-trip
• status/log visibility and diagnostics
• gateway startup/bootstrap with plugin metadata snapshots
• public SDK compatibility for real external plugins
• live-ish provider/channel probes only when safe credentials exist

First Checks

From the OpenClaw repo root:

pnpm docs:list
git status --short --branch
pnpm changed:lanes --json

Follow openclaw-testing for dependency ownership and the choice of local or remote proof.

Runner Choice

Prefer this order:

when it has the required dependencies and capabilities.

1. GitHub Package Acceptance for installable-package product proof.
2. Current dedicated Linux worker for trusted source/package/Docker proof

seeded dist, dist-runtime, and package caches.

1. ci-build-artifacts-testbox.yml Testbox when Docker/package lanes need

package-boundary checks, or focused Docker lanes.

1. ci-check-testbox.yml Testbox for source checks, targeted Vitest,

1. Workstation targeted commands only for small format/static/unit probes.

Avoid long package Docker runs from a stale sparse worktree. If Testbox sync reports hundreds of changed files or starts deleting package inputs, stop and warm a fresh box from current main, or switch to Package Acceptance.

Existing Baseline

Run or verify these before inventing new coverage:

OPENCLAW_TESTBOX=1 pnpm check:changed
pnpm run test:extensions:package-boundary:canary
pnpm run test:extensions:package-boundary:compile
pnpm test:docker:plugins
OPENCLAW_PLUGINS_E2E_CLAWHUB=0 pnpm test:docker:plugins
pnpm test:docker:plugin-update

For full bundled install/uninstall proof, shard the packaged sweep:

OPENCLAW_BUNDLED_PLUGIN_SWEEP_TOTAL=8 \
OPENCLAW_BUNDLED_PLUGIN_SWEEP_INDEX=<0-7> \
pnpm test:docker:bundled-plugin-install-uninstall

This example partitions the selected package's plugin inventory over shards 0-7. Private QA plugins are source-mode only unless a package explicitly includes them.

Confidence Matrix

Use this matrix for pre-release signoff. Record pass/fail, run URL/Testbox ID, package SHA/version, and skipped-live reason.

Package Acceptance Plan

Full Release Validation owns final candidate qualification with Q=C and the exact publication tarballs. Use standalone Package Acceptance for diagnostics or postpublish confidence, not as a replacement candidate qualification:

gh workflow run package-acceptance.yml \
  --repo openclaw/openclaw \
  --ref main \
  -f workflow_ref=main \
  -f source=ref \
  -f package_ref=<branch-or-sha> \
  -f suite_profile=custom \
  -f docker_lanes='plugins-offline plugin-update doctor-switch update-channel-switch config-reload mcp-channels npm-onboard-channel-agent' \
  -f telegram_mode=mock-openai

Use source=npm -f package_spec=openclaw@beta for published beta proof. These standalone diagnostic/confidence runs may use trusted current harness code; candidate qualification must retain the frozen candidate's harness instead.

For extended-stable shared publication, require complete exact-target Full Release Validation from the admitted candidate-owned release-ci/* harness at Q=C, with trusted P admission and publication tooling separate. Direct canonical-branch or main producers do not satisfy the protected publisher. Package Acceptance is a post-publish selector smoke:

gh workflow run package-acceptance.yml \
  --repo openclaw/openclaw \
  --ref main \
  -f workflow_ref=main \
  -f source=npm \
  -f package_spec=openclaw@extended-stable \
  -f suite_profile=package \
  -f telegram_mode=mock-openai

Record the resolved version. Still verify every package and selector in the tag's all-publishable inventory; one smoke is not registry readback.

Plugin npm Artifact Qualification

For a publication candidate, Full Release Validation owns plugin npm artifact qualification. Supply its publication selection at dispatch; the all-group parent invokes plugin-npm-release.yml in artifact-only mode against the exact Release SHA, requires successful tarball readback, and records the immutable aggregate descriptor in publicationArtifacts.pluginNpm. Release Prepare and publication must adopt that exact descriptor rather than repacking plugins.

Use a standalone trusted-workflow preflight only for a focused diagnostic or a selected-plugin repair that is outside a regular publication candidate:

release_sha="$(git rev-parse origin/release/2026.7.1)"
gh workflow run plugin-npm-release.yml \
  --repo openclaw/openclaw \
  --ref main \
  -f preflight_only=true \
  -f publish_scope=selected \
  -f plugins=@openclaw/meta-provider \
  -f ref="${release_sha}" \
  -f npm_dist_tag=default

Do not pass releasepublishrunid. Require the workflow to finish verifypluginnpmpreflight successfully. Record the run URL, workflow SHA, and source SHA. The workflow first creates the staging/readback artifact plugin-npm-package-source-- containing npm-pack.json, preflight-manifest.json, and the tarball. It then uploads the final consumer artifact plugin-npm-package----- containing the tarball and plugin-publication-manifest.json.

Record the final artifact name and digest separately. The manifest uses openclaw.plugin-publication-artifact/v1 and records the target SHA, package manifest hashes, publication route and policy, and tarball hashes and inventory. This standalone proof is validation-only; it does not authorize or stage publication and cannot replace Full Release Validation's manifest-bound descriptor. The artifact inventory, rather than the unpacked source tree, is the security and package-content boundary: source-only fixtures are irrelevant. Package-owned test and fixture paths outside shipped runtime and skill assets fail qualification; shipped runtime remains security-scanned. Bundled nodemodules stays with dependency evidence rather than plugin-source policy. The separate trustedpublisher_preflight=true OIDC check requires a protected release-publish/- dispatch tag and runs in npm-publish. Real publication also requires that tooling tag; a direct human dispatch waits for its npm-release approval job before publishing. For an already-published version, require npm dist.integrity and dist.shasum to match the verified tarball. Treat only missing or provably older dist-tags as repairable; newer or incomparable selectors are a blocker.

New Testbox Harness Plan

If more certainty is needed, add or run a plugin-lifecycle-matrix Docker lane that uses one package tarball and sharded plugin lists. Per plugin:

while its allow/deny entries, install record, managed directory, and bundled runtime load paths are gone. Use the existing harness's source-qualified uninstall expectations for historical targets.

1. Start with a clean HOME.
2. Capture plugins list --json.
3. plugins install .
4. plugins inspect --json.
5. plugins disable , then assert disabled visibility.
6. plugins enable , except config-required plugins without config.
7. plugins registry --refresh.
8. doctor --non-interactive.
9. plugins uninstall --force.
10. Assert the plugin's plugins.entries value is exactly { enabled: false },

secret-looking values.

1. Assert diagnostics contain no level: "error" and output redacts

Keep memory-lancedb special: it is config-required. First assert install does not enable it without embedding config, then run a second configured case.

Doctor/Fix Matrix

Seed bad states and require doctor --fix --non-interactive to repair them, then run doctor again and require idempotence:

• stale plugins.allow
• stale plugins.entries
• stale channel config for missing channel plugin
• invalid plugins.entries..config
• packaged bundled path in plugins.load.paths
• legacy plugins.installs
• disabled channel/plugin config that must not stage runtime deps
• root-owned global package tree that must remain unmodified

Gateway Bootstrap Matrix

Start packaged OpenClaw in Docker with clean state:

• provider plugins enabled, no credentials: ready with warnings, no crash
• channel plugins configured disabled: no runtime deps staged
• startup-activation plugins enabled: ready and reflected in status
• invalid single plugin config: bad plugin skipped/quarantined, others remain

Assert:

• gateway reaches ready
• openclaw status --json includes plugin diagnostics
• openclaw plugins inspect --all --json is parseable
• package tree is not mutated
• logs contain no raw tokens

Config Round-Trip Representatives

Use representative plugin families instead of every plugin for deep config round-trip:

• providers: openai, anthropic, mistral, openrouter
• channels: telegram, discord, slack, whatsapp
• memory: memory-lancedb
• feature/runtime: browser, acpx, tokenjuice

For each representative:

1. Write config through CLI when possible.
2. Read it back through config get or JSON.
3. Run plugins inspect.
4. Run doctor --non-interactive.
5. Trigger gateway config reload if applicable.
6. Compare config hash before/after no-op commands.