proof-video skill

Proof video

Use this developer skill for captioned Control UI recordings and native captures. Captions are burned into an MP4; they are visual narration, not synthesized audio. Keep the raw video as evidence, attach the polished MP4, and never commit media or .artifacts/ output. Inspect the entire capture for unrelated or private content before publishing under the PR's existing authorization.

Point out the claim

Choose marks that demonstrate the change. Use arrows to locate a control, outlines to identify a region, alignment guides to compare edges or centers, and spacing brackets to measure gaps. Measurements come from locator.boundingBox() after the asserted layout settles; the renderer calculates the labels. Keep the reference element first and use the same viewport, crop, scale, and edge before and after. Keep marks sparse and labels in empty space. Preserve the raw capture.

For screenshot markup or timed video marks, read annotations. Both paths use the same SVG geometry renderer. Annotations are composited before zoom, while narration stays fixed in the viewport. Re-measure and replace marks after scrolling or layout changes; a cue describes a stationary capture, not a live element tracker. Clear marks during movement when they would become stale.

Standalone recording

From the repository root, save this script as .artifacts/proof-video/record.mts (create the parent directory first), then run node --import tsx .artifacts/proof-video/record.mts. It starts an isolated mocked Control UI, records five captions, a real toolbar zoom, fast-forwarded startup, and an eight-second pause at 4× speed. Every invocation allocates a fresh evidence directory.

import { chromium } from "playwright";
import { createControlUiE2eArtifactDir } from "../../ui/src/test-helpers/control-ui-e2e-artifacts.ts";
import {
  installMockGateway,
  resolvePlaywrightChromiumExecutablePath,
  startControlUiE2eServer,
} from "../../ui/src/test-helpers/control-ui-e2e.ts";
import { startProofRecording } from "../../ui/src/test-helpers/proof-video.ts";

const dir = createControlUiE2eArtifactDir("captioned-toolbar", ".artifacts/proof-video");
const server = await startControlUiE2eServer(undefined, { source: true });
const browser = await chromium.launch({
  executablePath: resolvePlaywrightChromiumExecutablePath(chromium.executablePath()),
  args: ["--force-device-scale-factor=2"],
});
try {
  const recording = await startProofRecording(browser, {
    dir,
    contextOptions: { colorScheme: "dark", locale: "en-US", reducedMotion: "reduce" },
  });
  const { page } = recording;
  const sessions = ["Main", "Weekly planning", "Reading list"].map((label, index) => ({
    key: index === 0 ? "agent:main:main" : `agent:main:proof-${index}`,
    label,
    displayName: label,
    kind: "direct",
    status: "done",
    hasActiveRun: false,
    updatedAt: Date.parse("20

Use caption(text) to replace the current caption, clearCaption() to end it, zoom(locatorOrRect, { factor: 2, padding: 24, ease: 0.5 }) to start or replace a zoom, and zoomOut() to end it. Rectangles use viewport CSS pixels. Padding expands and clamps the target rectangle; its center determines the zoom focus. speed(factor) changes playback speed until the next call; speed(1) restores normal playback. finish() closes open cues at the finalized video's end, closes the recording context, rasterizes captions, writes cues.json, and returns { dir, videoPath, cuesPath }. The caller closes the browser and server.

The helper is test tooling at ui/src/test-helpers/proof-video.ts; production code must not import it. contextOptions carries ordinary Playwright context options, while the helper owns the viewport, recording size, and device scale. Keep narration to one or two short lines; oversized text fails instead of silently clipping. Caption PNGs are transparent viewport-width, 120-pixel-high strips with a centered dark rounded card and 30-pixel system text.

Render

Use the exact directory printed by the recording:

node --import tsx scripts/render-proof-video.mts --cues .artifacts/proof-video/<capture>/cues.json

The renderer prints the output path and duration. Optional --input overrides the raw input and --out chooses the output. Existing output files are refused, preserving raw evidence and earlier renders. It uses trusted system ffmpeg and ffprobe, never Playwright's bundled encoder. Install a full system build with brew install ffmpeg or apt install ffmpeg.

Speed segments split the raw timeline with trim/setpts/concat, then normalize it to the input frame rate. Caption windows and zoom transitions are remapped to output time. Zoom uses zoompan with d=1 and matching input FPS; a 100-frame, 4-second, 25-FPS synthetic capture retained exactly 100 frames and 4 seconds, with a visibly and numerically different zoom hold. Dynamic crop width/height expressions do not animate: ffmpeg evaluates them only at initialization. Output is viewport-sized H.264 MP4, yuv420p, CRF 23, with faststart and no audio.

Cue sidecar v1 and native captures

Save a hand-written cues.json beside an untouched native .mov or .webm. Use scale for actual raw pixels per viewport CSS pixel, not display metadata. Input dimensions must equal viewport × scale and frame rate must be constant. All times below are seconds in the RAW video. Caption image paths are relative to the sidecar directory. Zoom cues cannot overlap other zoom cues; speed cues cannot overlap other speed cues. Adjacent cues are allowed.

{
  "version": 1,
  "video": "capture.mov",
  "viewport": { "width": 1280, "height": 800 },
  "scale": 2,
  "captionHeight": 120,
  "cues": [
    {
      "kind": "caption",
      "start": 1,
      "end": 4,
      "text": "Inspect the session controls.",
      "image": "captions/0.png"
    },
    {
      "kind": "zoom",
      "start": 1,
      "end": 4,
      "rect": { "x": 20, "y": 150, "width": 280, "height": 80 },
      "factor": 2,
      "ease": 0.5
    },
    { "kind": "speed", "start": 5, "end": 9, "factor": 4 }
  ]
}

node --import tsx scripts/render-proof-video.mts --cues <dir>/cues.json --rasterize

--rasterize creates missing caption and annotation PNGs with fresh Playwright Chromium pages. The exported rasterizeProofOverlays(browser, cues, dir) also supports programmatic native-capture workflows. Neither path needs libass or drawtext: Chromium rasterizes text and geometry. ffmpeg overlays annotations before zoom and captions after zoom.

Verified capture mode and limitations

On macOS with Playwright 1.62.1 and Chromium 151.0.7922.34, context deviceScaleFactor: 2 produced a 2560×1600 WebM containing only 1280×800 page content plus gray padding. Launching with --force-device-scale-factor=2 and leaving context emulation at 1 recorded true 2× content: a 100-CSS-pixel marker measured 200×200 at raw coordinates (2360, 1400). The helper defaults to deviceScaleFactor: 2 for recording size and sidecar scale; the launch flag in the template is required. This parameter does not enable context emulation. For verified 1× capture, launch without the flag and pass deviceScaleFactor: 1. The renderer uses Lanczos pre-upscaling for 1× sources, but cannot restore lost detail: a 2× zoom on those recordings is softer.

The cue clock starts at context creation using performance.now(). Playwright's video clock can differ by roughly 100 ms, and startup/load can increase the skew; inspect action/caption alignment in the finalized recording. Avoid clipped, locator, or full-page screenshots during recording: Chromium can corrupt the screencast surface. Caption rasterization happens only after recording closes, in a fresh context. See the Control UI E2E skill's screenshot guidance.

GitHub's player ignores subtitle tracks, so captions must be burned in. The renderer requires crop, scale, overlay, setpts, trim, concat, fps, and zoompan in the system ffmpeg build. It does not use subtitles= or drawtext=. Variable frame-rate native captures need conversion to a separate constant-frame-rate working copy first; retain the original. Frame-rate quantization can shift speed-segment boundaries by about one output frame. Check output duration and extract caption, zoom-hold, and sped-segment frames before attaching the MP4.