kill-argument skill

Kill Argument Exercise: Adversarial Attack-Defense Review

Codex assurance: both fresh agents are OpenAI-family, so the base JSON

records review_independence: same-family and

acceptance_status: provisional. The mechanical count mapping may drive the

next step, but it is not cross-family acceptance. Call failure emits ERROR.

Stress-test the headline claims of a paper against the strongest possible rejection argument: $ARGUMENTS

Why This Exists

Standard score-based reviews (/research-review, /auto-paper-improvement-loop) tend to produce balanced weakness lists. Each weakness gets ~equal attention, ranked CRITICAL > MAJOR > MINOR. Empirically, this misses one specific failure mode: the single most damaging argument a reviewer would write in a rejection paragraph — the one sentence that, if a senior area chair reads it, kills the paper.

A balanced reviewer might list "scope-overclaim risk" as MAJOR alongside 3-5 other MAJORs, never quite committing. An adversarial reviewer must commit: their entire job is to convince the area chair to reject in 200 words.

This skill runs that adversarial pass deliberately, then forces a second fresh reviewer to defend point-by-point, classify each rejection as already-fixed / partially-fixed / still-unresolved, and surface what's actually load-bearing.

Empirical motivation: in a real submission run, after several rounds of standard improvement (score 7-8/10), the kill-argument exercise surfaced framing weaknesses that no prior review caught (e.g., a setting being mostly conditional rather than truly general, or a baseline being irrelevant to real systems). Author rebuttal forced explicit scope qualifications in abstract and discussion that weren't visible from the score-based reviews alone.

How This Differs From Other Review Skills

This skill is complementary, not a replacement. Run after standard reviews when you want to know what the worst-case reviewer paragraph would look like, before camera-ready or rebuttal preparation.

When To Use

• After 1-2 rounds of /auto-paper-improvement-loop settled at a stable score, but before submission. Surfaces what additional fixes would close the headline-attack gap.
• During rebuttal preparation, to predict reviewer-2's strongest objection so you can prepare the response in advance.
• For theory papers with a high-level title that may oversimplify the actual theorem (the most common reject-attack pattern).
• For papers where a reviewer might attack scope, assumption-vs-claim mismatch, missing proof obligations, or evidence-vs-headline gaps.

This skill is most valuable for theory papers with ≥5 theorem-class environments (so the headline depends on real proof obligations). For empirical papers without theorems, use /research-review instead.

Constants

• REVIEWERMODEL** = gpt-6-astra (default; specify gpt-5.4 if you want to fall back to the legacy default). Reviewer reasoning effort = ultra for the deep-audit core threads (capability fallback never below xhigh).
• CONTEXTPOLICY = fresh (REVIEWERBIASGUARD). Each thread is a fresh spawnagent call. Never use send_input. No prior review summary, fix list, or executor explanation enters either prompt.
• ATTACKLENGTH** = approximately 200 words (do not exceed 250). Single coherent argument, not a list.
• DEFENSEDECOMPOSITION** = 3-7 atomic rejection points extracted from the attack memo. Each gets its own classification.
• CLASSIFICATION = answeredbycurrenttext / partiallyanswered / still_unresolved. (Names chosen so the adjudicator does not assume "fixed" implies prior history of patching — they read the paper as a fresh reviewer would.)
• OUTPUT = KILLARGUMENT.md (human-readable) + KILLARGUMENT.json (machine-readable) in the paper directory.
• RENDERHTML = true — When true (default), auto-render KILLARGUMENT.md to HTML after writing the report via /render-html "/KILLARGUMENT.md" --json "/KILLARGUMENT.json". Uses full review gate (audit-class artifact). Set false to skip, or pass — render html: false. Non-blocking: failures don't invalidate the kill-argument verdict.

Workflow

The attack and adjudication calls are fresh, read-only Codex shards. They return structured per-point records with stable dedupkey identifiers; neither call writes the paper or emits cross-family acceptance. The parent computes the top-level mapping mechanically and records a same-family provisional verdict. If spawnagent is unavailable, use fresh sequential contexts where supported; otherwise emit BLOCKED rather than inventing a verdict. See fan-out-pattern.md.

Step 1: Discover paper files

Locate the paper directory and inventory the source.

PAPER_DIR="$ARGUMENTS"   # e.g., paper-overleaf/ or paper/
cd "$PAPER_DIR"

# Find the LaTeX entry point
ENTRY=$(grep -lE '^\\documentclass' *.tex 2>/dev/null | head -1)
echo "Entry: $ENTRY"

# Find all source files codex should read
find . -name "*.tex" -not -path "./.git/*" 2>/dev/null
find . -name "*.bib" -not -path "./.git/*" 2>/dev/null
find figures/ -name "*.pdf" -o -name "*.png" 2>/dev/null
ls -la *.pdf 2>/dev/null  # compiled PDF

If a compiled PDF is missing, the skill should still run on .tex source alone, but the prompt should mention this so the reviewer doesn't waste cycles trying to extract from a non-existent PDF.

Step 2: Attack memo (Thread 1, fresh codex)

Invoke spawnagent (NOT sendinput) with the following prompt structure. Use absolute or paper-directory-relative paths inside the prompt; do not rely on a cwd parameter.

spawn_agent:
  model: gpt-6-astra
  reasoning_effort: ultra
  message: |
    You are simulating a hostile NeurIPS / ICLR / ICML reviewer for a paper.
    This is a kill-argument adversarial check — your task is NOT to give a
    balanced review but to construct the **single strongest argument for
    rejecting this paper**.

    ## Files to read
    - LaTeX entry: <ENTRY>
    - All section files under sections/ or wherever they live
    - Macro files (math_commands.tex, etc.)
    - Compiled PDF: <main.pdf> (if available)

    Read the source carefully. Do not consult any prior reviews, fix lists,
    or summaries; this must be a fresh, zero-context adversarial pass.

    ## Your task
    Construct the single best argument to reject this paper in approximately
    200 words. Your goal is to write the worst-case rejection memo a senior
    NeurIPS area chair would produce after reading the paper.

    Focus on these axes (pick the most damaging combination, do not list all):
    1. Theorem validity: are central theorems actually proved as stated?
    2. Assumption-vs-claim mismatch: does the body silently retreat to a
       narrower object than the title/abstract advertise?
    3. M

Save the returned agent_id for the trace; do NOT pass it to Thread 2. Save the attack memo verbatim — both Thread 2 and the human-readable report use it.

Step 3: Adjudication memo (Thread 2, fresh codex with attack + paper)

Invoke a second spawnagent call (still NOT sendinput — Thread 2 is independent of Thread 1's Codex agent history):

spawn_agent:
  model: gpt-6-astra
  reasoning_effort: ultra
  message: |
    You are an independent area-chair adjudicator examining whether the
    current paper text answers a hostile reviewer's rejection memo.
    You are NOT the paper's defender — your job is to read the attack
    point-by-point and rule, from the current source files alone,
    whether each point stands or falls. Fresh, zero-context adjudication;
    do not reference any prior reviews / fix lists.

    ## Paper files
    [list paths same as Step 2]

    ## The hostile reviewer's rejection memo (the "attack")
    > <attack memo verbatim from Thread 1>

    ## Your task
    The attack is one continuous argument, but it makes multiple distinct
    rejection points that you must adjudicate separately. Decompose the
    attack into its atomic rejection points (3-7 of them), then for each
    point classify it:

    - answered_by_current_text: the current paper source already mitigates
      this point (cite specific file:line evidence)
    - partially_answered: paper has some response but not enough to refute
      the attack as written
    - still_unresolved: paper has no effective response

    The label `answer

Save the returned agent_id.

Step 4: Write KILLARGUMENT.md and KILLARGUMENT.json

Compose the human-readable report /KILL_ARGUMENT.md:

# Kill Argument Report — <paper title>

**Date**: <YYYY-MM-DD>
**Reviewer model**: gpt-6-astra ultra, fresh agents (no send_input)
**Attack agent**: <agent_id 1>
**Adjudicator agent**: <agent_id 2>
**Verdict**: <PASS / WARN / FAIL / NOT_APPLICABLE / BLOCKED / ERROR> (`reason_code: <...>`)

## Net assessment

<paragraph from adjudicator memo's "Net assessment">

## Attack memo (verbatim)

> <attack memo from Thread 1>

## Adjudication (per-point)

<copy verbatim from Thread 2 — uses labels answered_by_current_text / partially_answered / still_unresolved>

## Top action items

<copy from Thread 2>

## Recommendation

If P_4 (or whatever still_unresolved critical) is research-level, record
it as a known open problem in the conclusion / limitations. If it is
writing-level, queue for next /auto-paper-improvement-loop round.

Compose the machine-readable /KILL_ARGUMENT.json per the ARIS Audit Artifact Schema (shared-references/assurance-contract.md):

{
  "audit_skill": "kill-argument",
  "verdict": "PASS | WARN | FAIL | NOT_APPLICABLE | BLOCKED | ERROR",
  "reason_code": "<see verdict mapping below>",
  "summary": "<one-line summary, ~80 chars>",
  "audited_input_hashes": {
    "main.tex":                          "sha256:<...>",
    "sec/0.abstract.tex":                "sha256:<...>",
    "sec/<each-section>.tex":            "sha256:<...>",
    "references.bib":                    "sha256:<...>",
    "main.pdf":                          "sha256:<...>"
  },
  "trace_path": ".aris/traces/kill-argument/<date>_run<NN>/",
  "agent_id": "<defense agent_id — primary; attack agent_id in details>",
  "executor_model": "codex-gpt-6-astra",
  "executor_family": "openai",
  "reviewer_model": "gpt-6-astra",
  "reviewer_family": "openai",
  "review_independence": "same-family",
  "acceptance_status": "provisional",
  "reviewer_reasoning": "ultra",
  "generated_at": "<UTC ISO-8601>",
  "details": {
    "attack_agent_id": "<agent_id 1>",
    "defense_agent_id": "<agent_id 2 — same as top-level agent_id>",
    "attack_memo": "<verbatim>",
    "decomposed_points": [
      {
        "id": "P_1",
        "label": "<short label>",
        "attack_

Hash inputs (auditedinputhashes): use paper-relative paths, sha256 of every .tex consumed plus references.bib and the compiled main.pdf if it exists. The verifier rehashes these on verifypaperaudits.sh and flags STALE if the user edited the paper after running the audit.

Verdict mapping (every (counts, severity) tuple must hit exactly one row):

PASS requires stillunresolved == 0. With stillunresolved == 0, any partially_answered at major or higher makes the best available verdict WARN — never PASS.

The verdict is computed from the per-point counts; do NOT let the defense thread output the top-level verdict directly (that would let it self-grade). The skill code does the verdict mapping.

Step 5: Print summary

To the user:

🗡  Kill Argument complete.

  Attack: <one-sentence summary of the rejection thrust>

  Adjudication breakdown:
    answered_by_current_text:   X
    partially_answered:         Y
    still_unresolved:           Z   ← critical: <names>

  Verdict: <PASS / WARN / FAIL / NOT_APPLICABLE / BLOCKED / ERROR>
  Reason:  <reason_code, e.g., defense_survives, unresolved_critical>

  Top action items:
  1. ...
  2. ...
  3. ...

  Full report: <paper-dir>/KILL_ARGUMENT.md

Output Contract

• /KILL_ARGUMENT.md — human-readable report
• /KILL_ARGUMENT.json — machine-readable ledger
• .aris/traces/kill-argument/_runNN/ — per-thread codex traces (Attack memo + Adjudication memo)
• Optional: applied fixes if user explicitly requests; default is detect-only, do not auto-modify.
• /KILLARGUMENT.html (when RENDERHTML = true, default) — single-file HTML view auto-rendered via /render-html with the JSON sidecar. Full review gate applies. Non-blocking: if /render-html fails, the kill-argument verdict still counts as complete.

Key Rules

• Fresh agent per call. Both Attack and Adjudication use spawnagent, never sendinput. Thread 1 and Thread 2 must not share Codex context.
• Zero prior context. Neither thread receives prior round reviews, fix lists, executor summaries, or improvement-loop logs.
• Attack must commit. Single argument, ~200 words. No "consider also" hedge. The whole value is in forcing the reviewer to pick the most damaging line.
• Adjudicator must classify, not minimize. stillunresolved is honest if the paper has no effective response. Don't downgrade to partiallyanswered unless evidence is real.
• Author-chosen positions (e.g., deliberate title scope, deliberate omission of qualifier): mark partiallyanswered with note that the position is intentional, AND say whether the position is sustainable under the attack. Don't auto-grade as answeredbycurrenttext just because it's intentional.
• Verdict is computed by the skill, not by the adjudicator. The Codex thread emits per-point classifications; the skill code maps those to one of the 6 audit verdicts via the table in Step 4. Never let the adjudicator self-grade the top-level verdict.
• Detect-only by direct invocation; can be invoked by /auto-paper-improvement-loop Step 5.5 which then merges unresolved findings into its fix list. When a user runs /kill-argument paper/ directly, the output is informational and the human decides whether to act. When the skill is invoked from inside the auto-improvement loop, the loop reads KILLARGUMENT.json, deduplicates against its existing weakness list, and feeds novel stillunresolved points into Step 6 fixes — /kill-argument