citation-audit skill

Citation Audit

Codex assurance: base audit artifacts record

reviewindependence: same-family and acceptancestatus: provisional.

Cross-family overlays or deterministic metadata checks may record accepted;

unavailable semantic review emits BLOCKED.

Verify every \cite{...} in a paper against three independent layers:

1. Existence — the cited paper actually exists at the claimed arXiv ID / DOI / venue.
2. Metadata correctness — author names, year, venue, and title match canonical sources (DBLP, arXiv, ACL Anthology, Nature, OpenReview, etc.).
3. Context appropriateness — the cited paper actually supports the claim it is being used to support in the manuscript.

This skill is the fourth layer of \aris{}'s evidence-and-claim assurance, complementing experiment-audit (code), result-to-claim (science verdict), and paper-claim-audit (numerical claims). Together they form a bottom-up integrity stack from raw evaluation code to manuscript bibliography.

When to Use This Skill

Run before submission. The right gating point is:

• After paper-write has produced the LaTeX draft and bib file
• After paper-claim-audit has verified numerical claims
• Before final paper-compile for submission

Do not run this on a half-written draft — most of the work is in cross-checking each \cite against context, which is wasted on placeholder text.

What This Skill Catches

The dangerous citation problems are not wildly fake citations — those are easy to spot. The dangerous ones are:

• Wrong-context citations: real paper, but the cited claim is not what that paper actually establishes (e.g., citing Self-Refine to support "self-feedback produces correlated errors" — Self-Refine actually argues the opposite).
• Author hallucinations: anonymous-author placeholders that slipped through, missing co-authors, wrong order.
• Title drift: arXiv v1 vs v3 with different titles silently merged.
• Venue confusion: arXiv preprint cited but the official venue is now CVPR/ICML/NeurIPS — using the wrong record.
• Year mismatch: arXiv 2023 preprint with 2024 conference acceptance, year reported inconsistently.
• Phantom DOIs: DOI looks real but does not resolve.
• Self-citation drift: your own prior work cited with year off by one.

Constants

• REVIEWERMODEL = gpt-6-astra** — Fresh Codex reviewer with web access; same-family provisional in the base mirror.
• CONTEXTPOLICY = fresh — Each audit run uses a new reviewer thread (REVIEWERBIASGUARD). Continue only with sendinput when explicitly resuming the same audit.
• WEBSEARCH = required** — The reviewer must perform real web/DBLP/arXiv lookups, not pattern-match from memory.
• OUTPUT = CITATIONAUDIT.md** — Human-readable per-entry verdict report.
• STATE = CITATIONAUDIT.json** — Machine-readable verdict ledger consumable by downstream tools.
• SOFTONLY = false — When true (set via — soft-only / — softonly flag), the audit runs all three layers normally but forbids any .bib file mutation. Findings that would otherwise mutate the bib (FIX / REPLACE / REMOVE) are translated into per-occurrence sentence-rewrite proposals against the citing *.tex files. Used by /resubmit-pipeline Phase 1 to honor the user's hard "freeze the bib" constraint.
• RENDERHTML = true — When true (default), auto-render CITATIONAUDIT.md to HTML after writing the report. Uses full review gate (audit-class artifact). Set false to skip, or pass — render html: false. Non-blocking: failures don't invalidate the audit verdict.

Workflow

Step 1: Discover bib file and section files

Locate:

• references.bib (or paper.bib / similar) under the paper directory
• All *.tex files containing \cite{...} calls (typically sec/ or sections/)

If multiple bib files exist, audit each separately.

Step 2: Extract all (cite-key, context) pairs

For each \cite{key1,key2,...} invocation in the paper:

• Record the cite key
• Record the file + line number
• Record the surrounding sentence (≥ 1 full sentence around the cite, for context check)

Output a flat list of (key, file, line, surrounding_sentence) tuples.

Also build the inverse: for each bib entry, the list of all places it is cited.

Define two protocol sets used throughout the rest of the workflow: citedkeys is the set of unique cite keys appearing in any \cite{...} invocation across the audited .tex files (de-duplicated), and bibkeys is the set of keys parsed from the audited bib file(s). citedkeys drives Step 3 (audit only cited entries); bibkeys \ citedkeys is the uncited residual surfaced by the --uncited opt-in.

If the user passed --uncited, also compute the set difference bibkeys \ citedkeys here and stash it for use in Steps 5 and the JSON aggregation; see "Uncited Entry Detection (opt-in)" below for the protocol. The set-diff is a string operation only and does not consume reviewer budget.

Save the extracted contexts to paper/.aris/citation-audit/contexts.txt so the reviewer can read it directly. Use the paper-dir-relative path .aris/citation-audit/contexts.txt when recording the file in auditedinputhashes; do not stage under /tmp or other transient locations that the verifier cannot rehash later.

Step 3: Send each entry to a fresh reviewer (same-family provisional by default)

For each cited bib entry — i.e., each key in citedkeys with at least one extracted citation context — launch a fresh Codex reviewer agent. Do not reuse the same reviewer across entries. Do not spawn an agent for entries in bibkeys \ cited_keys; those are detect-only and surface only when --uncited is explicitly enabled (see "Uncited Entry Detection" below).

spawn_agent:
  model: gpt-6-astra
  reasoning_effort: xhigh
  message: |
    You are auditing a bibliographic entry. Use web/DBLP/arXiv search.

    ## Bib entry
    @article{key2024example,
      author = {...}, title = {...}, journal = {...}, year = {...}, ...
    }

    ## Where this entry is cited in the paper
    [paste extracted contexts]

    For this entry, verify:
    1. EXISTENCE: does this paper exist at the claimed arXiv ID / DOI / venue?
       Output: YES / NO / UNCERTAIN, with the verifying URL.
    2. METADATA: are author names, year, venue, title correct?
       For each, output: correct / wrong: should be ... / typo: ...
    3. CONTEXT: for each use, does the cited paper actually support the surrounding claim?
       Output per-use: SUPPORTS / WEAK / WRONG, with one-sentence reasoning.

    VERDICT: KEEP / FIX / REPLACE / REMOVE
    - KEEP: entry is clean, all uses are appropriate
    - FIX: metadata needs correction; uses are appropriate
    - REPLACE: cite is wrong-context, find a different paper that actually supports the claim
    - REMOVE: entry is hallucinated or unsupportable

    Be honest. If you cannot verify online, say UNCERTAIN; do not guess.

Save the response to .aris/traces/citation-audit/_runNN/.md per the review-tracing protocol.

Step 4: Aggregate verdicts

Build CITATIONAUDIT.json following the schema defined in "Submission Artifact Emission" below (single authoritative schema for this file). Per-entry ledger data goes under details.perentry, not under a top-level entries field. The top-level verdict is a single overall value (PASS / WARN / FAIL / NOT_APPLICABLE / BLOCKED / ERROR) derived from per-entry verdicts per the decision table in "Submission Artifact Emission"; the top-level summary is a one-line human-readable string.

Concretely, details carries the per-entry ledger:

"details": {
  "total_entries": 29,
  "counts": { "KEEP": 11, "FIX": 14, "REPLACE": 3, "REMOVE": 1 },
  "per_entry": [
    {
      "key": "lu2024aiscientist",
      "verdict": "KEEP",
      "axis_failures": [],
      "uses": [
        {"file": "sections/1.intro.tex", "line": 11, "verdict": "SUPPORTS"},
        {"file": "sections/6.related.tex", "line": 8, "verdict": "SUPPORTS"}
      ]
    },
    {
      "key": "madaan2023selfrefine",
      "verdict": "FIX",
      "axis_failures": ["CONTEXT"],
      "uses": [
        {"file": "sections/2.overview.tex", "line": 42, "verdict": "WRONG",
         "note": "Self-Refine demonstrates iterative improvement, not correlated errors"},
        {"file": "sections/6.related.tex", "line": 13, "verdict": "SUPPORTS"}
      ]
    }
  ]
}

See "Submission Artifact Emission" for the full artifact (top-level fields auditskill, verdict, reasoncode, summary, auditedinputhashes, tracepath, threadid, reviewermodel, reviewerreasoning, generated_at, details).

Step 5: Generate human-readable report

Write CITATION_AUDIT.md:

# Citation Audit Report

**Date**: 2026-04-19
**Bib file(s)**: references.bib
**Total entries**: 29

## Summary
| Verdict | Count |
|---------|------|
| KEEP    | 11   |
| FIX     | 14   |
| REPLACE | 3    |
| REMOVE  | 1    |

## Priority Fixes (CRITICAL — apply before submission)

### REMOVE: anon2025placeholder
- Author listed as "Anonymous" — canonical record exists with real authors and full title
- Title is incomplete
- ACTION: Replace key with the canonical citekey, update authors and title

### REPLACE-CONTEXT: example2023priorwork in sec/2.overview.tex:42
- Cited to support a specific technical claim
- The cited paper actually demonstrates a different (related but distinct) phenomenon
- ACTION: Rewrite the sentence; cite the prior work for what it actually establishes

[... continues for each entry ...]

## All-Clean Entries (no action needed)

[list of KEEP keys]

When --uncited is set, append the following section after "All-Clean Entries":

## Uncited Entries (opt-in)

The following bib entries are present in the audited bib file(s) but are not referenced by any `\cite{...}` in the paper body:

- `author2010example` — suggestion: prune (uncited; no local evidence of intent)
- `someone2015othercite` — suggestion: prune (uncited; no local evidence of intent)
- `third2024todo` — suggestion: check (a `% TODO: cite third2024todo` comment was found in `sections/3.related.tex`)

This section is detect-only; it does not change the top-level verdict.

Step 6: Apply fixes (interactive)

For each FIX/REPLACE/REMOVE verdict, prompt the user:

Fix [key]?
  Change: <description of change>
  Files affected: references.bib + sec/X.tex:Y
[Apply / Skip / Defer]

If AUTO_APPLY = true, apply all FIX-level changes (metadata corrections only). REPLACE and REMOVE always require human approval — they involve content changes.

Step 7: Recompile and verify

latexmk -C && latexmk -pdf -interaction=nonstopmode main.tex

Confirm:

• No new Citation undefined warnings
• No Reference undefined warnings
• Page count unchanged or only minimally affected by metadata fixes

Uncited Entry Detection (opt-in)

Default: disabled. Existing users see no behavior change — only \cite{...} keys are audited, and bib entries with no \cite reference in the manuscript are silently ignored.

Opt-in: pass --uncited on invocation. The skill then performs a set-diff after Step 2 and reports bib entries that appear in any audited bib file(s) but are not cited anywhere in the paper. Detect-only — uncited entries are not sent to the reviewer agent, so there is no extra reviewer/web-lookup cost.

Why opt-in

This skill's headline output is the three-axis audit on cited entries. Surfacing uncited bib entries by default would (a) change long-form output for every existing run, and (b) noise up the verdict for users who intentionally maintain a superset bib file (e.g., shared lab bib, in-progress section reorder where the cite has been removed but the entry intentionally retained). The flag preserves zero behavior change for existing callers.

Effect when enabled

When --uncited is set:

• CITATION_AUDIT.md gains a ## Uncited Entries (opt-in) section listing the keys with a one-line suggestion each: prune (entry is dead weight; recommend deleting) or check (entry might be intentional; flag for user review). Default suggestion is prune; only emit check when there is concrete local evidence (e.g., a TODO comment in a .tex file mentioning the key, or a recently removed \cite visible in git diff). Do not infer intent from the bib key string alone.
• CITATIONAUDIT.json details gains an uncitedentries array; see "Submission Artifact Emission" below for the schema.
• The top-level verdict is unchanged: uncited entries do not upgrade or downgrade the PASS / WARN / FAIL / etc. classification. The reasoncode and summary are likewise unchanged in shape; only the details.uncitedentries field appears.
• Verifier gates and downstream skills (paper-writing Phase 6, verifypaperaudits.sh) MUST NOT treat the presence of uncited_entries as a blocking signal.