osint-methodology skill

OSINT Methodology — External Red-Team Edition

0. When to use this skill / When NOT

Use this skill when:

• Planning or executing external reconnaissance against an authorized target (red team, bug bounty in-scope, ASM engagement).
• Mapping an organization's external attack surface end-to-end (subdomains → assets → exposure → attack paths).
• Investigating a person, entity, or threat actor where evidence discipline matters.
• Tracing cryptocurrency flows, geolocating media, performing image/video forensics, or chronolocating events.
• Building a structured OSINT campaign that needs reproducibility, severity grading, and clean handoffs.
• Producing client-facing deliverables (exec summaries, technical reports, reproduction packages) from offensive engagements.

Do NOT use this skill when:

• The user is asking for active exploitation, post-exploitation, lateral movement, AD privilege escalation, malware development, or anything beyond reconnaissance — those are out of scope.
• The user is asking for blue-team / defensive content (SIEM rules, detection engineering) — different domain.
• The target's authorization is unclear and the user is asking you to act against a third-party asset they don't own — see §1 below; gently surface the scope question before proceeding.

1. Authorization & Legal Posture

This skill is intended for assets the operator owns or has written authorization to assess (red-team rules of engagement, bug-bounty in-scope assets, ASM contracts).

Soft scope check: when a user asks you to act against a target whose authorization isn't established earlier in the conversation, ask once before proceeding:

"Quick scope check: is this a target you own or have written authorization to assess (e.g., a red-team engagement, in-scope bug-bounty asset, or your own infrastructure)? I want to make sure we stay on the right side of the engagement boundary."

Once authorization is asserted, proceed without re-asking. If the user explicitly states the engagement type (e.g., "this is for our pentest of acme.com under contract"), you don't need to ask again.

Always-on guardrails (regardless of authorization):

• Never weaken auth, rate limits, banners, or any safety control that enforces scope on the target side.
• Never run destructive probes (true SYN scans on production, masscan at line rate, fuzzing/brute-force) outside an explicit DEEP / --aggressive mode.
• Never paste real PII, valid credentials, session tokens, API keys, or other secrets into cloud-hosted LLMs or third-party services.
• Never take action against assets outside the documented scope, even if "obviously related" (subsidiaries, vendors, employees' personal accounts, etc.).

2. Confidence Levels

Every assertion you make during an engagement should carry a confidence level. Three levels:

Rule of three for attribution: require three independent weak signals, OR one strong + one weak, before asserting linkage. Don't single-source attribute.

2.1 Confidence Upgrade Workflows

Confidence isn't static — every TENTATIVE asset should have a documented path to FIRM and to CONFIRMED. Use these per-asset-type rules.

Default reporting posture: never claim CONFIRMED without explicit corroboration. When in doubt, downgrade. Operators trust under-claims more than over-claims.

3. Output Format Conventions

When you produce findings during an active session, structure each finding to match the schema below — it drops cleanly into asset-management tools.

Finding:
  id:           <stable hash or UUID>
  module:       <which technique discovered it; "manual" if hand-found>
  asset_key:    <typed key, e.g. sub:api.example.com or webapp:https://example.com/admin>
  category:     <e.g. SECRET_LEAK, MISSING_HSTS, OPEN_GRAPHQL_API, LEAKED_CRED, SSO_EXPOSURE>
  severity:     <info|low|medium|high|critical>
  confidence:   <tentative|firm|confirmed>
  title:        <one-line summary>
  description:  <2-5 sentences>
  evidence:
    url:        <where it was found>
    timestamp:  <UTC ISO8601>
    sha256:     <hash of any downloaded artifact>
    raw:        <truncated to 2 KiB>
  references:
    - <CVE-ID, advisory URL, vendor doc>
  remediation:  <action the asset owner can take>

Always use UTC timestamps. Local time creates correlation bugs across notes/screenshots/logs.

4. Source Hygiene & Citations

For every artifact you capture, record: URL + UTC timestamp + SHA-256 hash + tool version + runid**.

• Hash all downloaded files with SHA-256.
• Screenshot in PNG (lossless, smaller than full-page WARC for evidence packs).
• Capture raw HTTP requests/responses, capped at 2 KiB body to keep evidence packs small.
• Use JSONL (NDJSON) logs, one line per event, with a run_id so the entire engagement is replayable.
• Separate evidence read-only from working copies; never edit captured artifacts.

When citing a source in your output, prefer durable references (CVE, vendor advisory, ATT&CK technique ID, RFC) over ephemeral ones (a Twitter post, a forum thread). If the only source is ephemeral, archive it (archive.today, Wayback SavePageNow) before citing.

5. Do NOT (hard rules)

• DO NOT paste creds, session tokens, API keys, real PII, infostealer logs, or unique pivots into cloud LLMs (ChatGPT, Claude.ai, Gemini, Perplexity). Use local models (Ollama, LM Studio, GPT4All) for sensitive analysis.
• DO NOT assume vendor labels are ground truth. Cross-label sanity: TRM, Chainalysis, Arkham can disagree. Treat every label as a hypothesis.
• DO NOT assume 1:1 bridge flows. Bridges/mixers/wrappers introduce mint/burn semantics; validate with on-chain proofs.
• DO NOT assert ownership from a single signal. Favicon-hash overlap, shared CT issuer, shared NS — each is a hypothesis. Need rule-of-three.
• DO NOT run fuzzing, SYN scans, masscan, or nuclei fuzzing/* templates outside an explicit DEEP / --aggressive mode.
• DO NOT use a credential validator to do anything except read-only verification (no create/delete/send).
• DO NOT mirror-image (assume the target thinks like you do). Separate capability from intent and sponsorship.
• DO NOT confuse correlation with control.
• DO NOT escalate when you encounter active defenses; back off and document (see §6.4).

6. OpSec

6.1 Sock Puppets

A sock puppet is a fake account that cannot be linked to you. Build a posting history, age the account, use it from a separate browser profile.

Resources & techniques:

• Persona generation: Fake Name Generator, This Person Does Not Exist.
• Browser isolation: Firefox Multi-Account Containers, or dedicated profiles per persona.
• Disposable phone numbers: Burner, Silent Link (some platforms reject VoIP — keep a backlog of numbers).
• Hardware passkeys for any high-value persona; store recovery codes offline.
• Audit every browser extension before installation. Supply-chain attacks on popular extensions have repeatedly targeted investigators — assume the popular ones are at higher risk, not lower.
• Maintain chain-of-custody: timestamp every action, hash every key artifact, record tool versions per case.
• Personas should look like real low-engagement accounts: profile photo (synthetic), bio, a few low-effort posts spread across weeks before the persona is "used."

References:

• Effective Sock Puppets
• Ultimate Guide to Sock Puppets

6.2 Detectability & OpSec Tagging

Every probe leaves a footprint. Tag every operation in your notes with a detectability level so you can reason about the SIEM trail you're leaving on the target's side.

When working with a client, document the operations actually run and their detectability tag in the engagement report — clients appreciate knowing what their detection stack should have caught.

Defaults: passive by default. Active probes only when (a) explicitly authorized, (b) within agreed maintenance windows, and (c) with the operator's awareness of the resulting log volume.

6.3 Validator Discipline

When you discover a credential in the wild (a leaked API key, a sourcemap-exposed token, a hard-coded PMAK in a public Postman workspace), you may want to confirm it's live. Do this with read-only validators only.

Discipline:

• Read-only endpoint only (e.g., /me, /whoami, auth.test, sts:GetCallerIdentity).
• Never use the validated credential to create, modify, delete, or send anything.
• Tag the validation attempt with detectability — every validator generates an audit-log entry on the provider side.
• Record checked_at (UTC), the response (truncated), and the scope/account-ID returned.
• If the operator's rules of engagement forbid validation, mark the credential validationskippedby_policy and stop.

Concrete validator endpoints (Postman, AWS, GitHub, Slack, Anthropic, OpenAI, npm, Atlassian, DataDog) live in the companion offensive-osint skill.

6.4 Detection-Aware Probing (signs of detection + back-off)

Your probes will eventually hit detection. Recognize the signs and back off before you trip an active response.

Signs you've been detected (in roughly increasing severity):

1. Rate-limit responses — 429 Too Many Requests, Retry-After header set, X-RateLimit-Remaining: 0.
2. Captcha interstitials — Cloudflare interstitial page, hCaptcha challenge, AWS WAF page.
3. WAF page — explicit "Access denied" with provider branding (Cloudflare, Akamai, Imperva, F5 ASM, AWS WAF, Sucuri).
4. Status code drift — endpoints that previously returned 200/401 now return 403 only from your IP.
5. Banner change — server header shape or response timing changes consistently.
6. DNS poisoning back to NXDOMAIN — target's authoritative servers stop resolving subdomains (probably their CDN took over).
7. Honeypot bait — endpoints that look too good (/admin/db_dump.sql, exposed .env with credentials that don't validate). Real exposures rarely look this clean.
8. Direct contact — your sock-puppet email gets a "we noticed unusual activity" message; or, in extreme cases, your IP gets a courtesy abuse-contact email.

Back-off ladder:

1. Slow down. Halve your concurrency. Add 2–10s jitter between requests.
2. Switch endpoints. Stop hitting the path that triggered. Move to a different module of the recon pipeline.
3. Switch persona. New User-Agent (rotate among realistic browsers), new TLS fingerprint (different httpx/curl version).
4. Switch IP. Rotate to a new egress (residential proxy, Tor for sensitive lookups, a different cloud region).
5. Pause. Wait 1–24 hours. Many WAFs have rolling-window IP-based reputation; passive time often resets it.
6. Document and consult. If you've hit (3) WAF, (4) status drift, or (8) direct contact, stop active probing and consult the engagement lead. Continued probing past these signals risks scope violation.

Persona / IP rotation rules:

• Never rotate persona to one that's been used in a prior engagement against the same target.
• Use residential proxies (Bright Data, Smartproxy, IPRoyal) for high-detectability work — but be aware they're sometimes IP-blocklisted by Cloudflare.
• Tor exit nodes are useful for passive lookups (CT logs, archive sites) but are blocked by most active-probe targets.
• Cloud egress IPs (AWS / GCP / Azure) are often blocklisted aggressively for recon. Use sparingly.
• Document every rotation with timestamp + reason; reviewers will ask.

Don't:

• Don't try to "outsmart" a confirmed WAF block by sending more aggressive payloads. That's how clients get extra logs and how you get caught.
• Don't switch source IPs to evade an explicit block-list — that crosses into evasion territory and may breach the rules of engagement.
• Don't ignore signals because the dashboard says "still up." The probe is being silently logged; the response will come later.

7. External Red-Team Recon Pipeline

A 5-stage pipeline for any authorized external assessment. Stages are sequential; modules within a stage can run concurrently.

Stage 1 — Seed Discovery

Establish the ground truth of who/what the target is.

• WHOIS on the seed domain (registrant, dates, name servers).
• ASN enumeration: which AS does the org own/use? (Hurricane Electric BGP Toolkit, RIPEstat, BGPView.)
• DNS records (A/AAAA/MX/TXT/NS/SOA/CAA) — records-only, no walking yet.
• Certificate Transparency history for the root domain (crt.sh, Censys).