analysis-planner skill

Analysis Planner

Structure investigations before diving into data to prevent wasted time, ensure thoroughness, and deliver actionable insights.

Core Planning Framework

Phase 1: Define the Question

Start with: "What decision needs to be made?"

Quality checklist:

• [ ] Specific (not vague)
• [ ] Answerable with available data
• [ ] Tied to a decision or action
• [ ] Has clear success criteria
• [ ] Time-bounded

Good vs Bad:

• ✓ "Should we prioritize mobile app performance or new features next quarter?"
• ✓ "Which customer segment should we focus retention efforts on?"
• ✗ "Tell me about our users" (too broad)
• ✗ "Find something interesting" (no direction)

Phase 2: Define Success Criteria

Ask: "What would a good answer look like?"

Template:

A successful analysis will:
1. [Specific outcome]
2. [Specific outcome]

We'll know we're done when:
- [Criterion]
- [Criterion]

The answer will enable us to:
- [Decision/action that will be taken]

Phase 3: Generate Hypotheses

Process:

1. List potential drivers from KPI Tree
2. Add business context (recent changes, events)
3. Combine into testable hypotheses

Hypothesis quality criteria:

• [ ] Testable with available data
• [ ] Specific (not "something changed")
• [ ] Has clear validation method
• [ ] Mutually exclusive from others
• [ ] Collectively exhaustive

Prioritize by:

1. Probability: How likely?
2. Impact: How much does it explain?
3. Actionability: Can we do something about it?

Phase 4: Design Analysis Approach

For each hypothesis, provide detailed analysis steps in descriptive language (NOT SQL):

For each hypothesis:

1. Data to Collect - Describe in words what data points are needed (e.g., "number of sign-ups by day, with their marketing channel, time from sign-up to first action")
2. How to Analyze - Detail the visualization approach:

• Type of chart/table (line chart, bar chart, histogram, table, etc.)
• What data goes on which axis
• What lines/bars/segments to plot
• Any comparisons to show (before/after, segment A vs B)

1. What to Look For - Describe the patterns that would validate or invalidate the hypothesis:

• Changes in trend lines
• Differences in values between segments
• Timing of changes
• Magnitude of differences

1. Proposed Deep-Dives - Suggest follow-up analyses if hypothesis is validated

Critical: Never include SQL queries or code. Always describe analysis in plain language that any analyst can translate to their own tools.

Phase 5: Structure Output

Organize the analysis plan with:

• Executive Summary (problem, question, decision, timeline)
• Success Criteria (what done looks like)
• Context (metric definition, KPI tree, recent changes)
• Hypotheses (prioritized tiers with scores)
• Analysis Approach for each hypothesis

Do NOT include:

• Analysis Roadmap & Timeline section
• Data Requirements section with SQL
• Roles & Responsibilities section
• Technical implementation details

Analysis Types

Type 1: Root Cause Investigation

When: KPI changed, need to know why Timeline: 1-3 days Key techniques: KPI tree navigation, segmentation, timeline correlation Agents needed: 🔍 Root Cause Investigator, 👥 Segmentation Expert (to identify which customer groups drove the change)

When to use Segmentation Expert:

• KPI changed but don't know which customers drove it
• Need to understand "who" behind the "what"
• Investigating whether it's a mix effect or performance effect

Type 2: Opportunity Sizing

When: Evaluating potential initiative Timeline: 2-5 days Key techniques: Market sizing, segment analysis, conversion math Agents needed: 👥 Segmentation Expert (to identify target segments and their potential), 📊 Chart Advisor

When to use Segmentation Expert:

• Sizing opportunity by customer segment
• Understanding which segments would benefit most
• Estimating addressable market by segment

Type 3: Segment Deep Dive

When: Understanding specific customer group Timeline: 3-7 days Key techniques: Behavioral analysis, cohort tracking, feedback synthesis Agents needed: 👥 Segmentation Expert (primary), 📅 Cohort Analyst (to see how segment evolves), 💬 Feedback Synthesizer

When to use both Segmentation and Cohort Analysis:

• Understanding how different segments perform over time
• Comparing retention across customer segments
• Identifying which segments have best/worst long-term value

Type 4: Feature Performance Review

When: Evaluate feature success Timeline: 2-4 days Key techniques: Adoption funnel, user feedback, cohort comparison Agents needed: 📅 Cohort Analyst (to compare cohorts pre/post launch), 🔬 Influential Factors Detective

When to use Cohort Analysis Specialist:

• Comparing user cohorts before and after feature launch
• Tracking feature adoption over time by cohort
• Understanding if newer users adopt feature faster

Type 5: Experiment Analysis

When: Evaluate A/B test Timeline: 1-2 days Key techniques: Statistical testing, segment analysis, guardrail checks Agents needed: 🧪 A/B Testing Advisor, 👥 Segmentation Expert (to understand if effect varies by segment)

When to use Segmentation Expert:

• Checking if experiment effect consistent across segments
• Identifying which segments benefit most from change
• Understanding heterogeneous treatment effects

Type 6: Retention Analysis

When: Understanding why customers stay or leave Timeline: 3-5 days Key techniques: Cohort retention tables, segment comparison, leading indicators Agents needed: 📅 Cohort Analysis Specialist (primary), 👥 Segmentation Expert (to compare segments)

When to use Cohort Analysis Specialist:

• Measuring retention rates over time
• Comparing new vs old user behavior
• Identifying early warning signals in recent cohorts
• Calculating lifetime value by cohort
• Understanding maturation patterns

Analysis Approach Format