deeptools skill

deepTools: NGS Data Analysis Toolkit

Overview

deepTools is a comprehensive suite of Python command-line tools designed for processing and analyzing high-throughput sequencing data. Use deepTools to perform quality control, normalize data, compare samples, and generate publication-quality visualizations for ChIP-seq, RNA-seq, ATAC-seq, MNase-seq, and other NGS experiments.

Core capabilities:

• Convert BAM alignments to normalized coverage tracks (bigWig/bedGraph)
• Quality control assessment (fingerprint, correlation, coverage)
• Sample comparison and correlation analysis
• Heatmap and profile plot generation around genomic features
• Enrichment analysis and peak region visualization

When to Use This Skill

This skill should be used when:

• File conversion: "Convert BAM to bigWig", "generate coverage tracks", "normalize ChIP-seq data"
• Quality control: "check ChIP quality", "compare replicates", "assess sequencing depth", "QC analysis"
• Visualization: "create heatmap around TSS", "plot ChIP signal", "visualize enrichment", "generate profile plot"
• Sample comparison: "compare treatment vs control", "correlate samples", "PCA analysis"
• Analysis workflows: "analyze ChIP-seq data", "RNA-seq coverage", "ATAC-seq analysis", "complete workflow"
• Working with specific file types: BAM files, bigWig files, BED region files in genomics context

Quick Start

For users new to deepTools, start with file validation and common workflows:

1. Validate Input Files

Before running any analysis, validate BAM, bigWig, and BED files using the validation script:

python scripts/validate_files.py --bam sample1.bam sample2.bam --bed regions.bed

This checks file existence, BAM indices, and format correctness.

2. Generate Workflow Template

For standard analyses, use the workflow generator to create customized scripts:

# List available workflows
python scripts/workflow_generator.py --list

# Generate ChIP-seq QC workflow
python scripts/workflow_generator.py chipseq_qc -o qc_workflow.sh \
    --input-bam Input.bam --chip-bams "ChIP1.bam ChIP2.bam" \
    --genome-size 2913022398

# Make executable and run
chmod +x qc_workflow.sh
./qc_workflow.sh

3. Most Common Operations

See assets/quick_reference.md for frequently used commands and parameters.

Installation

uv pip install deepTools==3.5.6

Upstream recommends conda/bioconda for full dependency resolution, especially on shared HPC systems:

conda install -c conda-forge -c bioconda deeptools

On Apple Silicon, upstream documents either the PyPI route above or an osx-64 conda environment when native conda packages are unavailable.

Core Workflows and Tool Categories

Complete command sequences for ChIP-seq QC, full ChIP-seq analysis, RNA-seq coverage, and ATAC-seq analysis — plus the BAM/bigWig processing, quality control, and visualization tool categories — are in references/coreworkflows.md and references/workflows.md. Per-tool options are in references/toolsreference.md.

Normalization Methods

Choosing the correct normalization is critical for valid comparisons. Consult references/normalization_methods.md for comprehensive guidance.

Quick selection guide:

• ChIP-seq coverage: Use RPGC or CPM
• ChIP-seq comparison: Use bamCompare with log2 and readCount
• RNA-seq bins: Use CPM
• RNA-seq genes: Use RPKM (accounts for gene length)
• ATAC-seq: Use RPGC or CPM

Normalization methods:

• RPGC: 1× genome coverage (requires --effectiveGenomeSize)
• CPM: Counts per million mapped reads
• RPKM: Reads per kb per million (per-bin length and library-size scaling)
• BPM: Bins per million, analogous to TPM-style scaling over binned signal
• None: Raw counts (not recommended for comparisons)

Full explanation: references/normalization_methods.md

Effective Genome Sizes

RPGC normalization requires effective genome size. Common values:

Complete table with read-length-specific values: references/effectivegenomesizes.md

Common Parameters Across Tools

Many deepTools commands share these options:

Performance:

• --numberOfProcessors, -p: Enable parallel processing (always use available cores)
• max / max/2: Supported values for --numberOfProcessors; useful under schedulers because recent deepTools releases detect CPU affinity more carefully
• --region: Process specific regions for testing (e.g., chr1:1-1000000)

Read Filtering:

• --ignoreDuplicates: Remove PCR duplicates (recommended for most analyses)
• --minMappingQuality: Filter by alignment quality (e.g., --minMappingQuality 10)
• --minFragmentLength / --maxFragmentLength: Fragment length bounds
• --samFlagInclude / --samFlagExclude: SAM flag filtering

Read Processing:

• --extendReads: Extend to fragment length (ChIP-seq: YES, RNA-seq: NO)
• --centerReads: Center at fragment midpoint for sharper signals

Best Practices

File Validation

Always validate files first using scripts/validate_files.py to check:

• File existence and readability
• BAM indices present (.bai files)
• BED format correctness
• File sizes reasonable

Analysis Strategy

1. Start with QC: Run correlation, coverage, and fingerprint analysis before proceeding
2. Test on small regions: Use --region chr1:1-10000000 for parameter testing
3. Document commands: Save full command lines for reproducibility
4. Use consistent normalization: Apply same method across samples in comparisons
5. Verify genome assembly: Ensure BAM and BED files use matching genome builds

ChIP-seq Specific

• Always extend reads for ChIP-seq: --extendReads 200
• Remove duplicates: Use --ignoreDuplicates in most cases
• Check enrichment first: Run plotFingerprint before detailed analysis
• GC correction: Only apply if significant bias detected; never use --ignoreDuplicates after GC correction

RNA-seq Specific

• Never extend reads for RNA-seq (would span splice junctions)
• Strand-specific: Use --filterRNAstrand forward/reverse for common dUTP-style stranded libraries; confirm library orientation before interpreting strand labels
• Normalization: CPM for bins, RPKM for genes

ATAC-seq Specific

• Apply Tn5 correction: Use alignmentSieve with --ATACshift
• Use only proper pairs for shifting: --ATACshift is equivalent to --shift 4 -5 5 -4 and filters to properly paired fragments
• Fragment filtering: Set appropriate min/max fragment lengths
• Check nucleosome pattern: Fragment size plot should show ladder pattern

Performance Optimization

1. Use multiple processors: --numberOfProcessors 8 (or available cores)
2. Increase bin size for faster processing and smaller files
3. Process chromosomes separately for memory-limited systems
4. Pre-filter BAM files using alignmentSieve to create reusable filtered files
5. Use bigWig over bedGraph: Compressed and faster to process

Troubleshooting

Common Issues