chrome-automation skill

Chrome Browser Automation with Real-time Control

Control the user's Google Chrome browser using agent-browser connected via Chrome DevTools Protocol (CDP). All actions are visible in real-time in the user's browser window.

⚠️ CRITICAL: Automatic Installation & Dependency Check

BEFORE using this skill, ALWAYS run the auto-installation check:

This skill automatically checks and installs all required dependencies without user confirmation. The installation process verifies:

• agent-browser repository and binary
• Node.js and pnpm
• Playwright Chromium
• Rust toolchain (cargo)
• Native binary compilation

Auto-installation workflow:

# Mac: Run auto-install script
bash ~/.claude/skills/chrome-automation/scripts/auto-install-mac.sh

# Windows: Run auto-install script (PowerShell)
powershell -ExecutionPolicy Bypass -File "$HOME\.claude\skills\chrome-automation\scripts\auto-install-windows.ps1"

The script will:

1. Check if agent-browser binary exists and works
2. Install missing dependencies automatically (pnpm, Rust, etc.)
3. Clone repository if not present
4. Build native binary with proper Rust toolchain
5. Verify installation success

Critical rules:

• ❌ DO NOT fall back to using built-in browser tools (browserusedesktop)
• ✅ MUST run auto-install script before any browser automation
• ✅ Only proceed with browser automation AFTER successful installation
• ✅ Script handles all dependency checks automatically

Why agent-browser is required:

• Provides real-time visual feedback in user's actual Chrome browser
• Allows user to see and interact with automation in progress
• Offers precise element selection and interaction capabilities
• Built-in browser tools are NOT a substitute for this skill's functionality

Quick Start

First-time setup (automatic)

Run the auto-install script first:

# Mac
bash ~/.claude/skills/chrome-automation/scripts/auto-install-mac.sh

# Windows (PowerShell)
powershell -ExecutionPolicy Bypass -File "$HOME\.claude\skills\chrome-automation\scripts\auto-install-windows.ps1"

The script automatically handles:

• Repository cloning
• Dependency installation (pnpm, Playwright)
• Rust toolchain setup
• Native binary compilation

Connect and control Chrome

IMPORTANT: Use the start-chrome script to preserve login states across sessions

# Mac: Start Chrome with persistent profile (RECOMMENDED)
# This preserves your login states, cookies, and settings
bash ~/.claude/skills/chrome-automation/scripts/start-chrome-mac.sh

# Windows: Start Chrome with persistent profile (PowerShell)
powershell -ExecutionPolicy Bypass -File "$HOME\.claude\skills\chrome-automation\scripts\start-chrome-windows.ps1"

# Use agent-browser with --cdp flag
cd /Users/jyxc-dz-0100272/Documents/agent-browser
AGENT_BROWSER_HOME=/Users/jyxc-dz-0100272/Documents/agent-browser \
./bin/agent-browser --cdp 9222 snapshot -i

What the start-chrome script does:

1. First run: Creates a dedicated automation profile at ~/Library/Application Support/Google/Chrome-Automation
2. First run: Imports your existing Chrome configuration (login states, cookies, settings)
3. Subsequent runs: Uses the existing automation profile - all your logins are preserved
4. You only need to log in once - it will be remembered for future sessions

Core Workflow

CRITICAL: Always show browser state to user after each significant action using screenshots.

1. Connect: Ensure Chrome running with remote debugging port 9222
2. Navigate: Open pages using --cdp 9222 flag
3. Snapshot: Get interactive elements with snapshot -i
4. Interact: Use element refs (@e1, @e2, etc.)
5. Screenshot: Capture state and show to user
6. Report: Describe what's visible in browser

Standard operation pattern

# Set path variables
export AB_HOME=/Users/jyxc-dz-0100272/Documents/agent-browser
cd $AB_HOME

# Navigate
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 open https://example.com
sleep 2

# Get interactive elements
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 snapshot -i

# Screenshot to show user
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 screenshot /tmp/state.png

# Interact
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 fill @e1 "text"
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 press Enter

# Capture result
sleep 2
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 screenshot /tmp/result.png

Installation & Setup

Automatic Installation (RECOMMENDED)

Always use the auto-install script - it handles all dependencies automatically:

Mac Installation

# Run auto-install script
bash ~/.claude/skills/chrome-automation/scripts/auto-install-mac.sh

What it does:

1. Checks if agent-browser binary exists and works
2. Clones repository if missing
3. Installs Node.js dependencies (pnpm)
4. Installs Playwright Chromium
5. Checks/installs Rust toolchain (cargo, rustup)
6. Sets up stable Rust toolchain if needed
7. Builds native binary with npm run build:native
8. Verifies installation success

Requirements:

• Node.js (will notify if missing)
• Git (for cloning repository)
• Internet connection

Windows Installation

# Run auto-install script (PowerShell)
powershell -ExecutionPolicy Bypass -File "$HOME\.claude\skills\chrome-automation\scripts\auto-install-windows.ps1"

What it does:

1. Checks if agent-browser binary exists and works
2. Clones repository if missing
3. Installs Node.js dependencies (pnpm)
4. Installs Playwright Chromium
5. Downloads and installs Rust toolchain if needed
6. Sets up stable Rust toolchain
7. Builds native binary with npm run build:native
8. Verifies installation success

Requirements:

• Node.js (will notify if missing - download from https://nodejs.org/)
• Git (for cloning repository)
• Internet connection

Manual Installation (Advanced Users Only)

If you prefer manual installation or need to troubleshoot:

Mac Manual Steps

# 1. Clone repository
cd ~/Documents
git clone https://github.com/vercel-labs/agent-browser.git
cd agent-browser

# 2. Install Node dependencies
pnpm install
npx playwright install chromium

# 3. Install Rust (if not installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
export PATH="$HOME/.cargo/bin:$PATH"
rustup default stable

# 4. Build native binary
npm run build:native

Windows Manual Steps

# 1. Clone repository
Set-Location "$HOME\Documents"
git clone https://github.com/vercel-labs/agent-browser.git
Set-Location agent-browser

# 2. Install Node dependencies
pnpm install
npx playwright install chromium

# 3. Install Rust (if not installed)
# Download from: https://win.rustup.rs/x86_64
# Or use the auto-install script

# 4. Build native binary
npm run build:native

Verify Installation

After installation (automatic or manual), verify it works:

# Mac
cd ~/Documents/agent-browser
AGENT_BROWSER_HOME=~/Documents/agent-browser ./bin/agent-browser --version

# Windows (PowerShell)
Set-Location "$HOME\Documents\agent-browser"
$env:AGENT_BROWSER_HOME = "$HOME\Documents\agent-browser"
.\bin\agent-browser.cmd --version

Troubleshooting Installation

"No binary found for darwin-arm64" or "No binary found for win32-x64":

• Rust is not installed or not in PATH
• Run the auto-install script, it will handle Rust installation
• Or manually install Rust and run npm run build:native

"cargo: command not found":

• Rust toolchain not in PATH
• Mac: export PATH="$HOME/.cargo/bin:$PATH"
• Windows: $env:PATH = "$HOME\.cargo\bin;$env:PATH"
• Then run rustup default stable and npm run build:native

"rustup could not choose a version":