blackreach 5.0.0b1

Autonomous browser agent that accomplishes goals through web browsing

Blackreach

Autonomous Browser Agent - Give it a goal, watch it browse.

Blackreach is a CLI tool that uses AI to autonomously browse the web and accomplish tasks. It can navigate websites, search for content, download files (PDFs, images, datasets, etc.), and more.

blackreach run "find and download papers about machine learning from arxiv"

Features

• General-Purpose: Download any content type - papers, images, datasets, ebooks, etc.
• ReAct Pattern: Observe -> Think -> Act loop for intelligent browsing
• DOM Walker: Live browser DOM extraction gives the LLM numbered interactive elements
• Session Resume: Pause and resume interrupted sessions
• Smart Deduplication: Never download the same file twice (URL + hash checking)
• Memory System: Remembers successful patterns across sessions
• Multi-Provider: Ollama, OpenAI, Anthropic, Google, xAI
• Stealth Mode: Evades basic bot detection
• Stuck Detection: Automatically detects loops and recovers with alternate strategies

How It Works

Blackreach uses a DOM walker approach to let the LLM interact with web pages:

1. Observe: The DOM walker (dom_walker.py) runs JavaScript in the live browser to find all interactive elements (links, buttons, inputs, etc.) and assigns each a numeric [N] ID.
2. Think: The LLM receives the page text content and the numbered element list, then reasons about which action moves closest to the goal.
3. Act: The LLM outputs a JSON action referencing a specific element ID (e.g., {"action":"click","element":15}), and the agent executes it in the browser via Playwright.

This cycle repeats until the goal is accomplished or the step limit is reached. The agent includes stuck detection, automatic source failover, and error recovery to handle real-world browsing challenges.

Installation

Quick Install (pip)

pip install blackreach

Install with Cloud Providers

# With OpenAI support
pip install "blackreach[openai]"

# With Anthropic support
pip install "blackreach[anthropic]"

# With all providers
pip install "blackreach[all]"

From Source

git clone https://github.com/Null-Phnix/Blackreach
cd blackreach
pip install -e .

Post-Install: Browser Setup

Blackreach uses Playwright for browser automation. Install the browser:

playwright install chromium

Quick Start

First Run

blackreach

On first run, Blackreach will walk you through setup:

1. Install browser (if needed)
2. Choose AI provider (Ollama, OpenAI, etc.)
3. Configure API key (for cloud providers)

Basic Usage

# Interactive mode
blackreach

# Run with a specific goal
blackreach run "search wikipedia for artificial intelligence"

# Run headless (no browser window)
blackreach run --headless "download papers about transformers from arxiv"

# Use a specific provider/model
blackreach run -p openai -m gpt-4o "find papers about attention mechanisms"

# Resume an interrupted session
blackreach run --resume 42

Commands

Command	Description
blackreach	Interactive mode
blackreach run "goal"	Run agent with a goal
blackreach run --resume ID	Resume a paused session
blackreach sessions	List resumable sessions
blackreach config	Configure settings and API keys
blackreach models	List available models
blackreach status	Show current configuration
blackreach stats	Show performance metrics
blackreach setup	Run setup wizard
blackreach doctor	Check system requirements
blackreach health	Check content source availability
blackreach downloads	Show download history

Interactive Commands

In interactive mode, use these slash commands:

Command	Short	Description
/help	/h	Show help
/model	/m	Switch model
/provider	/p	Switch provider
/status	/s	Show status
/plan "goal"		Preview a plan without executing
/sessions		List resumable sessions
/resume ID		Resume a session
/logs	/l	View recent logs
/clear	/cls	Clear screen
/quit	/q	Exit

Supported AI Providers

Provider	Type	Models
Ollama	Local	qwen2.5:7b, llama3.2:3b, mistral:7b
xAI	Cloud	grok-2, grok-2-mini
OpenAI	Cloud	gpt-4o, gpt-4o-mini
Anthropic	Cloud	claude-sonnet-4-6, claude-haiku-4-5
Google	Cloud	gemini-2.5-pro, gemini-2.5-flash

Using Ollama (Local, Free, Private)

1. Install Ollama: https://ollama.ai
2. Pull a model: ollama pull qwen2.5:7b
3. Start Ollama: ollama serve
4. Use Blackreach: blackreach

Using Cloud Providers

1. Get API key from your provider
2. Configure: blackreach config -> Set API key
3. Switch provider: blackreach config -> Set default provider

Configuration

Config file: ~/.blackreach/config.yaml

Environment Variables

export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="sk-ant-..."
export GOOGLE_API_KEY="..."
export XAI_API_KEY="xai-..."

Examples

Research Papers

blackreach run "go to arxiv.org, search for 'attention mechanism', download 3 papers"

Images and Media

blackreach run "find and download landscape wallpapers from unsplash"

Datasets

blackreach run "download CSV files about climate data from kaggle"

Documentation

blackreach run "go to github.com/pytorch/pytorch and download the README"

Ebooks

blackreach run "find and download 'pride and prejudice' from project gutenberg"

Session Resume

Sessions are automatically saved when interrupted (Ctrl+C):

# Start a task
blackreach run "download 10 papers from arxiv"
# Press Ctrl+C to pause

# Later, resume where you left off
blackreach sessions  # See available sessions
blackreach run --resume 42  # Resume session #42

Architecture

blackreach/
├── agent.py          # ReAct loop coordinator
├── browser.py        # Playwright browser control (stealth, downloads)
├── dom_walker.py     # Live DOM extraction - assigns [N] IDs to interactive elements
├── observer.py       # Legacy HTML parsing (fallback utilities)
├── llm.py            # Multi-provider LLM integration
├── memory.py         # Session memory + SQLite persistence
├── detection.py      # CAPTCHA, login, paywall detection
├── knowledge.py      # Content source knowledge base
├── resilience.py     # Retry logic, circuit breaker
├── stuck_detector.py # Loop detection and recovery strategies
├── error_recovery.py # Error categorization and recovery
├── exceptions.py     # Error hierarchy
├── config.py         # Configuration management
├── logging.py        # Structured session logging
├── ui.py             # Rich terminal UI components
└── cli.py            # Command-line interface

Troubleshooting

Check System Status

blackreach doctor

Common Issues

Browser not found:

playwright install chromium

Ollama not running:

ollama serve

No API key configured:

blackreach config          # Interactive setup
# or set environment variable:
export OPENAI_API_KEY="sk-..."

Bot detection (418/403 errors):

• Some sites block headless browsers
• Try running without --headless
• Try a different browser: blackreach run -b firefox "your goal"
• Use different search engines (Google/Wikipedia work better than DuckDuckGo)

Session resume fails:

blackreach sessions  # Check if session exists

Memory and Learning

Blackreach maintains two types of memory:

1. Session Memory (RAM): Current session state
2. Persistent Memory (SQLite): Cross-session learning

The persistent memory tracks:

• All downloads (prevents re-downloading)
• Site patterns that worked
• Action success rates per domain
• Common failures to avoid

View stats:

blackreach stats

License

MIT

Contributing

Contributions welcome! Please open an issue or PR on GitHub.