analog-fetcher

JS-aware fetching for Analog. Two fetchers, both implementing the analog.fetcher.Fetcher protocol:

• AnalogFetcher — the recommended default. Does a cheap HTTP fetch first, analyzes the HTML to decide if rendering is actually needed, and only escalates to a real browser when it is. Caches the decision per-domain so subsequent fetches skip the probe.
• PlaywrightFetcher — direct Playwright access. Use when you know you want JS rendering on every fetch and don't want the probe overhead.

When analog-fetcher is installed alongside analog-sdk, AnalogFetcher becomes the SDK's default automatically — no code change required on the caller side.

Installation

pip install "analog-sdk[fetcher]"   # installs analog-sdk + analog-fetcher
# or
pip install analog-fetcher           # standalone (pulls analog-sdk too)

Then install the browser binaries Playwright uses:

playwright install chromium

Usage

The simplest path — let the SDK use the smart default:

from analog import analog

# AnalogFetcher is the default when analog-fetcher is installed.
result = analog("https://example.com")

Or construct one explicitly with custom behavior:

from analog import analog
from analog_fetcher import AnalogFetcher, PlaywrightFetcher

# Customize the rich fetcher used during escalation.
fetcher = AnalogFetcher(
    rich=PlaywrightFetcher(wait_for=".loaded", headless=True),
)
result = analog("https://example.com", fetcher=fetcher)

Or force Playwright on every fetch:

from analog_fetcher import PlaywrightFetcher

with PlaywrightFetcher(headless=True) as fetcher:
    r1 = analog("https://example.com/a", fetcher=fetcher)
    r2 = analog("https://example.com/b", fetcher=fetcher)
    # Browser launches once, is reused across both fetches.

Decision cache

AnalogFetcher records each domain's "needs JS?" verdict on disk so repeat fetches skip the probe. The cache lives under your per-user cache directory:

• Linux: ~/.cache/analog/fetch_decisions.json (or $XDG_CACHE_HOME/analog/…)
• macOS: ~/Library/Caches/analog/fetch_decisions.json

Set ANALOG_CACHE_DIR to relocate it, or pass cache_path= to AnalogFetcher. Verdicts expire after 7 days; pass cache_ttl_seconds=0 to re-probe on every fetch, or delete the file to reset sooner.

See https://getanalog.io for full documentation.

License

MIT — see LICENSE.