Skip to main content

Python client for Evomi web scraping API

Project description

Evomi Python Client

A production-ready Python client for the Evomi web scraping API. Provides both synchronous and asynchronous clients for maximum flexibility.

Features

  • Async & Sync Support - Use EvomiClient for async or EvomiClientSync for synchronous operations
  • Full API Coverage - All Evomi endpoints supported
  • Type Hints - Complete type annotations for IDE support
  • Minimal Dependencies - Only requires httpx

Installation

pip install evomi-client

Quick Start

Async Client

import asyncio
from evomi_client import EvomiClient

async def main():
    # Initialize with API key (or set EVOMI_API_KEY env var)
    client = EvomiClient(api_key="your-api-key")
    
    # Scrape a URL
    result = await client.scrape("https://example.com")
    print(result)
    
    # Get markdown output
    result = await client.scrape(
        "https://example.com",
        output="markdown",
        mode="auto"  # auto-detect JS requirement
    )
    
    # AI-powered extraction
    result = await client.scrape(
        "https://example.com/products",
        ai_enhance=True,
        ai_prompt="Extract product names and prices"
    )

asyncio.run(main())

Sync Client

from evomi_client import EvomiClientSync

# Initialize with API key (or set EVOMI_API_KEY env var)
client = EvomiClientSync(api_key="your-api-key")

# Scrape a URL
result = client.scrape("https://example.com")
print(result)

API Reference

Scraping Operations

scrape(url, ...)

Scrape a single URL with configurable options.

Parameter Type Default Description
url str required URL to scrape
mode str "auto" Scraping mode: "request" (fast), "browser" (JS), "auto" (detect)
output str "markdown" Output format: "html", "markdown", "screenshot", "pdf"
device str "windows" Device type: "windows", "macos", "android"
proxy_type str "residential" Proxy type: "datacenter", "residential"
proxy_country str "US" Two-letter country code
proxy_session_id str None Proxy session ID (6-8 chars)
wait_until str "domcontentloaded" Wait condition
ai_enhance bool False Enable AI enhancement
ai_prompt str None Prompt for AI extraction
ai_source str None AI source: "markdown", "screenshot"
js_instructions list None JS actions: click, wait, fill, wait_for
execute_js str None Raw JavaScript to execute
screenshot bool False Capture screenshot
pdf bool False Capture PDF
wait_seconds int 0 Seconds to wait after load
excluded_tags list None HTML tags to remove
excluded_selectors list None CSS selectors to remove
block_resources list None Resource types to block
additional_headers dict None Extra HTTP headers
capture_headers bool False Capture response headers
network_capture list None Network capture filters
async_mode bool False Return immediately with task ID
config_id str None Saved config ID
scheme_id str None Saved extraction schema ID
extract_scheme list None Inline extraction schema
storage_id str None Storage config ID
use_default_storage bool False Use default storage
no_html bool False Exclude HTML from response

crawl(domain, ...)

Crawl a website to discover and scrape multiple pages.

result = await client.crawl(
    domain="example.com",
    max_urls=100,
    depth=2,
    url_pattern="/products/.*",  # Regex filter
    async_mode=True  # Returns task_id
)

map_website(domain, ...)

Discover URLs from a website.

result = await client.map_website(
    domain="example.com",
    sources=["sitemap", "commoncrawl"],
    max_urls=500
)

search_domains(query, ...)

Find domains by searching the web.

result = await client.search_domains(
    query="best e-commerce sites",
    max_urls=20,
    region="us-en"
)

agent_request(message)

Send a natural language request to the AI agent.

result = await client.agent_request(
    "Scrape example.com and extract all product prices"
)

get_task_status(task_id, task_type)

Check the status of an async task.

result = await client.get_task_status(
    task_id="abc123",
    task_type="scrape"  # or "crawl", "map", "config_generate", "schema"
)

Config Management

# List configs
configs = await client.list_configs()

# Create config
config = await client.create_config(
    name="My Scraper",
    config={"mode": "browser", "output": "markdown"}
)

# Get config
config = await client.get_config("cfg_abc123")

# Update config
config = await client.update_config("cfg_abc123", name="New Name")

# Delete config
await client.delete_config("cfg_abc123")

# Generate config from natural language
config = await client.generate_config(
    name="Amazon Scraper",
    prompt="Scrape product title, price, and reviews from Amazon"
)

Schema Management

# List schemas
schemas = await client.list_schemas()

# Create schema with extraction rules
schema = await client.create_schema(
    name="Product Schema",
    config={
        "url": "https://example.com/product",
        "extract_scheme": [
            {"label": "title", "type": "content", "selector": "h1"},
            {"label": "price", "type": "content", "selector": ".price"}
        ]
    },
    test=True  # Test the schema
)

# Get schema status (for async testing)
status = await client.get_schema_status("sch_abc123")

Schedule Management

# Create a scheduled job
schedule = await client.create_schedule(
    name="Daily Price Check",
    config_id="cfg_abc123",
    interval_minutes=1440,  # Daily
    start_time="09:00"  # UTC
)

# List schedules
schedules = await client.list_schedules(active_only=True)

# Toggle schedule
await client.toggle_schedule("sched_abc123")

# Get execution history
runs = await client.list_schedule_runs("sched_abc123")

Storage Management

# Create S3 storage config
storage = await client.create_storage_config(
    name="My S3",
    storage_type="s3_compatible",
    config={
        "bucket": "my-bucket",
        "region": "us-east-1",
        "access_key": "...",
        "secret_key": "..."
    },
    set_as_default=True
)

# List storage configs
configs = await client.list_storage_configs()

Account Info

info = await client.get_account_info()
print(f"Credits remaining: {info.get('credits', 'N/A')}")

Pricing & Credits

All operations consume credits:

  • Base cost: 1 credit per request
  • Browser mode: 5x multiplier
  • Residential proxy: 2x multiplier
  • AI enhancement: +30 credits
  • Screenshot/PDF: +1 credit each

Credit information is returned in response headers and in _credits_used, _credits_remaining fields.

Error Handling

from evomi_client import EvomiClient

client = EvomiClient(api_key="your-key")

try:
    result = await client.scrape("https://example.com")
except httpx.HTTPStatusError as e:
    print(f"HTTP error: {e.response.status_code}")
    print(f"Response: {e.response.text}")
except httpx.RequestError as e:
    print(f"Request error: {e}")

Configuration

Set your API key via environment variable:

export EVOMI_API_KEY="your-api-key"

Or pass it directly:

client = EvomiClient(api_key="your-api-key")

Custom base URL (for testing):

client = EvomiClient(
    api_key="your-api-key",
    base_url="https://custom.evomi.com"
)

Links

License

MIT License - see LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

evomi_client-1.0.0.tar.gz (9.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

evomi_client-1.0.0-py3-none-any.whl (10.1 kB view details)

Uploaded Python 3

File details

Details for the file evomi_client-1.0.0.tar.gz.

File metadata

  • Download URL: evomi_client-1.0.0.tar.gz
  • Upload date:
  • Size: 9.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.1

File hashes

Hashes for evomi_client-1.0.0.tar.gz
Algorithm Hash digest
SHA256 62be487c8bf8a2599fc0de5ed8021268fb9be1f0f5e4278c1c778294eb783383
MD5 9efd729ded4fcdf32502e09139ef3758
BLAKE2b-256 0ae88ebefe7ded3b0736c300fdb5c3007ebae1a4dce5316d4861b84f59c6207e

See more details on using hashes here.

File details

Details for the file evomi_client-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: evomi_client-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 10.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.10.1

File hashes

Hashes for evomi_client-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 0347b4b12bf54c6c48e3d170a57ecfefaeb8a2c2539586c129051f49d52cb34f
MD5 e30d71fabdcc548a21b57f5804955a24
BLAKE2b-256 ef1deb04354da858d60ec33e5c6d8f01d4ff4edf7adadb79e3765d0f62a6426c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page