lobstrio

lobstrio
Command-line interface for the Lobstr.io scraping API

Ruff

Run web scrapers, manage squids, download results — all from your terminal.

Demo

lobstrio CLI demo

Installation

pip install lobstrio

Requires Python 3.10+.

Quick start

# Save your API token (find it at https://app.lobstr.io/dashboard/api)
lobstr config set-token YOUR_TOKEN

# One-command scrape: create squid, add tasks, run, download
lobstr go google-maps-leads-scraper "https://maps.google.com/maps/place/..." -o leads.csv

# Multiple URLs
lobstr go google-maps-leads-scraper url1 url2 url3 -o results.csv

# From a file
lobstr go google-maps-leads-scraper --file urls.txt -o results.csv

Commands

Account & config

lobstr whoami                          # Show account info and balance
lobstr config set-token TOKEN          # Save API token
lobstr config show                     # Show current config
lobstr config set-alias maps SQUID     # Create alias for a squid

Crawlers — browse the scraper catalog

lobstr crawlers ls                                   # List all available crawlers
lobstr crawlers show google-maps-leads-scraper       # Show crawler details
lobstr crawlers search "Google Maps"                 # Search by name
lobstr crawlers params google-maps-leads-scraper     # Show crawler parameters
lobstr crawlers attrs google-maps-leads-scraper      # Show result attributes

Squids — manage scraper instances

lobstr squid create google-maps-leads-scraper --name "My Scraper"
lobstr squid ls                        # List your squids
lobstr squid show SQUID                # Show details
lobstr squid update SQUID --concurrency 5 --param max_results=200
lobstr squid empty SQUID               # Remove all tasks
lobstr squid rm SQUID --force          # Delete squid

Tasks — manage input URLs and keywords

lobstr task add SQUID url1 url2        # Add tasks
lobstr task add SQUID "pizza" --key keyword  # Keyword-based crawlers
lobstr task ls SQUID                   # List tasks
lobstr task show FULL_TASK_HASH        # Show task details
lobstr task rm FULL_TASK_HASH          # Delete task
lobstr task upload SQUID tasks.csv     # Bulk upload from CSV
lobstr task upload-status UPLOAD_ID    # Check upload progress

Runs — start, monitor, and download

lobstr run start SQUID                 # Start a run
lobstr run start SQUID --wait          # Start and wait for completion
lobstr run start SQUID --download results.csv  # Start, wait, download
lobstr run ls SQUID                    # List runs
lobstr run show FULL_RUN_HASH          # Show run details
lobstr run stats FULL_RUN_HASH         # Show run statistics
lobstr run tasks FULL_RUN_HASH         # List tasks in a run
lobstr run watch FULL_RUN_HASH         # Live progress bar
lobstr run abort FULL_RUN_HASH         # Stop a run
lobstr run download FULL_RUN_HASH      # Download results CSV

Results — fetch scraped data

lobstr results get SQUID               # Fetch results (JSON)
lobstr results get SQUID --format csv  # Fetch as CSV
lobstr results get SQUID -o data.json  # Save to file

Accounts — manage connected platform accounts

lobstr accounts ls                     # List all accounts
lobstr accounts show ACCOUNT           # Show account details
lobstr accounts types                  # List available account types
lobstr accounts sync --type google --cookies-file cookies.json  # Sync account
lobstr accounts sync-status SYNC_ID    # Check sync progress
lobstr accounts update ACCOUNT --param daily_limit=100
lobstr accounts rm ACCOUNT --force     # Delete account

Delivery — configure result delivery

lobstr delivery email SQUID --email you@example.com
lobstr delivery googlesheet SQUID --url "https://docs.google.com/..."
lobstr delivery s3 SQUID --bucket my-bucket --target-path scrapes/
lobstr delivery webhook SQUID --url "https://your-server.com/hook"
lobstr delivery sftp SQUID --host ftp.example.com --username user --password pass

# Test connectivity
lobstr delivery test-email --email you@example.com
lobstr delivery test-s3 --bucket my-bucket

Go — full workflow in one command

# Basic usage
lobstr go google-maps-leads-scraper "https://maps.google.com/..." -o results.csv

# Keyword-based crawler
lobstr go google-search-scraper "pizza delivery" --key keyword

# With crawler parameters
lobstr go google-maps-leads-scraper url1 --param max_results=200 --param language=English

# Set concurrency
lobstr go google-maps-leads-scraper url1 --concurrency 3

# Start without waiting for download
lobstr go google-maps-leads-scraper url1 --no-download

# Reuse existing squid by name
lobstr go google-maps-leads-scraper url1 --name "My Leads"

# Clear old tasks when reusing squid
lobstr go google-maps-leads-scraper url1 --name "My Leads" --empty

# Delete squid after completion
lobstr go google-maps-leads-scraper url1 --delete

# Custom output file
lobstr go google-maps-leads-scraper url1 -o my_leads.csv

Global flags

Flag	Description
`--json`	Output raw JSON (for piping/scripting)
`--quiet`	Suppress non-essential output
`--verbose`	Show HTTP request details
`--token TOKEN`	Override API token for this command
`--version`	Show version

Aliases

Create shortcuts for frequently used squids:

lobstr config set-alias maps abc123def456...
lobstr task ls @maps
lobstr run start @maps

Identifier resolution

Resource	Resolution order	Example
Crawlers	Hash prefix → Slug (exact/prefix) → Name (exact/substring)	`google-maps`, `4734d096`, `"Google Maps"`
Squids	`@alias` → Hash prefix → Name (exact/substring)	`@maps`, `abc1`, `"My Scraper"`
Accounts	Hash prefix → Username (exact/substring)	`f9a2`, `"john@gmail.com"`
Runs & Tasks	Full 32-character hash only	`a1b2c3d4e5f6...`

Configuration

Config is stored at ~/.config/lobstr/config.toml. The API token can also be set via the LOBSTR_TOKEN environment variable.

CLI vs SDK

	CLI (`pip install lobstrio`)	SDK (`pip install lobstrio-sdk`)
Use case	Terminal workflows, quick scrapes, cron jobs	Scripts, pipelines, applications
Interface	Shell commands	Python API
Output	Rich tables, progress bars, CSV files	Typed dataclass models
Async	No	Yes (`AsyncLobstrClient`)
Pagination	Manual (`--page`, `--limit`)	Auto (`client.squids.iter()`)

For programmatic access, see lobstrio-sdk.

FAQ

Where do I get an API token?

Go to Dashboard → API to find your token. It's always available there, pre-generated.

How do I use keyword-based crawlers?

Some crawlers accept keywords instead of URLs. Use the --key flag:

lobstr go google-search-scraper "pizza delivery" --key keyword

Use lobstr crawlers params <crawler> to see what parameters a crawler accepts.

Can I use short hashes for runs and tasks?

No. Run and task endpoints require the full 32-character hash. Use lobstr run ls SQUID or lobstr task ls SQUID to see full hashes. Crawlers and squids support prefix matching.

How do I pipe results to other tools?

Use --json for machine-readable output:

lobstr --json results get SQUID | jq '.[].email'
lobstr --json crawlers ls | jq '.[] | .name'

Can I run scrapes in the background?

Yes. Use --no-download with go, or start a run without --wait:

lobstr go google-maps-leads-scraper urls.txt --no-download
lobstr run start SQUID  # returns immediately
lobstr run watch RUN_HASH  # check progress later

Releasing

Publishing to PyPI (the lobstrio package) is triggered by a GitHub Release, not by merging to main. Merging alone does not publish. To cut a release:

Bump version in pyproject.toml (e.g. 0.5.1).
Add a matching entry to CHANGELOG.md.
Merge to main (via PR; CI must be green).

Create a GitHub Release tagged vX.Y.Z on main:

gh release create v0.5.1 --target main --title "v0.5.1" --notes "…"

Creating the release fires .github/workflows/publish.yml (on: release: published), which builds with uv and publishes to PyPI using the PYPI_TOKEN repo secret. Confirm it went out: pip index versions lobstrio (or check the PyPI page). Note: a PyPI version cannot be overwritten — bump the version for every release.

Contributing

Contributions are welcome! See CONTRIBUTING.md for development setup, code style, and versioning guidelines.

Changelog

See CHANGELOG.md for release history.

License

Apache 2.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lobstrio-0.6.0.tar.gz (572.3 kB view details)

Uploaded Sep 10, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

lobstrio-0.6.0-py3-none-any.whl (31.9 kB view details)

Uploaded Sep 10, 2026 Python 3

File details

Details for the file lobstrio-0.6.0.tar.gz.

File metadata

Download URL: lobstrio-0.6.0.tar.gz
Upload date: Sep 10, 2026
Size: 572.3 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.12.12 {"installer":{"name":"uv","version":"0.12.12","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for lobstrio-0.6.0.tar.gz
Algorithm	Hash digest
SHA256	`c27a1fd154613e4675bec42f3a334123078aee6b1cc9feff211b44fea3ed8365`
MD5	`d10c299381d038254636483e353979ff`
BLAKE2b-256	`ec75bfbc28446f024be77f424b861797f65f582ceeeb626e78bbbcdc1986b752`

See more details on using hashes here.

File details

Details for the file lobstrio-0.6.0-py3-none-any.whl.

File metadata

Download URL: lobstrio-0.6.0-py3-none-any.whl
Upload date: Sep 10, 2026
Size: 31.9 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.12.12 {"installer":{"name":"uv","version":"0.12.12","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for lobstrio-0.6.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`e82ba4488df8b34a537de65bb9b1a68a17ced4c7dd8ebb56de48d8a4c34e735d`
MD5	`675180f9b81851cb131b05b023b2e1ed`
BLAKE2b-256	`6325f795155e52e184d672ac9e57da9c76ddad6481b2596273a7d1917d3b0c5d`