lobstrio

lobstrio
Command-line interface for the Lobstr.io scraping API

Ruff

Run web scrapers, manage squids, download results — all from your terminal.

Demo

lobstrio CLI demo

Installation

pip install lobstrio

Requires Python 3.10+.

Quick start

# Save your API token (find it at https://app.lobstr.io/dashboard/api)
lobstr config set-token YOUR_TOKEN

# One-command scrape: create squid, add tasks, run, download
lobstr go google-maps-leads-scraper "https://maps.google.com/maps/place/..." -o leads.csv

# Multiple URLs
lobstr go google-maps-leads-scraper url1 url2 url3 -o results.csv

# From a file
lobstr go google-maps-leads-scraper --file urls.txt -o results.csv

Commands

Account & config

lobstr whoami                          # Show account info and balance
lobstr config set-token TOKEN          # Save API token
lobstr config show                     # Show current config
lobstr config set-alias maps SQUID     # Create alias for a squid

Crawlers — browse the scraper catalog

lobstr crawlers ls                                   # List all available crawlers
lobstr crawlers show google-maps-leads-scraper       # Show crawler details
lobstr crawlers search "Google Maps"                 # Search by name
lobstr crawlers params google-maps-leads-scraper     # Show crawler parameters
lobstr crawlers attrs google-maps-leads-scraper      # Show result attributes

ls, show, and search show which account type a crawler needs (e.g. sales-nav-sync) in the "Needs Account" column, instead of a plain yes/no — that's the value to pass --account's matching account for. --json output is unaffected.

Squids — manage scraper instances

lobstr squid create google-maps-leads-scraper --name "My Scraper"
lobstr squid ls                        # List your squids
lobstr squid show SQUID                # Show details
lobstr squid update SQUID --concurrency 5 --param max_results=200

# Link an account (crawlers that need one, e.g. sales-navigator-leads-scraper).
# Merged into whatever the squid already has by default — the API's `accounts`
# field is full-replace, so a naive call would detach anything already linked:
lobstr squid update SQUID --account jane@example.com
# Replace the whole account list instead of merging (empty = detach all):
lobstr squid update SQUID --replace-accounts --account jane@example.com

lobstr squid empty SQUID               # Remove all tasks
lobstr squid rm SQUID --force          # Delete squid

squid create has no --account — the API doesn't accept accounts at creation, only on squid update. lobstr go on an account-backed crawler handles this for you (see below): create, then link, in one command.

Tasks — manage input URLs and keywords

lobstr task add SQUID url1 url2        # Add tasks
lobstr task add SQUID "pizza" --key keyword  # Keyword-based crawlers
lobstr task ls SQUID                   # List tasks
lobstr task show FULL_TASK_HASH        # Show task details
lobstr task rm FULL_TASK_HASH          # Delete task
lobstr task upload SQUID tasks.csv     # Bulk upload from CSV
lobstr task upload-status UPLOAD_ID    # Check upload progress

Runs — start, monitor, and download

lobstr run start SQUID                 # Start a run
lobstr run start SQUID --wait          # Start and wait for completion
lobstr run start SQUID --download results.csv  # Start, wait, download
lobstr run ls SQUID                    # List runs
lobstr run show FULL_RUN_HASH          # Show run details
lobstr run stats FULL_RUN_HASH         # Show run statistics
lobstr run tasks FULL_RUN_HASH         # List tasks in a run
lobstr run watch FULL_RUN_HASH         # Live progress bar
lobstr run abort FULL_RUN_HASH         # Stop a run
lobstr run download FULL_RUN_HASH      # Download results CSV

Results — fetch scraped data

lobstr results get SQUID               # Fetch results (JSON)
lobstr results get SQUID --format csv  # Fetch as CSV
lobstr results get SQUID -o data.json  # Save to file

Accounts — manage connected platform accounts

lobstr accounts ls                     # List all accounts
lobstr accounts show ACCOUNT           # Show account details
lobstr accounts types                  # List available account types
lobstr accounts sync --type google --cookies-file cookies.json  # Sync account
lobstr accounts sync-status SYNC_ID    # Check sync progress
lobstr accounts update ACCOUNT --param daily_limit=100
lobstr accounts rm ACCOUNT --force     # Delete account

Delivery — configure result delivery

lobstr delivery email SQUID --email you@example.com
lobstr delivery googlesheet SQUID --url "https://docs.google.com/..."
lobstr delivery s3 SQUID --bucket my-bucket --target-path scrapes/
lobstr delivery webhook SQUID --url "https://your-server.com/hook"
lobstr delivery sftp SQUID --host ftp.example.com --username user --password pass

# Test connectivity
lobstr delivery test-email --email you@example.com
lobstr delivery test-s3 --bucket my-bucket

Go — full workflow in one command

# Basic usage
lobstr go google-maps-leads-scraper "https://maps.google.com/..." -o results.csv

# Keyword-based crawler
lobstr go google-search-scraper "pizza delivery" --key keyword

# With crawler parameters
lobstr go google-maps-leads-scraper url1 --param max_results=200 --param language=English

# Set concurrency
lobstr go google-maps-leads-scraper url1 --concurrency 3

# Start without waiting for download
lobstr go google-maps-leads-scraper url1 --no-download

# Reuse existing squid by name
lobstr go google-maps-leads-scraper url1 --name "My Leads"

# Clear old tasks when reusing squid
lobstr go google-maps-leads-scraper url1 --name "My Leads" --empty

# Delete squid after completion
lobstr go google-maps-leads-scraper url1 --delete

# Custom output file
lobstr go google-maps-leads-scraper url1 -o my_leads.csv

# Crawlers that need a synced account (e.g. LinkedIn, Sales Navigator):
# auto-picks the one healthy account of the right type if there's exactly one
lobstr go sales-navigator-leads-scraper "https://linkedin.com/..."

# ...or name it explicitly (needed when you have more than one candidate)
lobstr go sales-navigator-leads-scraper "https://linkedin.com/..." --account jane@example.com

For an account-backed crawler, go links an account automatically: exactly one healthy account of the crawler's account type (not its name — two LinkedIn crawlers can need different types) is picked for you; zero or several fail with a message telling you what to do, and pass --account to pick one yourself. A reused squid (--name) that already has an account attached is left alone — go never auto-picks onto it.

Global flags

Flag	Description
`--json`	Output raw JSON (for piping/scripting)
`--quiet`	Suppress non-essential output
`--verbose`	Show HTTP request details
`--token TOKEN`	Override API token for this command
`--version`	Show version

Aliases

Create shortcuts for frequently used squids:

lobstr config set-alias maps abc123def456...
lobstr task ls @maps
lobstr run start @maps

Identifier resolution

Resource	Resolution order	Example
Crawlers	Hash prefix → Slug (exact/prefix) → Name (exact/substring)	`google-maps`, `4734d096`, `"Google Maps"`
Squids	`@alias` → Hash prefix → Name (exact/substring)	`@maps`, `abc1`, `"My Scraper"`
Accounts	Hash prefix → Username (exact/substring)	`f9a2`, `"john@gmail.com"`
Runs & Tasks	Full 32-character hash only	`a1b2c3d4e5f6...`

Configuration

Config is stored at ~/.config/lobstr/config.toml. The API token can also be set via the LOBSTR_TOKEN environment variable.

CLI vs SDK

	CLI (`pip install lobstrio`)	SDK (`pip install lobstrio-sdk`)
Use case	Terminal workflows, quick scrapes, cron jobs	Scripts, pipelines, applications
Interface	Shell commands	Python API
Output	Rich tables, progress bars, CSV files	Typed dataclass models
Async	No	Yes (`AsyncLobstrClient`)
Pagination	Manual (`--page`, `--limit`)	Auto (`client.squids.iter()`)

For programmatic access, see lobstrio-sdk.

FAQ

Where do I get an API token?

Go to Dashboard → API to find your token. It's always available there, pre-generated.

How do I use keyword-based crawlers?

Some crawlers accept keywords instead of URLs. Use the --key flag:

lobstr go google-search-scraper "pizza delivery" --key keyword

Use lobstr crawlers params <crawler> to see what parameters a crawler accepts.

Can I use short hashes for runs and tasks?

No. Run and task endpoints require the full 32-character hash. Use lobstr run ls SQUID or lobstr task ls SQUID to see full hashes. Crawlers and squids support prefix matching.

How do I pipe results to other tools?

Use --json for machine-readable output:

lobstr --json results get SQUID | jq '.[].email'
lobstr --json crawlers ls | jq '.[] | .name'

Can I run scrapes in the background?

Yes. Use --no-download with go, or start a run without --wait:

lobstr go google-maps-leads-scraper urls.txt --no-download
lobstr run start SQUID  # returns immediately
lobstr run watch RUN_HASH  # check progress later

Releasing

Publishing to PyPI (the lobstrio package) is triggered by a GitHub Release, not by merging to main. Merging alone does not publish. To cut a release:

Bump version in pyproject.toml (e.g. 0.5.1).
Add a matching entry to CHANGELOG.md.
Merge to main (via PR; CI must be green).

Create a GitHub Release tagged vX.Y.Z on main:

gh release create v0.5.1 --target main --title "v0.5.1" --notes "…"

Creating the release fires .github/workflows/publish.yml (on: release: published), which builds with uv and publishes to PyPI using the PYPI_TOKEN repo secret. Confirm it went out: pip index versions lobstrio (or check the PyPI page). Note: a PyPI version cannot be overwritten — bump the version for every release.

Contributing

Contributions are welcome! See CONTRIBUTING.md for development setup, code style, and versioning guidelines.

Changelog

See CHANGELOG.md for release history.

License

Apache 2.0

Metadata

Release files for lobstrio 0.7.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for lobstrio 0.7.0
File	Size	Uploaded	Provenance	Actions
lobstrio-0.7.0.tar.gz	588.3 kB	Sep 15, 2026	Without provenance	Details for lobstrio-0.7.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for lobstrio 0.7.0
File	Interpreter	ABI	Platform	Actions Reset
lobstrio-0.7.0-py3-none-any.whl 37.7 kB Sep 15, 2026 Without provenance	Python 3	none	any	Details for lobstrio-0.7.0-py3-none-any.whl

Total release size: 626.0 kB

Release files / lobstrio-0.7.0.tar.gz

Download URL	lobstrio-0.7.0.tar.gz
Size	588.3 kB
Tags	Source
SHA-256 checksum How to use checksums	`aa151cc4f140415819da8074b65e4edbdf4354c13814c3028575714abcfb0feb`
BLAKE2b-256 checksum How to use checksums	`368386923f6496f6dc486eef9898866f1b2a070e8239a592e7789d947d6c1ace`
Upload date	Sep 15, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.14 {"installer":{"name":"uv","version":"0.12.14","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Release files / lobstrio-0.7.0-py3-none-any.whl

Download URL	lobstrio-0.7.0-py3-none-any.whl
Size	37.7 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`2708fab40d74e46add8d1f33c26a5575cf415bd46fca3b4cf5daaa2c18615867`
BLAKE2b-256 checksum How to use checksums	`e3c8d4587e18f4f905bd4c66558fabe557b0763eb1f81c1aefae8f390d4b74f2`
Upload date	Sep 15, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.14 {"installer":{"name":"uv","version":"0.12.14","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Security for lobstrio 0.7.0

Report project as malware

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

0.7.2

Sep 28, 2026 2 release files No provenance

0.7.1

Sep 16, 2026 2 release files No provenance

This release

0.7.0 This release

Sep 15, 2026 2 release files No provenance

lobstrio 0.7.0