Skip to main content

DataImpulse Scraper (MCP)

Give your AI agent — Claude Code, Cursor, Codex, Claude Desktop, … — the power to scrape the web from any country, through DataImpulse residential proxies, with full JavaScript rendering to clean, LLM‑ready markdown.

The only thing you configure is your DataImpulse login + password. No clone, no Python setup, no .env.

🟢 Get your DataImpulse residential proxy plan → — residential proxies from $1/GB, pay‑as‑you‑go, no expiry.


Install

Grab your credentials first: open your DataImpulse dashboard → → Residential Proxy → Proxy Access → copy your Login and Password. Then pick a lane:

⚡ Option 1 — one command (macOS / Linux)

curl -LsSf https://raw.githubusercontent.com/sonnysangha/dataimpulse-scraper/main/install.sh | sh

That's the whole install. The script sets up uv if you don't have it, asks for your DataImpulse login + password, verifies them with a live proxy check, and auto‑configures Claude Code, Cursor, and Claude Desktop — whichever you have. (Non‑interactive? DI_USER=... DI_PASS=... sh install.sh)

🖱️ Option 2 — one click (Cursor)

Install MCP Server

Click, approve, then replace the two YOUR_DATAIMPULSE_* placeholders with your login and password. Done. (Needs uv installed — see Option 4, step 1.)

📦 Option 3 — double‑click (Claude Desktop)

Download dataimpulse-scraper.mcpb, double‑click it (or drag it onto Claude Desktop), and type your login + password into the form it shows you — the password is stored in your OS keychain, not a file.

🛠️ Option 4 — manual (any client, incl. Windows)

1. Install uv — the tiny, fast runner that launches the server and auto‑downloads everything else:

# macOS / Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Windows (PowerShell)
powershell -c "irm https://astral.sh/uv/install.ps1 | iex"

2. Add the server to your AI app:

Claude Code — one command:

claude mcp add dataimpulse-scraper \
  --env DI_USER=YOUR_LOGIN \
  --env DI_PASS=YOUR_PASSWORD \
  -- uvx dataimpulse-scraper

Cursor / Claude Desktop / Codex / anything else — paste into your MCP config (e.g. .cursor/mcp.json):

{
  "mcpServers": {
    "dataimpulse-scraper": {
      "command": "uvx",
      "args": ["dataimpulse-scraper"],
      "env": {
        "DI_USER": "your_dataimpulse_login",
        "DI_PASS": "your_dataimpulse_password"
      }
    }
  }
}

Then: restart your AI app and ask

MCP servers load at startup (in Claude Code, /mcp should show dataimpulse-scraper connected). Then just ask:

🏠 "Read https://www.zillow.com/homes/for_sale/ and list the first 5 homes."

🌏 "Check our exit IP from Japan."

🆚 "Compare what's trending on Reddit in the US, UK, and Japan."

That's it. 🎉 The first read_page downloads a headless browser (~1 min, one‑time, auto‑cached) — every run after is instant.

📓 More copy‑paste recipes with real output in EXAMPLES.md.


The tools your agent gets

Tool What it does When to use
read_page(url, country) Primary. Real browser → clean markdown (Crawl4AI) Any page, especially JS‑heavy / SPAs
read_page_from_regions(url, regions) The same page from many countries at once → {region: markdown} Compare prices / stock / content by region
fetch_html(url, country) Raw HTML / JSON, fast (no browser) Static pages, APIs
check_proxy(country) Exit IP + geolocation Prove the proxy / geo works

Country codes are 2‑letter ISO (us, de, jp, gb, …). Every request routes through a fresh residential IP in that country — just ask in plain language ("read this as a German visitor") and the agent picks the right tool.

What works vs what needs a login

read_page fetches public pages like a real browser. It does not log in or bypass auth walls.

✅ Works (public) ❌ Needs a login
Reddit, YouTube, Bluesky, Mastodon, news, e‑commerce, SERPs X/Twitter, Instagram, Facebook, LinkedIn

A residential proxy beats IP‑based blocking and geo‑walls — not authentication. For login‑gated platforms, use their official API.

Notes

  • Country targeting only — city/state/ZIP filters bill at 2×, so stay country‑level.
  • Each call gets a fresh residential IP; change country for geo or to dodge IP rate‑limits.
  • If the headless browser ever fails to auto‑install: uvx --from dataimpulse-scraper playwright install chromium
  • Respect each site's Terms of Service and robots.txt. Scrape public data responsibly.
  • Security: credentials live in your MCP client's env block. Never commit proxy passwords — rotate them in the dashboard if exposed.

Local development (running from this folder)

Until the package is on PyPI — or when hacking on your own fork — run it straight from a checkout:

git clone https://github.com/sonnysangha/dataimpulse-scraper
cd dataimpulse-scraper
uv sync                                   # creates .venv, installs deps
cp .env.example .env                      # local dev only — add DI_USER / DI_PASS
uv run dataimpulse-scraper --selftest us  # prints a US exit IP → you're live

Then point your MCP config at the checkout instead of PyPI:

{
  "mcpServers": {
    "dataimpulse-scraper": {
      "command": "uvx",
      "args": [
        "--from",
        "/absolute/path/to/dataimpulse-scraper",
        "dataimpulse-scraper"
      ],
      "env": { "DI_USER": "your_login", "DI_PASS": "your_password" }
    }
  }
}

.env is only for local development (it's gitignored). End users never create one.

Maintainer? Publishing to PyPI (GitHub Action, trusted publishing, release loop) is covered step‑by‑step in PUBLISHING.md.


Built with DataImpulse residential proxies. Get a plan — $1/GB →

Metadata

Release files for dataimpulse-scraper 1.0.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dataimpulse-scraper 1.0.1
File Size Uploaded Provenance Actions
dataimpulse_scraper-1.0.1.tar.gz 319.2 kB Without provenance Details for dataimpulse_scraper-1.0.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for dataimpulse-scraper 1.0.1
File Interpreter ABI Platform Actions
dataimpulse_scraper-1.0.1-py3-none-any.whl Python 3 none any Details for dataimpulse_scraper-1.0.1-py3-none-any.whl

Total release size: 329.7 kB

Release files / dataimpulse_scraper-1.0.1.tar.gz

Download URL dataimpulse_scraper-1.0.1.tar.gz
Size 319.2 kB
Tags Source
SHA-256 checksum
How to use checksums
ead43ca6d92cf167c51ea983b5be118d5c9c4297b62493c82506f20384cb0506
BLAKE2b-256 checksum
How to use checksums
7f7d8e450b387617eee4cf85a5d7906c80a4ebea236d8fff74bbf6b474c6dac3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / dataimpulse_scraper-1.0.1-py3-none-any.whl

Download URL dataimpulse_scraper-1.0.1-py3-none-any.whl
Size 10.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
dea71f2c1ea946246d3d8d3c44f6d6816903d94bb76c3e07e41865b2c6b01669
BLAKE2b-256 checksum
How to use checksums
313e5f517dcb561aecf1d0472e453754fdd60ed2e4fb5cd18eb8dc3a492020c2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.11.19 {"installer":{"name":"uv","version":"0.11.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for dataimpulse-scraper 1.0.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.0.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page