Web intelligence that only thinks when the web changes. Bright Data + LLM agent cache — zero tokens on unchanged pages.
Project description
Indra fetches live web data through Bright Data on every run — no stale snapshots, no geo-blocks, no bot detection failures. It stores a fingerprint of what it saw. Next run, if the page hasn't changed, the LLM is skipped entirely. If it has, the LLM sees only the diff.
Run 1: fetch 10 pages via Bright Data → analyse all 10 → cache insights
Run 2: fetch 10 pages via Bright Data → 8 unchanged → 2 changed → LLM fires twice
Run 3: fetch 10 pages via Bright Data → 9 unchanged → 1 changed → LLM fires once
Over 24 hours of hourly checks across 10 pages: 240 Bright Data fetches, ~20 LLM calls instead of 240.
pip install indra-ai
export BRIGHTDATA_API_KEY="your-key"
python examples/competitor_monitor_demo.py
How it works
Every monitored URL goes through three steps on each run:
1. Fetch live via Bright Data Bright Data's Web Unlocker bypasses bot detection, CAPTCHAs, and geo-blocks. Every check uses fresh data — there is no local cache of the web content itself.
2. Fingerprint and compare Indra hashes the response and compares it to the last stored hash. This is instant and costs nothing.
3. LLM only on change
- No change → return the cached insight. Zero tokens, sub-millisecond.
- Changed → extract the diff, send only the delta to your LLM. Tokens proportional to what changed, not the full page.
import indra
agent = indra.init(brightdata_api_key="your-key")
result = agent.watch(
url="https://competitor.com/pricing",
question="Did any prices change? What are the implications?",
generation_fn=my_llm_call, # only called when the page actually changed
)
print(result.changed) # True / False
print(result.insight) # LLM analysis, or cached answer if unchanged
print(result.diff) # what changed (empty if no change)
print(result.tokens_saved) # tokens skipped this run
The demo
examples/competitor_monitor_demo.py watches 5 real pages across 3 rounds:
Round 1 — Baseline (first run)
unchanged · openai.com/api/pricing/ .............. first run
unchanged · anthropic.com/pricing ................ first run
unchanged · news.ycombinator.com/ ................ first run
unchanged · techcrunch.com/ai/ ................... first run
unchanged · github.com/trending .................. first run
Round 2 — Incremental check
unchanged · openai.com/api/pricing/ .............. saved 1500 tokens
unchanged · anthropic.com/pricing ................ saved 1500 tokens
CHANGED ↑ · news.ycombinator.com/ ............... LLM fired · saved 1200 tokens on diff
unchanged · techcrunch.com/ai/ ................... saved 1500 tokens
unchanged · github.com/trending .................. saved 1500 tokens
──────────────────────────────────────────────────
Indra Session Summary
──────────────────────────────────────────────────
Bright Data fetches : 15
Changes detected : 1
LLM calls fired : 6
Cache hits : 9
Tokens saved : 12,000
Cost saved : $0.0360
Efficiency : 80%
──────────────────────────────────────────────────
Install
pip install indra-ai
# required
export BRIGHTDATA_API_KEY="your-bright-data-api-key"
# optional — enables LLM analysis on changes
export ANTHROPIC_API_KEY="your-anthropic-key"
Get $250 in Bright Data credits at brightdata.com — enough to run thousands of monitored pages.
API
indra.init()
agent = indra.init(
brightdata_api_key="...", # or set BRIGHTDATA_API_KEY env var
db_path="indra.db", # where snapshots are stored
silent=False, # suppress per-URL console output
)
agent.watch(url, question, generation_fn)
Watch a single URL. Returns a WatchResult.
result = agent.watch(
url="https://example.com",
question="What changed and why does it matter?",
generation_fn=my_llm_fn, # fn(prompt: str) -> str
render_js=False, # True for JS-heavy pages (Bright Data headless)
ttl=3600, # skip Bright Data fetch if snapshot < ttl seconds old
)
agent.watch_all(urls, question, generation_fn)
Watch multiple URLs in one call.
results = agent.watch_all(
urls=["https://site1.com", "https://site2.com"],
question="Any significant changes?",
generation_fn=my_llm_fn,
)
agent.search_watch(query, question, generation_fn)
Watch live SERP results for a query. Fires LLM only when the result set changes.
result = agent.search_watch(
query="openai new model announcement",
question="Is there a major new release?",
generation_fn=my_llm_fn,
)
WatchResult
| Field | Type | Description |
|---|---|---|
changed |
bool | Whether content changed since last run |
insight |
str | LLM analysis (or cached answer if unchanged) |
diff |
str | Unified diff of what changed |
tokens_saved |
int | Tokens skipped this run |
cost_saved_usd |
float | Dollar value of skipped tokens |
latency_ms |
float | Total time for this watch call |
brightdata_called |
bool | Whether Bright Data was queried |
change_count |
int | Total times this URL has changed |
summary |
str | Human-readable change summary |
agent.stats() / agent.print_stats()
agent.print_stats()
# ──────────────────────────────────────────────────
# Indra Session Summary
# ──────────────────────────────────────────────────
# Bright Data fetches : 24
# Changes detected : 3
# LLM calls fired : 3
# Cache hits : 21
# Tokens saved : 31,500
# Cost saved : $0.0945
# Efficiency : 87%
# ──────────────────────────────────────────────────
Use cases
Competitor pricing monitor — check 20 competitor pages every hour. LLM summarises only when a price changes.
News and signal tracker — watch industry news sites. Alert only when genuinely new stories appear, not every hourly check.
Supply chain watcher — monitor supplier pages for stock or lead time changes. Zero noise on stable days.
Regulatory tracker — watch government or compliance pages. LLM fires when policy text changes; silent otherwise.
SEO and ranking monitor — SERP watch for branded or competitive queries. Analyse only when rankings shift.
Architecture
┌─────────────────────────────────────────────────┐
│ Your agent / script │
└────────────────────┬────────────────────────────┘
│ agent.watch(url, question)
┌────────────────────▼────────────────────────────┐
│ Indra │
│ │
│ 1. Fetch via Bright Data Web Unlocker │
│ (bypasses bot detection, geo-blocks) │
│ │
│ 2. Fingerprint content (SHA-256) │
│ Compare to stored hash in SQLite │
│ │
│ 3a. No change → return cached insight │
│ 0 tokens · sub-millisecond │
│ │
│ 3b. Changed → extract diff → LLM(diff only) │
│ tokens ∝ what changed, not page size │
└─────────────────────────────────────────────────┘
Indra is built on Mnemon — an execution cache for LLM agents.
Why Bright Data
Standard web fetching breaks on modern sites: JavaScript rendering, bot detection, CAPTCHAs, geo-restrictions. Monitoring agents that hit these walls silently return stale or empty content — and the LLM never knows.
Bright Data solves all of this transparently. Every agent.watch() call reaches the live page regardless of what protection it has. The change detection layer is only useful if the data underneath is actually fresh — Bright Data guarantees that.
License
MIT — free to use and build on.
Built for the Web Data UNLOCKED Hackathon by Mahika Jadhav.
Questions: mahikajadhav22@gmail.com
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file indra_ai-1.0.0.tar.gz.
File metadata
- Download URL: indra_ai-1.0.0.tar.gz
- Upload date:
- Size: 139.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
caa3033b5f9d5ae3f12f629e25d1b5e0d0b2c93b3d157a20b26e1ebb45380736
|
|
| MD5 |
a6d9401f59730a76208429a36c54cdcd
|
|
| BLAKE2b-256 |
09b33c6cc4114e66f5ee4c3655ec6af76238e7c8971652bc78fd3c11a7077e73
|
File details
Details for the file indra_ai-1.0.0-py3-none-any.whl.
File metadata
- Download URL: indra_ai-1.0.0-py3-none-any.whl
- Upload date:
- Size: 154.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
0dced12ed5b8dcc81cc392e515c6793d1381c79493d219b999b14dbee42d6ef1
|
|
| MD5 |
902f66c30dd26ea97c2f1b72cbb862e6
|
|
| BLAKE2b-256 |
34859fa7b8572cfa2e5182f5120047e182c853445185d8e7f424d1abf104cc5e
|