promptbeacon 1.2.0

The open-source GEO engine to measure, track, and CI-test how AI (ChatGPT, Claude, Gemini) recommends your brand

Does AI recommend your brand?
The open-source engine to measure, track, and CI-test your visibility across ChatGPT, Claude, Gemini, Mistral & more. pip install, zero keys to start.

📖 Documentation  ·  Quickstart  ·  Examples  ·  API Reference  ·  PyPI

---

People used to Google "best running shoes" and click a link. Now they ask ChatGPT — and get one synthesized answer with no click. If the AI doesn't mention you, you're invisible. PromptBeacon measures whether it does, with the statistical rigor a real monitoring pipeline needs — and you can run it in your terminal or your CI in 60 seconds.

Try it now — no API keys

pip install promptbeacon
promptbeacon demo "Nike"

demo runs against a realistic offline mock, so you see exactly what a real scan produces without spending a cent. When you're ready, add an API key and drop --demo.

from promptbeacon import Beacon

# Keyless: works the moment you install
report = Beacon("Nike").demo().with_competitors("Adidas", "Puma").scan()

print(f"Visibility:      {report.visibility_score}/100")
print(f"Share of Voice:  {report.share_of_voice.target_share:.0%} (rank {report.share_of_voice.target_rank})")

[!TIP] Liked that? The documentation covers Share of Voice, stability scoring, smart mode, and wiring PromptBeacon into your CI — every example runs keyless. New here → Quickstart.

Why PromptBeacon

The AI-visibility (GEO / AEO) space is dominated by $29–490/month SaaS dashboards (Profound, Peec, Otterly…). They're built for marketers to look at. PromptBeacon is built for developers and agencies to build on — the open-source measurement engine you can script, schedule, embed in a product, or gate a deploy with.

	SaaS dashboards	PromptBeacon
Price	$29–490+/mo, per seat	Free, Apache-2.0
Where your data lives	Their cloud	Your machine (local-first)
Try without paying	Trial / credit card	pip install → keyless demo
Programmable	Limited API	It's a Python library
Reproducibility	One number	Confidence intervals + stability
CI / regression testing	✗	pytest plugin + GitHub Action
Providers in one run	Tier-gated	6, simultaneously
Measures live AI search	Opaque / varies	Web-grounded, with real citations
Funnel-level visibility	✗ (final citations only)	Glass-box: where you drop out

Who it's for

• Indie devs & technical founders — "does ChatGPT recommend my product?", answered in code.
• GEO/SEO agencies & consultants — one engine, every client, build your own dashboards on top.
• AI / eval engineers — track brand visibility as a CI check next to your other evals.

The three things that make it rigorous

1. Share of Voice — the metric everyone wants

Of all the brand presence across your prompt set (you + competitors), what fraction is yours?

report = Beacon("Nike").demo().with_competitors("Adidas", "Puma").scan()
sov = report.share_of_voice
print(sov.target_share)        # 0.34  (34% share of voice)
print(sov.target_presence_rate)# 0.88  (appears in 88% of prompts)
print(sov.target_rank)         # 2     (rank by appearances)

2. Stability — don't trust a single answer

LLM answers are probabilistic: in the wild, only ~30% of brands stay visible from one answer to the next. PromptBeacon repeats each prompt N times and tells you how much to trust the number — a 0–100 stability score, a confidence interval, and which prompts flip-flop.

report = Beacon("Nike").demo().with_stability(5).scan_stability()
s = report.stability
print(s.stability_score)            # 78.5  (higher = more trustworthy)
print(s.score_confidence_interval)  # (61.0, 84.0)
print(s.flip_flop_count)            # prompts that appeared in some runs but not others

3. CI-native — gate your deploys on AI visibility

No other tool lets you fail a build when AI stops recommending you.

# In code
Beacon("Nike").scan().assert_visibility(min_score=50, min_share_of_voice=0.3)

# As a pytest check (plugin auto-registers; skips cleanly without keys)
import pytest

@pytest.mark.visibility(brand="Nike", competitors=["Adidas"], min_score=40)
def test_brand_is_visible():
    ...

# As a GitHub Action
- uses: yotambraun/promptbeacon@v1
  with:
    brand: "Nike"
    competitors: "Adidas Puma"
    min-share-of-voice: "0.3"
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

Measure what users actually see — not just model memory

A plain LLM call reflects the model's training memory. Real users get web-grounded answers — the engine searches the live web and cites sources. PromptBeacon measures that, and is honest about which is which.

# Real web-grounded scan: provider web search + the real cited sources
promptbeacon scan "Nike" -c "Adidas" --grounded -p openai -p anthropic

--grounded uses each provider's native web search via its official SDK — OpenAI, Anthropic, Gemini, and Perplexity — and captures the real citations (Mistral/Cohere fall back to base completion). Every report carries an honest measurement_tier (demo / base_model / api_grounded) so training-memory is never mistaken for live AI search. Install with pip install 'promptbeacon[grounded]'.

Source attribution — which sites feed your visibility

Grounded answers cite their sources. PromptBeacon ranks the domains the engines trust for your category and flags which cite you — the actionable GEO lever ("get cited on these sites").

promptbeacon sources "Nike" --competitor "Adidas" --demo

Glass-box funnel — see where you drop out

Modern AI search is agentic: it fans your query into 8–12 sub-queries, retrieves, reranks, then cites. Citation trackers see only the survivors. PromptBeacon runs an observable model of that funnel and shows where your brand drops out:

promptbeacon funnel "Nike" --category "running shoes" --demo

measurement: funnel_model
Coverage (brand retrieved):  88%
Rerank survival:             86%
Retrieval → citation:        29%     ← retrieved often, cited rarely
Dominant drop-off stage:     citation

For live web search, use a Tavily key (TAVILY_API_KEY, free at tavily.com) and drop --demo; add --smart for an LLM planner + LLM-judge reranker. No $29–490/mo dashboard shows you this.

Shareable dashboard (no SaaS)

promptbeacon dashboard "Nike" --competitor "Adidas" --demo

Writes a single, self-contained HTML file — Share-of-Voice bar, score breakdown, sentiment donut, stability band — that you can hand to a stakeholder. No server, no subscription. (sample)

Real scans (with keys)

export OPENAI_API_KEY="sk-..."          # https://platform.openai.com/api-keys
export ANTHROPIC_API_KEY="sk-ant-..."   # https://console.anthropic.com/settings/keys
promptbeacon providers                   # check what's configured

Keys can live in your environment or a .env file in the project (auto-loaded). TAVILY_API_KEY powers promptbeacon funnel's live web search.

from promptbeacon import Beacon, Provider

report = (
    Beacon("Nike")
    .with_aliases("Nike Inc", "Nike Corporation")       # count all name variants
    .with_competitors("Adidas", "Puma", "New Balance")
    .with_providers(Provider.OPENAI, Provider.ANTHROPIC)
    .with_industry("ecommerce")                          # industry-tuned prompts
    .with_cache()                                        # skip duplicate queries
    .with_storage("~/.promptbeacon/nike.db")             # track history over time
    .scan()
)

print(f"Score: {report.visibility_score}/100  |  SoV: {report.share_of_voice.target_share:.0%}")
for name, score in report.competitor_comparison.items():
    print(f"  {name}: {score.visibility_score:.1f}")

Smart mode — LLM accuracy + actionable advice

Regex extraction is fast and offline, but heuristic. --smart (or .with_smart_extraction()) uses a cheap LLM with structured output to read each response — catching paraphrases and nuance regex misses — and .with_smart_recommendations() turns the scan's own data into specific "why you're invisible and how to fix it" guidance. Opt-in (one extra LLM call each); falls back to regex/rule-based on any error.

promptbeacon scan "Nike" -c "Adidas" --smart

BeaconGuard: real-time brand safety (bonus)

Shipping a customer-facing AI chatbot? BeaconGuard flags when an LLM output recommends a competitor or trashes your brand — local, no API calls.

from promptbeacon import BeaconGuard

guard = BeaconGuard("Nike", competitors=["Adidas", "Puma"])
result = guard.analyze("Try Adidas instead — Nike has quality issues.")
print(result.risk_level)  # "high"

Works as middleware in any pipeline, or with LangChain (pip install 'promptbeacon[langchain]'). See Advanced Usage.

CLI

promptbeacon demo "Nike"                                  # keyless, instant
promptbeacon scan "Nike" -c "Adidas" -p openai -p anthropic
promptbeacon scan "Nike" -c "Adidas" --grounded           # real web-grounded scan
promptbeacon scan "Nike" --stability 5                    # repeat for a stability score
promptbeacon scan "Nike" --assert-min-score 50           # CI gate (exit 1 on fail)
promptbeacon scan --protocol nike.json                    # pinned, reproducible run
promptbeacon sources "Nike" --demo                        # which domains AI cites
promptbeacon funnel "Nike" -t "running shoes" --demo      # where you drop out (glass-box)
promptbeacon dashboard "Nike" --demo                      # shareable HTML
promptbeacon compare "Nike" --against "Adidas"
promptbeacon history "Nike" --days 30
promptbeacon providers

Features

Feature	Description
Keyless demo mode	pip install → realistic scan with zero API keys
Web-grounded scanning	--grounded: real provider web search + the actual cited sources (OpenAI, Anthropic, Gemini, Perplexity)
Source attribution	Rank the domains AI cites for your category — and which cite you (promptbeacon sources)
Glass-box funnel	See where your brand drops out of the agentic search funnel — retrieve → rerank → cite (promptbeacon funnel)
Measurement tiers	Honest demo / base_model / api_grounded label on every scan
Reproducible protocols	Pin a scan in JSON for comparable CI runs (scan --protocol)
Smart mode (LLM)	--smart swaps regex for LLM extraction + evidence-linked, actionable recommendations
Share of Voice	Presence-based SoV vs competitors, per-provider + aggregate + rank
Stability scoring	Repeat-N-times trust score, confidence interval, flip-flop detection
CI-native	assert_visibility(), pytest plugin, GitHub Action
HTML dashboard	Single-file, shareable, no SaaS
6 LLM Providers	OpenAI, Anthropic, Google, Mistral, Cohere, Perplexity — queried together
Citation Tracking	Which sources LLMs cite when discussing your brand
Brand Aliases	"Nike Inc", "Nike Corporation" all count as Nike
Industry Templates	ecommerce, SaaS, finance, healthcare, travel, food, tech
Historical Tracking	DuckDB-powered local storage for trends
Score Breakdown	See which of 4 factors drives your score
5 Export Formats	JSON, CSV, Markdown, HTML, pandas DataFrame
BeaconGuard	Real-time brand-safety guard for LLM outputs
Local-First	Your data stays on your machine — no cloud, no subscription

Supported Providers

Provider	Default Model	Env Variable
OpenAI	gpt-4o-mini	OPENAI_API_KEY
Anthropic	claude-haiku-4-5	ANTHROPIC_API_KEY
Google	gemini-2.0-flash	GOOGLE_API_KEY
Mistral	mistral-small-latest	MISTRAL_API_KEY
Cohere	command-r	COHERE_API_KEY
Perplexity	sonar	PERPLEXITY_API_KEY

Documentation

📖 Full docs & guides →

• Quickstart — up and running in 5 minutes (keyless)
• Share of Voice & Stability — the rigor features
• CI & pytest plugin — gate deploys on AI visibility
• API Reference · Providers · Storage

Development

git clone https://github.com/yotambraun/promptbeacon
cd promptbeacon
uv venv && uv sync --all-extras

uv run pytest --cov -v        # tests
uv run ruff check .           # lint
uv run ruff format .          # format

Contributing

Contributions welcome! See TODO.md for the roadmap.

License

Apache License 2.0 — see LICENSE.

Acknowledgements

Built with LiteLLM, Pydantic, DuckDB, Typer, and Rich.