pypi-tea

A Transparency Exchange API (TEA) server that extracts and serves SBOMs from PyPI packages.

Python wheels can include SBOMs in .dist-info/sboms/ (PEP 770). pypi-tea makes these discoverable through the TEA protocol — give it a PURL like pkg:pypi/requests@2.31.0 and it returns any SBOMs found in the package's wheel files.

How it works

1. A TEA client queries with a TEI or PURL identifier.
2. If the release is already indexed, the answer comes straight from the local SQLite store.
3. If not, the API queues it and waits briefly while the worker:
   • fetches the release from the PyPI JSON API
   • reads each wheel's ZIP central directory and .dist-info/sboms/ members with HTTP range requests (a few KB per wheel, not the whole file)
   • detects and validates each SBOM's format, and checks PEP 740 attestations
4. The result is stored permanently: wheels are immutable, so a wheel is never fetched twice.

Architecture

graph LR
    Client -->|TEI / PURL| API["pypi-tea-serve<br/>(gunicorn + uvicorn workers)"]
    Crawler["Crawlers"] -->|HTML pages, sitemap| API
    API <-->|reads, enqueue| DB[("SQLite (WAL)")]
    Worker["pypi-tea-worker"] <-->|jobs, results| DB
    Worker -->|JSON API, range requests| PyPI

The service is split so that traffic can never turn into upstream work it can't afford:

• The API process never talks to PyPI. Every page and TEA response is built from indexed SQLite lookups. Crawlers fetching package pages, listings, the sitemap or SBOM downloads cost a millisecond each, however many arrive at once.
• All upstream I/O happens in the worker, at a fixed concurrency. A burst of lookups for unknown releases makes the queue longer; it does not multiply outbound connections, threads or memory.
• On-demand lookups are bounded. A lookup for an unknown release waits at most PYPI_TEA_RESOLVE_WAIT_SECONDS. Past that, or once PYPI_TEA_MAX_PENDING_LOOKUPS lookups are queued, the API answers 503 with Retry-After instead of piling up.
• Every request has a deadline (PYPI_TEA_REQUEST_TIMEOUT_SECONDS), and each API process caps open connections (PYPI_TEA_MAX_CONNECTIONS_PER_WORKER). Over either limit it sheds load with a fast 503 rather than stalling.
• Responses are edge-cacheable, with stale-if-error, so Cloudflare can keep serving pages through an origin restart.

Storage

A single SQLite database in WAL mode, so any number of API processes read while the worker writes.

Nothing expires. The worker re-checks every known project for new releases once a week; re-resolving a known release costs one PyPI JSON request, because already-inspected wheels are reused.

Statistics

GET / is a live dashboard; GET /stats, /stats/timeseries, /stats/usage and /stats/invalid-sboms return the raw JSON. They are computed from the tables above and cached for 60 s per process.

Quick start

Prerequisites

• Python 3.14+
• uv

Install and run

git clone https://github.com/sbomify/pypi-tea.git
cd pypi-tea
uv sync
# API and worker in one process, for development:
PYPI_TEA_EMBEDDED_WORKER=true uv run uvicorn pypi_tea.app:app

The server starts at http://localhost:8000 and writes pypi-tea.db in the working directory.

In production, run the two halves separately:

pypi-tea-serve      # API: one worker process per CPU
pypi-tea-worker     # ingest worker

Configuration

All settings are configurable via environment variables with the PYPI_TEA_ prefix:

Analytics

pypi-tea sends usage events to PostHog so we can see how the service is used. Nobody signs in, so events are anonymous and create no person profiles. Server-side, a client's id is a keyed hash of its IP and User-Agent that changes daily. Neither value is sent; only the client family and version parsed from the User-Agent (libtea, curl, browser, ...) are.

HTML pages and SBOM downloads are mostly answered by the edge cache, so their server-side events only count cache misses. The browser events cover what visitors do on the site. To tag a new element, give it data-ph-event="name" and a data-ph-<property> attribute for each property.

Error tracking uses Sentry's own variables and applies to the API, the worker and pypi-tea-refresh alike (events carry a component tag):

Queue maintenance

pypi-tea-refresh --status            # queue depth per priority
pypi-tea-refresh                     # check every known project for a new release now
pypi-tea-refresh --existing          # ...and re-resolve every known release

Migrating from 0.x (Redis)

Since 26.10.0 (the switch to CalVer), pypi-tea stores everything in SQLite and no longer uses Redis. To carry over the set of known releases, queue them from the old keyspace; the worker re-resolves them, releases with SBOMs first:

uvx --from 'pypi-tea[import-redis]' pypi-tea-refresh --import-redis redis://localhost:6379/0

While a release is still queued, its package page answers 503 with Retry-After, so search engines retry rather than drop it. TEA UUIDs are deterministic and unchanged, except for projects with a . in their name: names are now normalised as PyPI does (PEP 503), so zope.interface and zope-interface are one product, zope-interface, and old /package/zope.interface/… URLs redirect there.

TEA endpoints

Non-TEA endpoints

Example

Discover SBOMs for a package:

curl "http://localhost:8000/discovery?tei=urn:tei:purl:localhost:pkg:pypi/cyclonedx-python-lib@8.4.0"

Search by PURL:

curl "http://localhost:8000/products?idType=PURL&idValue=pkg:pypi/cyclonedx-python-lib@8.4.0"

Data model

All UUIDs are deterministic (UUID v5 with a fixed namespace) so they're stable across requests and server restarts.

Deployment

A systemd setup is included in deploy/:

sudo ./deploy/setup.sh 26.10.1    # installs into /opt/pypi-tea, enables both services
sudoedit /etc/default/pypi-tea   # PYPI_TEA_SERVER_ROOT_URL=https://tea.example.com
sudo systemctl restart pypi-tea pypi-tea-worker

The services run from a fixed virtualenv, so restarts don't depend on PyPI being reachable. The database lives in /var/lib/pypi-tea/.

Logs:

journalctl -u pypi-tea -f
journalctl -u pypi-tea-worker -f

Development

# Install dev dependencies
uv sync --group dev

# Run tests (no services needed; the conformance suite reaches live PyPI)
uv run pytest

# Lint
uv run ruff check src/ tests/

# Format
uv run ruff format src/ tests/

# Type check
uv run mypy src/

Conformance

The test suite runs libtea's conformance checks against an in-process server. 21 of 26 checks pass; 5 CLE (Collection Lifecycle Event) checks are skipped as CLE is not implemented.

Versioning

Versions are CalVer, YY.M.PATCH, matching the other sbomify projects: the two-digit year and month of the release, then a counter that restarts at 0 each month. Releases are tagged vYY.M.PATCH (for example v26.10.0). 26.10.0 follows 0.6.2, so version ordering still holds for pip and uv.

License

Apache-2.0