TEA Data Engine (teadata)
teadata is a snapshot-first Python engine for Texas education data.
It provides:
DistrictandCampusdomain models- a fluent query DSL using
>> - geospatial lookups (nearest charter, campuses in district boundaries, private school overlap)
- config-driven enrichment from TAPR, accountability, transfers, PEIMS financials, and closure datasets
- sidecar sqlite stores for fast boundary/map/entity lookup
Installation
Python 3.11 through 3.15 are supported. Development and local checks default to
the exact Python version in .python-version (currently 3.14.10).
TEADATA_CHECK_PYTHON overrides that default for compatibility checks. The lock
retains interpreter-specific versions where dependency compatibility requires
them. On macOS 3.15, the optional documentation tools build
Watchdog's native FSEvents extension and require the
documented Xcode tools.
PyYAML's supported pure Python implementation is tested on 3.15.
PyPI
uv venv
uv pip install teadata==0.0.132
Development (recommended)
GIT_LFS_SKIP_SMUDGE=1 git clone https://github.com/adpena/teadata.git
cd teadata
uv sync --extra dev --extra database --extra scripts --extra speedups
Source datasets are separate from the checkout; a verified release bundle is enough
for runtime use. Maintenance scripts default to teadata/data/ beside the
installed package, independent of the current working directory. Set
TEADATA_DATA_DIR to use another dataset tree, or pass the scripts' explicit
input/output options. See Configuration & Resolution
for examples and onboarding details.
Dependency security
pyproject.toml publishes security floors for the core and each optional extra;
uv.lock retains the reviewed versions across supported interpreters. Validate
the selected environment with scripts/check; changing the lock also requires
a new producer-bound data release before publishing replacement artifacts.
The notebook extra uses Bleach 6.4 for its published fixes. Its opt-in
parse_email=True linkifier still has a known
regex denial-of-service issue.
TEA Data does not enable that path, and its nbconvert dependency uses Bleach's
HTML/CSS sanitization. The upstream limitation remains relevant to notebook
code that explicitly enables email linkification; upgrading Bleach alone does
not resolve it.
Existing relational stores
Publisher rating categories are stored as TEXT, including full labels such as
Not Rated: Minimum Size rules. Existing PostgreSQL stores with VARCHAR(16)
rating columns need a caller-owned schema migration widening both
districts.rating and campuses.rating to TEXT, or a fresh destination.
ensure_schema() creates missing tables; it does not alter existing columns.
TEA Data does not migrate user databases automatically or truncate these labels.
Exports leave transaction commit and rollback to the caller.
Campus identity and affiliation
Campus.lookup_key and DataEngine.campus_by_key() use nine-digit TEA campus
numbers for public schools and source-qualified keys such as
tefa:rec1kXXvIucG7UkoA for private schools. Private source IDs retain their exact
case and prefixes; URI-component encoding preserves other characters. The
shared identifiers helper validates the encoded ASCII key's 1024-byte limit.
campus_number is null for private schools. Their source_namespace and
source_id are explicit fields, and their governing district_id and
district_number are null. A directory's reported district is retained separately
as an unverified directory claim, never as governance.
Public campus UUIDs, numbers, and URLs remain unchanged. Private UUIDs derive
from their source-qualified identity. Existing entity, map, and snapshot bundles
using fabricated private TEA numbers require a source-bound rebuild; no numeric
alias is provided. Analytical schema 2 uses the exact campus lookup key as
entity_id, with separate nullable TEA-number and source-identity attributes.
Existing relational destinations also need the caller-owned identity-schema
migration or a fresh destination; ensure_schema() does not alter old tables.
Private comparisons use private_campuses_within(), which includes points inside
or on district boundaries. The former private district-membership operator is
retired. Private profile maps report every covering traditional public district
(INDEPENDENT or COMMON) and the boundary dataset/version; overlaps are not
collapsed to a chosen parent. Charter service areas are not attendance-boundary
comparators. Schools without coordinates remain queryable, with no geographic
comparison or guessed district.
The optional native PostgreSQL regression uses TEADATA_TEST_POSTGRES_URL
pointing to a fresh empty teadata_test_* database. It is separate from the
default SQLite checks because SQLite does not enforce declared VARCHAR widths.
Quick Start
Python packages contain code and metadata; data bundles are distributed separately. Select an immutable bundle before running the example:
export TEADATA_RELEASE_URL="https://github.com/adpena/teadata/releases/download/v0.0.128/release.json"
export TEADATA_RELEASE_SHA256="9dd062128273cbb90f3b9c952b072d74a629de6ecbcff8065c4295b511eaa9aa"
For a downloaded bundle, set TEADATA_RELEASE to its local release.json
instead. The manifest binds the selected artifacts to their content hashes.
Use the TAPR and TPRS supplements
to access separately versioned assessment, attendance, dropout, and student profile tables.
from teadata import DataEngine
# Load the configured, hash-verified release.
engine = DataEngine.load_default()
# Look up a six-digit TEA district number.
aldine = (engine >> ("district", "101902")).first()
if aldine is not None:
print(aldine.name)
# Campuses assigned to this district.
for campus in aldine.campuses[:5]:
print(campus.name, campus.campus_number)
The district operation returns a Query. .first() selects its first match or
returns None; iterate the query to inspect every match when a name or wildcard
can identify multiple districts.
DataEngine() creates an empty engine by default (autoload=False).
load_default() and from_snapshot() without a path use the configured release:
TEADATA_RELEASE, then TEADATA_RELEASE_URL, then the package's
teadata/.cache/release.json, unless a release is already pinned by
using_release().
Public API Surface
Primary imports:
from teadata import DataEngine, District, Campus
Core behaviors:
DataEngine.load_default()loads the snapshot and boundary store selected by the configured release manifest.DataEngine.from_snapshot(path)loads a trusted local catalog-bound snapshot; omit the path to use the configured release.- One release manifest binds the snapshot and sidecars to a catalog; selected artifact hashes are checked as they are loaded.
DistrictandCampussupport dynamic metadata attributes throughmeta.Campus.to_dict()always includespercent_enrollment_change(numeric when available, otherwise"N/A").
Snapshot and Asset Behavior
teadata is intentionally cache-first.
Source definitions can include dataset-specific reference periods, publisher
details, URLs, citations, and provenance record identifiers. Use
cfg.resolve_dataset(...) to query configured metadata; use ProvenanceStore
to preserve immutable acquisition and processing history with archived file
bytes. Unknown historical download dates remain unknown. See
dataset metadata and provenance.
Artifacts typically used at runtime:
snapshot_<build-id>.pkl.gz(engine snapshot)boundaries_*.sqlite(boundary WKB sidecar)maps_<build-id>.sqlite(map payload sidecar)entities_*.sqlite(entity lookup sidecar)
The authoritative release.json binds the snapshot and sidecars to one catalog and verified hashes. Hydrate tracked assets with Git LFS, or configure an immutable remote release URL. Metadata, provenance, masking, estimates, and rebuild instructions are documented in Dataset contracts and data releases.
For academic use, cite the software and the data build
in plain text or BibTeX with teadata.citation().
Environment Variables
TEADATA_RELEASE: local release manifest.TEADATA_RELEASE_URL: immutable remote release manifest.TEADATA_RELEASE_SHA256: trusted lowercase SHA-256 of the selected release manifest.TEADATA_RELEASE_OFFLINE: set to1for cache-only access to remote release files or0to allow downloads. Offline remote manifests require a SHA-256 pin.TEADATA_DATA_DIR: source data root for scripts and configureddata/...inputs.TEADATA_CONFIG,TEADATA_PROVENANCE_DIR,TEADATA_OUTPUT_DIR: build locations.TEADATA_DISABLE_INDEXES: disable spatial acceleration indexes.TEADATA_LOG_MEMORY: log memory snapshots.
Query DSL
DataEngine and Query chains use >>.
# Resolve district then expand to district-operated campuses.
q = engine >> ("district", "ALDINE ISD") >> ("campuses_in",)
# Filter, sort, and take.
top = (
q
>> ("filter", lambda c: (c.enrollment or 0) > 1000)
>> ("sort", lambda c: c.enrollment or 0, True)
>> ("take", 10)
)
rows = top.to_df(columns=["name", "campus_number", "enrollment"])
Supported lookup semantics include:
- case-insensitive district and campus name matching
- wildcard patterns (
*,?, SQL-like%/_) - normalized district number handling (for example
"123"and"'000123")
Spatial and transfer helpers include:
- nearest-campus/nearest-charter queries
nearest_charter_same_type(...)- transfer graph methods such as
transfers_out(...)/transfers_in(...)
Enrichment Pipeline
teadata/enrichment provides registered enrichers for district and campus datasets.
Included enrichers cover:
- district accountability and district TAPR profile data
- campus accountability, TAPR profile/historical enrollment, PEIMS financials
- planned closure overlays
- charter network augmentation
See entity enrichment sources and years for the inputs in the published entity release and the editions configured for each source.
Required source, schema, methodology, and transfer-join failures stop publication. The previous release remains active when a staged build fails. Geometry coverage and pending source review remain explicit in the processing receipt.
Data Build Pipeline
teadata/load_data.py builds a full DataEngine and updates cached artifacts.
uv run python -m teadata.load_data
At a high level, it:
- resolves year-aware source paths from
teadata/teadata_sources.yaml - verifies and records selected source files and schema contracts
- builds districts/campuses from verified inputs
- applies enrichment datasets
- validates a staged snapshot and sidecars, then activates
teadata/.cache/release.json
Config and CLI (teadata-config)
teadata/teadata_config.py provides YAML/TOML config loading, year resolution, schema checks, and dataset joins.
CLI entrypoint:
uv run teadata-config --help
Subcommands:
init <out.yaml>resolve <cfg> <section> <dataset> <year>report <cfg> [--json] [--min N] [--max N]join <cfg> <year> [--datasets a,b,c] [--parquet out.parquet] [--duckdb out.duckdb --table t]
Testing
scripts/check
scripts/check --verify-latest
Local checks are the primary verification path. They run Ruff, ty, deptry, and
pytest in a locked, isolated uv environment without replacing the development
.venv. Logs and immutable per-run JSON receipts stay in ignored .local/checks/.
Verification rejects receipts after the Git revision, source, lockfile, check
policy, tool/dependency environment, or captured logs change. A commit requires
a fresh run even when it contains previously checked working-tree contents.
Focused checks supplied programmatically retain their own receipts without
replacing the full-policy pointer used by --verify-latest and the push guard.
Each Python runtime and selected release has a separate pointer. An earlier
receipt from the same check scope still needs to match the current inputs.
After rebuilding changed serialized data, also verify the retained release:
scripts/check --release teadata/.cache/release.json
scripts/check --release teadata/.cache/release.json --verify-latest
TEADATA_CHECK_PYTHON=3.11 scripts/check
Code-only receipts do not certify raw inputs, data quality, or data artifacts. Release checks verify all selected artifact hashes and bindings and reject a manifest built from different processing code or lockfile. Use the same Python selection when verifying an earlier run. Checks make no implicit data refresh.
An optional scripts/check --install-hook installs a pre-push code-check guard.
It verifies a current passing receipt or runs checks and then preserves Git LFS
pre-push processing. The guard requires a clean checkout so an uncommitted fix
cannot qualify a different pushed revision. Pushed branches/tags must point to
the checked HEAD; check out other revisions before pushing them. Custom hooks or core.hooksPath are left untouched; compose
the check command with those hooks manually. Check subprocesses cannot consume
the Git LFS ref updates from the hook's stdin. Inherited pytest control variables
are removed so shell settings cannot silently reduce the required suite.
Hosted checks remain available through manual GitHub Actions dispatch; pushes, pull requests, and schedules do not consume hosted minutes automatically. A passing local receipt is command evidence, not a signed attestation or an independent review approval. Adversarial review of the final diff remains required before landing.
PyPI Size Limits and Current Packaging Status
PyPI defaults currently documented at:
- per-file upload limit:
100 MB - total project limit:
10 GB
Reference: https://docs.pypi.org/project-management/storage-limits/
Before packaging trim, 0.0.118 artifacts were above the per-file limit:
- wheel: about
448 MB - sdist: about
446 MB
The Python distribution includes code, source contracts, and local methodology documents. Data artifacts are distributed separately and selected through one release manifest. Source checkouts hydrate Git LFS; HTTP deployments use a complete adjacent bundle as described in data releases. Installed wheels require TEADATA_RELEASE or TEADATA_RELEASE_URL. Publishing code does not refresh production data.
Release Policy
- Versioning uses thousandths-place tags (
v0.0.101,v0.0.102, ...). - Preserve published tags, releases, and assets for reproducibility; see the release policy in
AGENTS.md.
License
Apache License 2.0. See LICENSE.
Metadata
Release files for teadata 0.0.148
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| teadata-0.0.148.tar.gz | 7.3 MB | Without provenance | Details for teadata-0.0.148.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| teadata-0.0.148-py3-none-any.whl | Python 3 | none | any | Details for teadata-0.0.148-py3-none-any.whl |
Total release size: 13.4 MB
Release files / teadata-0.0.148.tar.gz
| Download URL | teadata-0.0.148.tar.gz |
|---|---|
| Size | 7.3 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
9a918c51bebb4ed9d9a3d6d9bc94dc638901d067ee20a483018a833f43a57a1d
|
|
BLAKE2b-256 checksum How to use checksums |
479750ec7be414c2e21a20472df8d5a7b7f374abadbe32fd5dc9035bb95d6b82
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / teadata-0.0.148-py3-none-any.whl
| Download URL | teadata-0.0.148-py3-none-any.whl |
|---|---|
| Size | 6.1 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
62872de2bb5be8f6a7f96b5999318d607fea8caecc549765ba29d492b636c512
|
|
BLAKE2b-256 checksum How to use checksums |
c8459b9837cde65eaa35701a3385a504d9025c73c35d07af19ade1c37a1589d6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.7 {"installer":{"name":"uv","version":"0.13.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for teadata 0.0.148
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.