usdata
Reproducible acquisition of U.S. public scientific data. One Python SDK and CLI discovers curated datasets from agencies such as NOAA, USGS, and EPA, fetches their files, and keeps a record of every input: a manifest names them, a lockfile pins them by checksum, and the record of what was fetched is what a methods section cites. Analysis stays in pandas and xarray; usdata only acquires. If you need every product one agency publishes, that agency's own library is the better tool; usdata is for pinning inputs across sources and proving later that they have not changed.
pip install "usdata[pandas]"
usdata search precipitation --location Oklahoma
usdata fetch noaa:ghcn-daily -p stations=USW00013967 --start 2024-05-06 --end 2024-05-07
Save the same station query as dataset.yaml in your working directory:
name: first-station
sources:
- dataset: noaa:ghcn-daily
start: 2024-05-06
end: 2024-05-07
params:
stations: USW00013967
Then fetch and pin its inputs, and print their citation:
usdata pull dataset.yaml && usdata cite dataset.yaml
The first pull writes dataset.lock.json; later pulls restore its pinned assets.
From Python, using the same dataset.yaml:
from usdata import build_query, fetch, get, pull
query = build_query(start="2024-05-06", end="2024-05-07", stations="USW00013967")
items = fetch(get("noaa:ghcn-daily"), query)
frame = items[0].open() # pandas DataFrame, units and provenance in frame.attrs
result = pull("dataset.yaml") # every manifest source, pinned in dataset.lock.json
print(result.fetched[0].provenance.checksum) # what usdata cite reports
Alpha. Releases are tested and lockfiles are stable, but the Python API and CLI can change between minor versions; the changelog marks breaking changes.
Where to look
| usdata.dev | What it is for: the dataset browser and worked examples with saved results |
| docs.usdata.dev | How to use it: install, getting started, guides, dataset notes, and reference |
| Severe-weather case study | One tornado, six sources, one manifest and lockfile, ending in a citation |
Thirty-four datasets are available today and seventeen more are planned, grouped by agency and product family in the catalog.
How this compares
usdata resolves a place, a time window, and a few parameters to the exact files that satisfy them, then pins those files so the same inputs can be restored and verified later. Neighbouring tools each do part of that.
| Tool | What it does | What usdata adds |
|---|---|---|
| pooch | Downloads and verifies files from a registry of names, hashes, and a base URL you write | Discovery: the query resolves to the URLs, and the lockfile is written, not authored |
| intake | Describes datasets in catalogs and opens them through drivers | Checksums, lockfiles, and provenance; usdata stops at handing a file to pandas or xarray |
| DVC | Versions data files and can track one URL by its ETag or hash | Knowing which URLs a scientific question needs, across agencies |
| Herbie | Deep access to model output, including subsetting GRIB2 through index files | One interface across observations, radar, satellite, and models, with a lockfile; usdata's index-file fetch reproduces Herbie's for that reason |
| dataretrieval | Full coverage of USGS water data | Cross-agency pinning; use dataretrieval for USGS breadth |
Contributing
Setup, commands, checks, and the release procedure are in CONTRIBUTING.md. The project record lives beside it: architecture, testing, decisions, roadmap, and versioning. The main way the project grows is a new dataset, following adding a dataset.
License
Metadata
Release files for usdata 0.34.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| usdata-0.34.0.tar.gz | 330.7 kB | Without provenance | Details for usdata-0.34.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| usdata-0.34.0-py3-none-any.whl | Python 3 | none | any | Details for usdata-0.34.0-py3-none-any.whl |
Total release size: 714.5 kB
Release files / usdata-0.34.0.tar.gz
| Download URL | usdata-0.34.0.tar.gz |
|---|---|
| Size | 330.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
cb95eeb2c2eb934d114eaea460bb94c75fb5ec07a4e4918f0cda64f3c3537b2f
|
|
BLAKE2b-256 checksum How to use checksums |
3cc2794d985249b636a8e05b213ff5e19b880d615278e1e1599952d1f69e1ca9
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.21 {"installer":{"name":"uv","version":"0.12.21","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / usdata-0.34.0-py3-none-any.whl
| Download URL | usdata-0.34.0-py3-none-any.whl |
|---|---|
| Size | 383.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
afd0956adf67c12b1543076fcd3e478bf11fc43fd16272edd2ac0627683f51f2
|
|
BLAKE2b-256 checksum How to use checksums |
47ccb5ee70e25c54594d95a95b441c49ad89f460fca7e33d8f5160dcf96cba8c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.21 {"installer":{"name":"uv","version":"0.12.21","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for usdata 0.34.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.