usdata
Reproducible acquisition of U.S. public scientific data. One Python SDK and CLI discovers curated NOAA and USGS datasets, fetches their files, and keeps a record of every input: a manifest names them, a lockfile pins them by checksum, and the record of what was fetched is what a methods section cites. Analysis stays in pandas and xarray; usdata only acquires. If you need every product one agency publishes, that agency's own library is the better tool; usdata is for pinning inputs across sources and proving later that they have not changed.
pip install "usdata[pandas]"
usdata search precipitation --location Oklahoma
usdata fetch noaa:ghcn-daily -p stations=USW00013967 --start 2024-05-06 --end 2024-05-07
usdata pull dataset.yaml && usdata cite dataset.yaml
The same four steps from Python:
from usdata import build_query, fetch, get, pull
query = build_query(start="2024-05-06", end="2024-05-07", stations="USW00013967")
items = fetch(get("noaa:ghcn-daily"), query)
frame = items[0].open() # pandas DataFrame, units and provenance in frame.attrs
result = pull("dataset.yaml") # every manifest source, pinned in dataset.lock.json
print(result.fetched[0].provenance.checksum) # what usdata cite reports
Alpha. Releases are tested and lockfiles are stable, but the Python API and CLI can change between minor versions; the changelog marks breaking changes.
Where to look
| usdata.dev | What it is for: the dataset browser and worked examples with saved results |
| docs.usdata.dev | How to use it: install, getting started, guides, dataset notes, and reference |
| Severe-weather case study | One tornado, six sources, one manifest and lockfile, ending in a citation |
Twenty-five datasets are available today and twenty more are planned, grouped by agency and product family in the catalog.
How this compares
usdata resolves a place, a time window, and a few parameters to the exact files that satisfy them, then pins those files so the same inputs can be restored and verified later. Neighbouring tools each do part of that.
| Tool | What it does | What usdata adds |
|---|---|---|
| pooch | Downloads and verifies files from a registry of names, hashes, and a base URL you write | Discovery: the query resolves to the URLs, and the lockfile is written, not authored |
| intake | Describes datasets in catalogs and opens them through drivers | Checksums, lockfiles, and provenance; usdata stops at handing a file to pandas or xarray |
| DVC | Versions data files and can track one URL by its ETag or hash | Knowing which URLs a scientific question needs, across agencies |
| Herbie | Deep access to model output, including subsetting GRIB2 through index files | One interface across observations, radar, satellite, and models, with a lockfile; usdata's index-file fetch reproduces Herbie's for that reason |
| dataretrieval | Full coverage of USGS water data | Cross-agency pinning; use dataretrieval for USGS breadth |
Contributing
Setup, commands, checks, and the release procedure are in CONTRIBUTING.md. The project record lives beside it: architecture, testing, decisions, roadmap, and versioning. The main way the project grows is a new dataset, following adding a dataset.
License
1 maintainer
Release files for usdata 0.22.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| usdata-0.22.0.tar.gz | 267.2 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| usdata-0.22.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 575.9 kB
Release files / usdata-0.22.0.tar.gz
| Download URL | usdata-0.22.0.tar.gz |
|---|---|
| Size | 267.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
6338a2ca59e3635b032a3cb859d3ca9e494b588a99f5e9ae21fe002b924e4acc
|
|
BLAKE2b-256 checksum How to use checksums |
ca3f9adfd32cfe456840d277aa58a6171c9e734b3ffe4ab6a20e422f90e7f78b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.17 {"installer":{"name":"uv","version":"0.12.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / usdata-0.22.0-py3-none-any.whl
| Download URL | usdata-0.22.0-py3-none-any.whl |
|---|---|
| Size | 308.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ae8de2ceb48e2a3e4083aedbdbb462c2fad61ef6d48d93178157b43fb49bc118
|
|
BLAKE2b-256 checksum How to use checksums |
90edc97f7457b26bb75e587775afd2ca3463022dda15070a919d7a27339507a0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.17 {"installer":{"name":"uv","version":"0.12.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
