usdata
Reproducible acquisition of U.S. public scientific data. One Python SDK and CLI discovers curated NOAA and USGS datasets, fetches their files, and keeps a record of every input: a manifest names them, a lockfile pins them by checksum, and the record of what was fetched is what a methods section cites. Analysis stays in pandas and xarray; usdata only acquires. If you need every product one agency publishes, that agency's own library is the better tool; usdata is for pinning inputs across sources and proving later that they have not changed.
pip install "usdata[pandas]"
usdata search precipitation --location Oklahoma
usdata fetch noaa:ghcn-daily -p stations=USW00013967 --start 2024-05-06 --end 2024-05-07
usdata pull dataset.yaml && usdata cite dataset.yaml
The same four steps from Python:
from usdata import build_query, fetch, get, pull
query = build_query(start="2024-05-06", end="2024-05-07", stations="USW00013967")
items = fetch(get("noaa:ghcn-daily"), query)
frame = items[0].open() # pandas DataFrame, units and provenance in frame.attrs
result = pull("dataset.yaml") # every manifest source, pinned in dataset.lock.json
print(result.fetched[0].provenance.checksum) # what usdata cite reports
Alpha. Releases are tested and lockfiles are stable, but the Python API and CLI can change between minor versions; the changelog marks breaking changes.
Where to look
| usdata.dev | What it is for: the dataset browser and worked examples with saved results |
| docs.usdata.dev | How to use it: install, getting started, guides, dataset notes, and reference |
| Severe-weather case study | One tornado, six sources, one manifest and lockfile, ending in a citation |
Twenty-four datasets are available today and twenty more are planned, grouped by agency and product family in the catalog.
How this compares
usdata resolves a place, a time window, and a few parameters to the exact files that satisfy them, then pins those files so the same inputs can be restored and verified later. Neighbouring tools each do part of that.
| Tool | What it does | What usdata adds |
|---|---|---|
| pooch | Downloads and verifies files from a registry of names, hashes, and a base URL you write | Discovery: the query resolves to the URLs, and the lockfile is written, not authored |
| intake | Describes datasets in catalogs and opens them through drivers | Checksums, lockfiles, and provenance; usdata stops at handing a file to pandas or xarray |
| DVC | Versions data files and can track one URL by its ETag or hash | Knowing which URLs a scientific question needs, across agencies |
| Herbie | Deep access to model output, including subsetting GRIB2 through index files | One interface across observations, radar, satellite, and models, with a lockfile; usdata's index-file fetch reproduces Herbie's for that reason |
| dataretrieval | Full coverage of USGS water data | Cross-agency pinning; use dataretrieval for USGS breadth |
Contributing
Setup, commands, checks, and the release procedure are in CONTRIBUTING.md. The project record lives beside it: architecture, testing, decisions, roadmap, and versioning. The main way the project grows is a new dataset, following adding a dataset.
License
1 maintainer
Release files for usdata 0.21.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| usdata-0.21.0.tar.gz | 263.0 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| usdata-0.21.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 566.4 kB
Release files / usdata-0.21.0.tar.gz
| Download URL | usdata-0.21.0.tar.gz |
|---|---|
| Size | 263.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
689b45ba8a79368a5410639728a2f7793943313c4a23aa34b51231b0ba0dd36b
|
|
BLAKE2b-256 checksum How to use checksums |
91c7948cff58c4cfeb912c5146fb2032d935fbae7923764cbe33ebf1fed3c8de
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.16 {"installer":{"name":"uv","version":"0.12.16","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / usdata-0.21.0-py3-none-any.whl
| Download URL | usdata-0.21.0-py3-none-any.whl |
|---|---|
| Size | 303.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
14ba320c9838038a620667afb1ebb45720a04f9a98dac1e38e6257d5dcfd0cd1
|
|
BLAKE2b-256 checksum How to use checksums |
06346ed7aeb6ccff9c743eac878c379fb9090d33539460ccd8a5474580182cf3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.16 {"installer":{"name":"uv","version":"0.12.16","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
