umfive

A PP and Fields file reader that mimics relevant parts of the pyfive high-level API, with support for lazy metadata loading and parallel data loading when necessary.

This is NOT an alternative to pyfive, it is a package which makes pp and fields files look like and taste ("quack like") chunked HDF5 files written by a NetCDF library.

API Contract (umfive vs pyfive)

umfive is designed to be pyfive-compatible as far as likely users of pyfive would need such compatability. That is, it produces a chunk index (for kerchunk) if you really want that, and it supports all the information needed by CF/CFDM workflows to make pp-data as CF-compliant as possible.

For the canonical pyfive interface, see the pyfive docs:

What maps directly to pyfive

  • umfive.File is registered as a umfive.File virtual type (when umfive is installed).
  • umfive.Variable is registered as a umfive.Dataset virtual type.
  • File/root-level members expected by pyfive-style callers are present:
    • attrs, groups, variables, dimensions, name, path, parent
  • Mapping-style access works:
    • f["var"], iteration over variables, items(), len(f)
  • Dataset-like variable members needed by downstream consumers are present:
    • shape, dtype, ndim, size, attrs, chunks/chunk_shape, file, parent
    • slicing/indexing via __getitem__ (for example f["var"][:])
    • read_direct, astype(...), iter_chunks(...)

Like pyfive, umfive is read-only:

- `File(..., mode="r")` is supported; write/update modes are not.

Important differences from pyfive

  • No nested groups in PP/Fields model:
    • f["group/var"] is not supported.
    • groups exists for compatibility but is empty for current PP/Fields inputs.
  • get_lazy_view(...) fallback behavior:
    • pyfive-style API entry exists, but returns the normal variable view with an info log.
  • Some Dataset properties are intentionally placeholders (None) because they are not meaningful for PP/Fields records:
    • compression, compression_opts, shuffle, fletcher32, maxshape, fillvalue, dims, scaleoffset.
  • Compatibility metadata is synthesized for CF/cfdm bridging:
    • dimension-scale datasets and DIMENSION_LIST are created where needed.
    • rotated-grid helper variables (for example latitude, longitude, rotated_latitude_longitude) may be exposed when implied by UM headers.
  • umfive-specific extension API:
    • File.set_parallelism(thread_count=..., cat_range_allowed=...) is provided by umfive and is not part of pyfive.

Practical expectation

If your code treats umfive.File as a pyfive-like read object, common analysis workflows should work, including chunk-level access (PP records are treated as chunks).

Authorship

Key authors of this package are:

  • David Hassell (@davidhassell)
  • Alan Iwi (@alaniwi)
  • Bryan Lawrence (@bnlawrence)

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

2 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for davidhassell from gravatar.com davidhassell Avatar for valeriupredoi from gravatar.com valeriupredoi

Credits

Author: David Hassell, Alan Iwi, Bryan Lawrence, Valeriu Predoi - package

License

MIT

Requires

Python >=3.11

Provides Extra

io test

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

umfive-0.3.0.tar.gz (19.9 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

umfive-0.3.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (188.3 kB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

umfive-0.3.0-cp314-cp314-macosx_11_0_arm64.whl (159.7 kB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

umfive-0.3.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (188.2 kB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

umfive-0.3.0-cp313-cp313-macosx_11_0_arm64.whl (159.7 kB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

umfive-0.3.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (188.1 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

umfive-0.3.0-cp312-cp312-macosx_11_0_arm64.whl (159.7 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

umfive-0.3.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (188.3 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

umfive-0.3.0-cp311-cp311-macosx_11_0_arm64.whl (159.5 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

File details

Details for the file umfive-0.3.0.tar.gz.

File metadata

  • Download URL: umfive-0.3.0.tar.gz
  • Upload date:
  • Size: 19.9 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for umfive-0.3.0.tar.gz
Algorithm Hash digest
SHA256 34cf8cbc95e1a8c022d73c543c014c9339b06bba4cfe1d9d455296c382c40b32
MD5 419f9169078bda0c181809600ec2056d
BLAKE2b-256 9f396dcafe05223fe7efa501cf77e0d36376f8943a0b4432c20fb2598f2e0871

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.3.0.tar.gz:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.3.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for umfive-0.3.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 c1eb2022b993d408c7bdf921671bcf8fe116ccc004af15a317fca80f2438156f
MD5 732d95b868f9480448a321d3e6bad896
BLAKE2b-256 1030601618566589a28a4dddb012bcc88cb175daf9f8d1ecfd1ba33b6da3bed5

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.3.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.3.0-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for umfive-0.3.0-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 d1daedddde8b6ca5460056a1e3e524d04101624432e5ce6443d4298f189ec514
MD5 fddd9c7b7a6a6feb821ed893525dabda
BLAKE2b-256 a252000668cdd756eef159e025c78e000ae5585c4abcd45a285cfd3e6194d5c0

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.3.0-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.3.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for umfive-0.3.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 cee1770ba1599b24dae5dd746163509580bb5842cb85bb624ce4e023ed5a6e8b
MD5 b400c62988192530ac7eb91b7cc95124
BLAKE2b-256 300e879f127c838132dad08d12adaaf4b317e382a056f9f3204151acb18bfcfd

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.3.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.3.0-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for umfive-0.3.0-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 5f3332dbe0e7f2574e5dbc640ab61c2ec4139a84bf0d65f5ce8268fad4ea684a
MD5 f5dc9c394280ca77e655680b3a603570
BLAKE2b-256 3d0c979ec53ad260e946b61769dc50ee3afd2d82f8c1037093e1e10d409e00f7

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.3.0-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.3.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for umfive-0.3.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ea87a9eefb7e78f90df97bbf7a4728920b5f3bae88f5f97d51e3416e8d3f90cf
MD5 7fec66f1973fb30c08e7042e39d3fa61
BLAKE2b-256 a5bf0a126ab20e495b2256a311d6a58727b2fe4bfb2ba8950508c2df46164527

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.3.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.3.0-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for umfive-0.3.0-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b230f82836bef750cf7d4d42e8732d812d646307c67c2688b37efed8bccb86b4
MD5 4c3831f906a7b4b43044dcf20dcbeda0
BLAKE2b-256 1f99929b46e5d052cd8e7aca2c86cd0a1339f14a237c7e2c98d94feafc0ff8a8

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.3.0-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.3.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for umfive-0.3.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 7d2d676cea22a837d02ed1e92be1a550a2d4577e970bbfdfa11cd8780e933ddc
MD5 ad62e7b19f2b06d8504dd45c98841d5f
BLAKE2b-256 13a211d23f5c409c28ff1fa0d61d74f5463b89560077484eb01cc9d38231e1d1

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.3.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.3.0-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for umfive-0.3.0-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 b820f9d7a9d2863ce681bcfdab870884e588e46f32a4bad8d4cbb0eb415b302a
MD5 32c6b73e385b373967c81c2707c6b645
BLAKE2b-256 8719ee6c96b49436272b461205765c925b1004eefba014554d95a4650299583e

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.3.0-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.3.0 This release

9 files

0.2.2

9 files

0.2.1

9 files

0.2.0

9 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page