umfive

A PP and Fields file reader that mimics relevant parts of the pyfive high-level API, with support for lazy metadata loading and parallel data loading when necessary.

This is NOT an alternative to pyfive, it is a package which makes pp and fields files look like and taste ("quack like") chunked HDF5 files written by a NetCDF library.

API Contract (umfive vs pyfive)

umfive is designed to be pyfive-compatible as far as likely users of pyfive would need such compatability. That is, it produces a chunk index (for kerchunk) if you really want that, and it supports all the information needed by CF/CFDM workflows to make pp-data as CF-compliant as possible.

For the canonical pyfive interface, see the pyfive docs:

What maps directly to pyfive

  • umfive.File is registered as a umfive.File virtual type (when umfive is installed).
  • umfive.Variable is registered as a umfive.Dataset virtual type.
  • File/root-level members expected by pyfive-style callers are present:
    • attrs, groups, variables, dimensions, name, path, parent
  • Mapping-style access works:
    • f["var"], iteration over variables, items(), len(f)
  • Dataset-like variable members needed by downstream consumers are present:
    • shape, dtype, ndim, size, attrs, chunks/chunk_shape, file, parent
    • slicing/indexing via __getitem__ (for example f["var"][:])
    • read_direct, astype(...), iter_chunks(...)

Like pyfive, umfive is read-only:

- `File(..., mode="r")` is supported; write/update modes are not.

Important differences from pyfive

  • No nested groups in PP/Fields model:
    • f["group/var"] is not supported.
    • groups exists for compatibility but is empty for current PP/Fields inputs.
  • get_lazy_view(...) fallback behavior:
    • pyfive-style API entry exists, but returns the normal variable view with an info log.
  • Some Dataset properties are intentionally placeholders (None) because they are not meaningful for PP/Fields records:
    • compression, compression_opts, shuffle, fletcher32, maxshape, fillvalue, dims, scaleoffset.
  • Compatibility metadata is synthesized for CF/cfdm bridging:
    • dimension-scale datasets and DIMENSION_LIST are created where needed.
    • rotated-grid helper variables (for example latitude, longitude, rotated_latitude_longitude) may be exposed when implied by UM headers.
  • umfive-specific extension API:
    • File.set_parallelism(thread_count=..., cat_range_allowed=...) is provided by umfive and is not part of pyfive.

Practical expectation

If your code treats umfive.File as a pyfive-like read object, common analysis workflows should work, including chunk-level access (PP records are treated as chunks).

Authorship

Key authors of this package are:

  • David Hassell (@davidhassell)
  • Alan Iwi (@alaniwi)
  • Bryan Lawrence (@bnlawrence)

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.3.0)

2 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for davidhassell from gravatar.com davidhassell Avatar for valeriupredoi from gravatar.com valeriupredoi

Credits

Author: David Hassell, Alan Iwi, Bryan Lawrence, Valeriu Predoi - package

License

MIT

Requires

Python >=3.11

Provides Extra

io test

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

umfive-0.2.2.tar.gz (19.9 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

umfive-0.2.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (188.3 kB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

umfive-0.2.2-cp314-cp314-macosx_11_0_arm64.whl (159.6 kB view details)

Uploaded CPython 3.14macOS 11.0+ ARM64

umfive-0.2.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (188.1 kB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

umfive-0.2.2-cp313-cp313-macosx_11_0_arm64.whl (159.6 kB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

umfive-0.2.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (188.1 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

umfive-0.2.2-cp312-cp312-macosx_11_0_arm64.whl (159.6 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

umfive-0.2.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (188.3 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

umfive-0.2.2-cp311-cp311-macosx_11_0_arm64.whl (159.5 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

File details

Details for the file umfive-0.2.2.tar.gz.

File metadata

  • Download URL: umfive-0.2.2.tar.gz
  • Upload date:
  • Size: 19.9 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for umfive-0.2.2.tar.gz
Algorithm Hash digest
SHA256 46d40b4bb83c644e4ca013b42632cf8afde96c9c4e32b17355fd9acfc2383d9a
MD5 b504850b17e7f61b2484d48935e53e7f
BLAKE2b-256 7fe9c2b5a3678fc799a01a7a0db32da11aa3b3ac2c9cb73027ef8f59fef44d7e

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.2.2.tar.gz:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.2.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for umfive-0.2.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 7fdecd4e13afd8b036a441eff62cc5d1f4ee9c715b6d28c2862faeb71a8c2426
MD5 bbbd3ceda4a49179dd69252e3031fbe1
BLAKE2b-256 57a0ab563c2eba744aaf322ad293a7625db20e7c563d3fc75561568869a66bc7

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.2.2-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.2.2-cp314-cp314-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for umfive-0.2.2-cp314-cp314-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 4fb18a61379484d689132092d1654b12a4f5c62990f5601bc7a1a8147772dce8
MD5 c9f0f75e947c784271d5965048d07dd4
BLAKE2b-256 6fe8845578732aed1683c12349d456f4d9c1672bf685e1492b8d335d4f4c32af

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.2.2-cp314-cp314-macosx_11_0_arm64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.2.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for umfive-0.2.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 e28d3d0854b341ee0f835223c22b5479af39d9134694f44d0ef26490dc6de985
MD5 0b985cd0b16d831a5e940ca26b95dc07
BLAKE2b-256 3d7bb1c8ccc5523a03f92295a98c681d8c1467e9965acb33a170a8359c424ef7

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.2.2-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.2.2-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for umfive-0.2.2-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 7ef9bdcf8ef8553f205e1d8eca88a6e44928df17e41167cb37d339c9c0d4e8d0
MD5 696771e045f5cf7cdd29d4705d04ae5f
BLAKE2b-256 1457983efe5ce6f42eceb6ad8989a11c5eccefe5e188a2cc16f9ff5e9b03863b

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.2.2-cp313-cp313-macosx_11_0_arm64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.2.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for umfive-0.2.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 692b1879f650c2d3a43c4b5aa809d722086cb2e739e15563d4d88960d332dc3b
MD5 f8fca7aa0640ce14106e8b1d99b4ce20
BLAKE2b-256 20f8244325a72dd5541988fc17b9ab11f54f176f602ff1b58a1b694d13644aa5

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.2.2-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.2.2-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for umfive-0.2.2-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ed370805efa715e0c8fed0e3e0e9565805d05e10c78fffe816a7024ff42891d7
MD5 2e0e98f77f5219a6c82799088e91e7e3
BLAKE2b-256 809aad78bd6168d39d87d0ae801dccc7bbfacfeefdba9f5762075fe8b14995d2

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.2.2-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.2.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for umfive-0.2.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 035da09dce23c9b5f0f5928893d2dc220d9c6e7ba9644e27a46f9a83a4acbec3
MD5 a47b072b9c0dbc29f09083cbb6e9b3d6
BLAKE2b-256 cdc21e019f1f99bbb5f4ff58f2db76e2b3e6e785d4cf5865cd81193a9e6ef415

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.2.2-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file umfive-0.2.2-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for umfive-0.2.2-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 932e0cbe5120a2ae5642feaae8485377995ff97bc1d1706c608f9d85b3dae7d7
MD5 e91b692f877a73639191fb2c18544f58
BLAKE2b-256 bfb400873ee843301b6b817b31302e83b05414b6f54ed75183365dec17c8701c

See more details on using hashes here.

Provenance

The following attestation bundles were made for umfive-0.2.2-cp311-cp311-macosx_11_0_arm64.whl:

Publisher: build-and-deploy-wheels-on-pypi.yml on NCAS-CMS/umfive

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.3.0

9 files

This release

0.2.2 This release

9 files

0.2.1

9 files

0.2.0

9 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page