Skip to main content

Quick searches of Hierarchical Data Format (HDF)-Files and dict-like data structures based on JSONPath.

Project description

HDFPath

A python package to enable quick searches of Hierarchical Data Format (HDF)-Files based on JSONPath. These files include the HDF5-Format provided by h5py as well as more recent libraries like zarr. Like jsonpath-ng any data structure consisting of dict-like objects conforming to the collections.abc.Mapping-interface and lists is also supported.

This package is derived from the jsonpath-ng library. For the query syntax and capabilities please refer to the original documentation at https://github.com/h2non/jsonpath-ng

Installation

Using pip

pip install hdfpath

From source

git clone https://github.com/mortacious/hdfpath.git
cd hdfpath
python setup.py install

Usage

As HDF-Files are organized as groups containing datasets with both containing optional metadata attributes, this package adds support to use these attributes directly inside queries. With the optional metadata_attribute parameter to the parse function, the attribute to retrieve the metadata can be chosen.

from hdfpath import parse
import h5py as h5

with h5.File("<HDF5-File>") as f:
    # query for all groups/datasets of type "scan" with the num_points attribute being larger than 40_000_000
    expr = parse('$..*[?(@._type == "scan" & @._num_points > 40000000)]', metadata_attribute='attrs')
    val = [match.value for match in expr.find(f)]
    print(val)

The metadata attributes are accessible inside the query using the _ prefix. Additionally, the use of regular expressions to match the fields is available through the `regex()` function. For example `regex(\\\d+)` will only match groups/datasets that can be parsed into an integer number.

TODOs

  • code examples

Copyright and License

Copyright 2021 - Felix Igelbrink

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hdfpath-0.1.0.tar.gz (30.1 kB view details)

Uploaded Source

Built Distribution

hdfpath-0.1.0-py3-none-any.whl (26.7 kB view details)

Uploaded Python 3

File details

Details for the file hdfpath-0.1.0.tar.gz.

File metadata

  • Download URL: hdfpath-0.1.0.tar.gz
  • Upload date:
  • Size: 30.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12

File hashes

Hashes for hdfpath-0.1.0.tar.gz
Algorithm Hash digest
SHA256 674e4d514afa31de89c4918eb63434b38164725751ed32b24147cd61f54e29f4
MD5 00e8ec603a9caa95085cbb0f0a929139
BLAKE2b-256 bc0327582b1f92bcaba829c60775fa292b8060731f73c1ed381a4db7612f749c

See more details on using hashes here.

File details

Details for the file hdfpath-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: hdfpath-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 26.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.8.2 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.8.12

File hashes

Hashes for hdfpath-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 32660824308f3af79ee6eeb1f7562fee37e6dfd0b1ccc333ec8df2bdf46f82c3
MD5 177a0ec6a4d9e9e64995f83e9d005c75
BLAKE2b-256 8d5cb3a99fa81f99f32ac124f68cf6c2afe0b5b3e78680be849bfc5c22d026d3

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page