Project Maturity ESEE

ci Documentation Status

Concept • Installation • Example • Testing • Documentation

Polytope is a library for extracting complex data from datacubes. It provides an API for non-orthogonal access to data, where the stencil used to extract data from the datacube can be any arbitrary n-dimensional polygon (called a polytope). This can be used to efficiently extract complex features from a datacube, such as polygon regions or spatio-temporal paths.

Polytope is designed to extend different datacube backends:

  • XArray dataarrays
  • FDB object stores (through the GribJump software)

Polytope supports datacubes which have branching, non-uniform indexing, and even cyclic axes. If the datacube backend supports byte-addressability and efficient random access (either in-memory or direct from storage), Polytope can be used to dramatically decrease overall I/O load.

[!IMPORTANT] This software is Incubating and subject to ECMWF's guidelines on Software Maturity.

Concept

Polytope is designed to enable extraction of arbitrary extraction of data from a datacube. Instead of the typical range-based bounding-box approach, Polytope can extract any shape of data from a datacube using a "polytope" (n-dimensional polygon) stencil.

Polytope Concept

The Polytope algorithm can for example be used to extract:

  • 2D cut-outs, such as country cut-outs, from a datacube

    Greece cut-out

  • timeseries from a datacube

    Timeseries

  • more complicated spatio-temporal paths, such as flight paths, from a datacube

    Flight path

  • and many more high-dimensional shapes in arbitrary dimensions...

For more information about the Polytope algorithm, refer to our paper. If this project is useful for your work, please consider citing this paper.

Installation

Install the polytope software with Python 3 (>=3.7) from GitHub directly with the command

python3 -m pip install git+ssh://git@github.com/ecmwf/polytope.git@develop

or from PyPI with the command

python3 -m pip install polytope-python

Example

Here is a step-by-step example of how to use this software.

  1. In this example, we first specify the data which will be in our Xarray datacube. Note that the data here comes from the GRIB file called "winds.grib", which is 3-dimensional with dimensions: step, latitude and longitude.

        import xarray as xr

    array = xr.open_dataset("winds.grib", engine="cfgrib")

    We then construct the Polytope object, passing in some additional metadata describing properties of the longitude axis.

        options = {"longitude": {"cyclic": [0, 360.0]}}

    from polytope_feature.polytope import Polytope

    p = Polytope(datacube=array, axis_options=options)

  2. Next, we create a request shape to extract from the datacube.
    In this example, we want to extract a simple 2D box in latitude and longitude at step 0. We thus create the two relevant shapes we need to build this 3-dimensional object,

        import numpy as np
    from polytope_feature.shapes import Box, Select

    box = Box(["latitude", "longitude"], [0, 0], [1, 1])
    step_point = Select("step", [np.timedelta64(0, "s")])

    which we then incorporate into a Polytope request.

        from polytope_feature.polytope import Request

    request = Request(box, step_point)

  3. Finally, extract the request from the datacube.

        result = p.retrieve(request)

    The result is stored as an IndexTree containing the retrieved data organised hierarchically with axis indices for each point.

        result.pprint()
    

    Output IndexTree: 

        ↳root=None
            ↳step=0 days 00:00:00
                    ↳latitude=0.0
                            ↳longitude=0.0
                            ↳longitude=1.0
                    ↳latitude=1.0
                            ↳longitude=0.0
                            ↳longitude=1.0

Testing

Additional Dependencies

The Polytope tests and examples require additional Python packages compared to the main Polytope algorithm. The additional dependencies are provided in the requirements_test.txt and requirements_examples.txt files, which can respectively be found in the tests and examples folders. Moreover, Polytope's tests and examples also require the installation of eccodes and GDAL. It is possible to install both of these dependencies using either a package manager or manually.

Contributing

The main repository is hosted on GitHub; testing, bug reports and contributions are highly welcomed and appreciated. Please see the Contributing document for the best way to help.

Main contributors:

See also the contributors for a more complete list.

License

Copyright 2021 European Centre for Medium-Range Weather Forecasts (ECMWF)

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0).

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

In applying this licence, ECMWF does not waive the privileges and immunities
granted to it by virtue of its status as an intergovernmental organisation nor
does it submit to any jurisdiction.

Citing

If this software is useful in your work, please consider citing our paper as

Leuridan, M., Hawkes, J., Smart, S., Danovaro, E., & Quintino, T. (2025, November). Polytope: An Algorithm for Efficient Feature Extraction on Hypercubes. In Journal of Big Data (pp. 1-25).

Other papers include:

Leuridan, M., Bradley, C., Hawkes, J., Quintino, T., & Schultz, M. (2025, June). Performance Analysis of an Efficient Algorithm for Feature Extraction from Large Scale Meteorological Data Stores. In Proceedings of the Platform for Advanced Scientific Computing Conference (pp. 1-9).

Acknowledgements

Past and current funding and support for Polytope is listed in the adjoining Acknowledgements.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ecmwf from gravatar.com ecmwf

Credits

Author: European Centre for Medium-Range Weather Forecasts (ECMWF)

Maintainer: James Hawkes

License

Apache Software License (Apache License Version 2.0)

Requires

Python >3.8

Provides Extra

tests unstructured catalogue

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

polytope_python-2.1.19.tar.gz (14.3 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

polytope_python-2.1.19-cp312-cp312-manylinux_2_28_x86_64.whl (566.4 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

polytope_python-2.1.19-cp312-cp312-macosx_11_0_arm64.whl (518.1 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

polytope_python-2.1.19-cp311-cp311-manylinux_2_28_x86_64.whl (571.6 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

polytope_python-2.1.19-cp311-cp311-macosx_11_0_arm64.whl (522.9 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

polytope_python-2.1.19-cp310-cp310-manylinux_2_28_x86_64.whl (570.1 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

polytope_python-2.1.19-cp310-cp310-macosx_11_0_arm64.whl (522.8 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

polytope_python-2.1.19-cp39-cp39-manylinux_2_28_x86_64.whl (574.2 kB view details)

Uploaded CPython 3.9manylinux: glibc 2.28+ x86-64

polytope_python-2.1.19-cp39-cp39-macosx_11_0_arm64.whl (525.8 kB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

File details

Details for the file polytope_python-2.1.19.tar.gz.

File metadata

  • Download URL: polytope_python-2.1.19.tar.gz
  • Upload date:
  • Size: 14.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.14.7

File hashes

Hashes for polytope_python-2.1.19.tar.gz
Algorithm Hash digest
SHA256 b2d5317a05fd8f79fa2c6ba87709159b4bae7150847f06c1d6e0dee8cad6c443
MD5 a69392afa8d4193ab9d6ba48e23ab64e
BLAKE2b-256 2cf7d24d28bef8968c5126871601c7ac787c09493ddd26f056f791076fe709b1

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.19-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.19-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 2180c8dbbecab4f0bc84e89345cfd04a55514b8acb64f6036de5f49dda29e7f1
MD5 5fc8bf8c32192773de118f73b033b446
BLAKE2b-256 dab4a45f6037602168193a9a43eb579627ffc10610494d9d0a03da070d9340fc

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.19-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.19-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 1a395d0ea221b1eea0135f14fe13abe0b0e2bad38636b44c0155b1d9e1b67f9c
MD5 0c55f13f0d99d026ad1ad627e8c40491
BLAKE2b-256 bf802327b5d936b8324aee0a284db292e7cf16dfe5bf8c14545232b2d8285312

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.19-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.19-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 1065b996b5702071d2fa826c55c8ac9eb6038c2e15a627e68f71c3301dfdafb7
MD5 709f8f347252b3697d1225352ed29894
BLAKE2b-256 eb2aa598f7991b5fb2dc0d27959d18325300431db86b8e7255fd90bed09300ec

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.19-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.19-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 38bf6719f54e3d4ee0f736084bc34af5dd59992975eab2395a29e4b6276ae208
MD5 84cce16d9ebd3e8e827d6319501b51d4
BLAKE2b-256 23707c6d060d4f83cda92b7f2053d7a27be4ffb83755309ab6f9e063f10d9e33

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.19-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.19-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 cf4ba112def3d54850cc47214efff568bbaa74da49c51e29ff7818c06075fd83
MD5 1c8b9fbc494559489dbd37ee54d6bffd
BLAKE2b-256 4052d7d725c8d8282ddcce9b85e3590f20de9e1230071fe7ca629c60e608efdc

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.19-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.19-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 92d6c9a82a74e30405ae420235cb997604e70ac2a0055bac19fac1acc9c5fb2b
MD5 0e092f77d5708f15ad35b651242a6809
BLAKE2b-256 87f8d49c443c4b5c4f800cfdf7f65bbd066f0f3df7e032cd8767b63efb35ec4d

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.19-cp39-cp39-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.19-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 cecadc8da4445a09b379ae57e70ec4d4ebeef401e3f5b28405059a4b0ff7945e
MD5 789328d39fc75617223ec5d28cb76cfe
BLAKE2b-256 41fcb58caf8d36f1e0760621aa846f01daa995970c1cf38b114c255563523769

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.19-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.19-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2025b820767e6cc083a15afc5551865464cf28077d64af703bea3f3e16788c26
MD5 b50f11bc36e47ed81b3d0522a2ccab39
BLAKE2b-256 27ee8051eba1f76849c0de785f4af9b54b7ecd2e5b8eb5a3568536a8a1554f6d

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

2.1.19 This release

9 files

2.1.18

9 files

2.1.17

10 files

2.1.15

12 files

2.1.13

12 files

2.1.12

12 files

2.1.10

12 files

2.1.9

12 files

2.1.8

12 files

2.1.7

12 files

2.1.6

12 files

2.1.5

12 files

2.1.4

12 files

2.1.3

12 files

2.1.2

12 files

2.1.1

12 files

2.1.0

12 files

2.0.4

12 files

2.0.3

12 files

2.0.2

12 files

2.0.1

12 files

2.0.0

9 files

1.1.1

2 files

1.1.0

2 files

1.0.42

2 files

1.0.41

2 files

1.0.40

2 files

1.0.39

2 files

1.0.38

2 files

1.0.37

2 files

1.0.36

2 files

1.0.35

2 files

1.0.34

2 files

1.0.33

2 files

1.0.32

2 files

1.0.31

2 files

1.0.30

1 file

1.0.29

1 file

1.0.28

1 file

1.0.27

1 file

1.0.26

1 file

1.0.25

1 file

1.0.23

1 file

1.0.22

1 file

1.0.21

1 file

1.0.20

1 file

1.0.19

1 file

1.0.18

1 file

1.0.16

1 file

1.0.15

1 file

1.0.14

1 file

1.0.13

1 file

1.0.12

1 file

1.0.11

1 file

1.0.10

1 file

1.0.9

1 file

1.0.8

1 file

1.0.7

1 file

1.0.6

1 file

1.0.5

1 file

1.0.4

1 file

1.0.3

1 file

1.0.2

1 file

1.0.1

1 file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page