Project Maturity ESEE

ci Documentation Status

Concept • Installation • Example • Testing • Documentation

Polytope is a library for extracting complex data from datacubes. It provides an API for non-orthogonal access to data, where the stencil used to extract data from the datacube can be any arbitrary n-dimensional polygon (called a polytope). This can be used to efficiently extract complex features from a datacube, such as polygon regions or spatio-temporal paths.

Polytope is designed to extend different datacube backends:

  • XArray dataarrays
  • FDB object stores (through the GribJump software)

Polytope supports datacubes which have branching, non-uniform indexing, and even cyclic axes. If the datacube backend supports byte-addressability and efficient random access (either in-memory or direct from storage), Polytope can be used to dramatically decrease overall I/O load.

[!IMPORTANT] This software is Incubating and subject to ECMWF's guidelines on Software Maturity.

Concept

Polytope is designed to enable extraction of arbitrary extraction of data from a datacube. Instead of the typical range-based bounding-box approach, Polytope can extract any shape of data from a datacube using a "polytope" (n-dimensional polygon) stencil.

Polytope Concept

The Polytope algorithm can for example be used to extract:

  • 2D cut-outs, such as country cut-outs, from a datacube

    Greece cut-out

  • timeseries from a datacube

    Timeseries

  • more complicated spatio-temporal paths, such as flight paths, from a datacube

    Flight path

  • and many more high-dimensional shapes in arbitrary dimensions...

For more information about the Polytope algorithm, refer to our paper. If this project is useful for your work, please consider citing this paper.

Installation

Install the polytope software with Python 3 (>=3.7) from GitHub directly with the command

python3 -m pip install git+ssh://git@github.com/ecmwf/polytope.git@develop

or from PyPI with the command

python3 -m pip install polytope-python

Example

Here is a step-by-step example of how to use this software.

  1. In this example, we first specify the data which will be in our Xarray datacube. Note that the data here comes from the GRIB file called "winds.grib", which is 3-dimensional with dimensions: step, latitude and longitude.

        import xarray as xr

    array = xr.open_dataset("winds.grib", engine="cfgrib")

    We then construct the Polytope object, passing in some additional metadata describing properties of the longitude axis.

        options = {"longitude": {"cyclic": [0, 360.0]}}

    from polytope_feature.polytope import Polytope

    p = Polytope(datacube=array, axis_options=options)

  2. Next, we create a request shape to extract from the datacube.
    In this example, we want to extract a simple 2D box in latitude and longitude at step 0. We thus create the two relevant shapes we need to build this 3-dimensional object,

        import numpy as np
    from polytope_feature.shapes import Box, Select

    box = Box(["latitude", "longitude"], [0, 0], [1, 1])
    step_point = Select("step", [np.timedelta64(0, "s")])

    which we then incorporate into a Polytope request.

        from polytope_feature.polytope import Request

    request = Request(box, step_point)

  3. Finally, extract the request from the datacube.

        result = p.retrieve(request)

    The result is stored as an IndexTree containing the retrieved data organised hierarchically with axis indices for each point.

        result.pprint()
    

    Output IndexTree: 

        ↳root=None
            ↳step=0 days 00:00:00
                    ↳latitude=0.0
                            ↳longitude=0.0
                            ↳longitude=1.0
                    ↳latitude=1.0
                            ↳longitude=0.0
                            ↳longitude=1.0

Testing

Additional Dependencies

The Polytope tests and examples require additional Python packages compared to the main Polytope algorithm. The additional dependencies are provided in the requirements_test.txt and requirements_examples.txt files, which can respectively be found in the tests and examples folders. Moreover, Polytope's tests and examples also require the installation of eccodes and GDAL. It is possible to install both of these dependencies using either a package manager or manually.

Contributing

The main repository is hosted on GitHub; testing, bug reports and contributions are highly welcomed and appreciated. Please see the Contributing document for the best way to help.

Main contributors:

See also the contributors for a more complete list.

License

Copyright 2021 European Centre for Medium-Range Weather Forecasts (ECMWF)

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0).

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

In applying this licence, ECMWF does not waive the privileges and immunities
granted to it by virtue of its status as an intergovernmental organisation nor
does it submit to any jurisdiction.

Citing

If this software is useful in your work, please consider citing our paper as

Leuridan, M., Hawkes, J., Smart, S., Danovaro, E., & Quintino, T. (2025, November). Polytope: An Algorithm for Efficient Feature Extraction on Hypercubes. In Journal of Big Data (pp. 1-25).

Other papers include:

Leuridan, M., Bradley, C., Hawkes, J., Quintino, T., & Schultz, M. (2025, June). Performance Analysis of an Efficient Algorithm for Feature Extraction from Large Scale Meteorological Data Stores. In Proceedings of the Platform for Advanced Scientific Computing Conference (pp. 1-9).

Acknowledgements

Past and current funding and support for Polytope is listed in the adjoining Acknowledgements.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2.1.19)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ecmwf from gravatar.com ecmwf

Credits

Author: European Centre for Medium-Range Weather Forecasts (ECMWF)

Maintainer: James Hawkes

License

Apache Software License (Apache License Version 2.0)

Requires

Python >3.8

Provides Extra

tests unstructured catalogue

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

polytope_python-2.1.17.tar.gz (14.3 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

polytope_python-2.1.17-py3-none-any.whl (182.8 kB view details)

Uploaded Python 3

polytope_python-2.1.17-cp312-cp312-manylinux_2_28_x86_64.whl (560.2 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

polytope_python-2.1.17-cp312-cp312-macosx_11_0_arm64.whl (512.2 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

polytope_python-2.1.17-cp311-cp311-manylinux_2_28_x86_64.whl (564.3 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

polytope_python-2.1.17-cp311-cp311-macosx_11_0_arm64.whl (518.9 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

polytope_python-2.1.17-cp310-cp310-manylinux_2_28_x86_64.whl (563.6 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

polytope_python-2.1.17-cp310-cp310-macosx_11_0_arm64.whl (518.8 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

polytope_python-2.1.17-cp39-cp39-manylinux_2_28_x86_64.whl (567.9 kB view details)

Uploaded CPython 3.9manylinux: glibc 2.28+ x86-64

polytope_python-2.1.17-cp39-cp39-macosx_11_0_arm64.whl (521.1 kB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

File details

Details for the file polytope_python-2.1.17.tar.gz.

File metadata

  • Download URL: polytope_python-2.1.17.tar.gz
  • Upload date:
  • Size: 14.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.14.6

File hashes

Hashes for polytope_python-2.1.17.tar.gz
Algorithm Hash digest
SHA256 ecc78333ca05efde0faaf7e3a5ba1b99abeafc789bda05bea8f75197327b5e71
MD5 c3ac51376fb9e6056006a71e25d545fa
BLAKE2b-256 fd275d9fe19ff3655b2d37c2e81dbc15ff18503885cf530fb46b9e3c30068ea1

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.17-py3-none-any.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.17-py3-none-any.whl
Algorithm Hash digest
SHA256 b9fa4762f8c839a83be7b5e9454b6706e49f8d72443d1a73ed8b2891c56b8ca3
MD5 5a22d15426895c17d91ae80c5cfc076a
BLAKE2b-256 8c9356662f8c8db747ea224a515d76694d9899faa3c13ca45c5e9854357b7ecf

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.17-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.17-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 cac48e2ab1c005ab28942f9630ede98919c2f9d930c24133196e889c8426f54a
MD5 97402a6b497756612ab4842eb28d9445
BLAKE2b-256 610130343e560ccb0809ddcad2564df22d421a9ab73be8bf5dcec7320935cb4f

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.17-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.17-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 851ee60721a45dcfbc8d6d1213f5c0a3c5e5b2b09da304eb54914002c44fd5e1
MD5 da0d974316f092c04c6dd2723ea70c70
BLAKE2b-256 9dd38f8ed6e6e794c843a7e018eea0691ec07a1ae486a8111fce4d84a10702f9

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.17-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.17-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 996e55342c0e5abdd4d8f4a6a46943227657adb8909d25983bb0a0db95d92312
MD5 145247c4166f44c06a012fb2dc827233
BLAKE2b-256 d2705f13d9c5c6d6d45e98c83feff2f37074cb02fb603500d05b8bff52678a07

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.17-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.17-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 dc16e814dded2afef259f583a427bca14e5cd784504a9f0c0b7fe24aab8a9356
MD5 e46031dfdaf59204480c1bde9e8d61b5
BLAKE2b-256 1c2f4878e7a952be430513b5cc2f911965362bfcffda13ed850d6d6c18142146

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.17-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.17-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 432d63474e0622ff77afa76bbee3e8601ded9d34992716e102076660200e6fed
MD5 2bd5c04175f1ab87b8f63a97b79e773c
BLAKE2b-256 889659f96e8acb82712d276d138342d556e0ec6b225cbdbe7a1538f9f9fa6484

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.17-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.17-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 f72b7d87f8ebe509282b7c07fc23af2b8f763bd3f9ae1b110fbd6824e6c8eff7
MD5 6bce22fae4fbec38e31efe41db31ec90
BLAKE2b-256 8a35c4869458d2731c028a6ccdbab9b9e95864c352605bfe4962a8f734739cbf

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.17-cp39-cp39-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.17-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 95bcf5f4ec5884c7237b483344e1e3858e53787db21bae1c72b83a730b1c916b
MD5 d1dd92a9f3fc1be18cc18dc963ad18ff
BLAKE2b-256 9cfe345a88b8da3e72be2bf8421c91dce377079764498a31354d647f090b02d5

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.17-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.17-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 8855927125793f142d694b05e61f3b2fafbbefe0ed73fee98d5a7b8cbb3cd2d5
MD5 edcddb0229204a8a94c36b23c3992006
BLAKE2b-256 7107cf78f4e4b6d056928bc18f886f36273d551f9f5271d0e02836412c4a5065

See more details on using hashes here.

Release history Release notifications | RSS feed

2.1.19

9 files

2.1.18

9 files

This release

2.1.17 This release

10 files

2.1.15

12 files

2.1.13

12 files

2.1.12

12 files

2.1.10

12 files

2.1.9

12 files

2.1.8

12 files

2.1.7

12 files

2.1.6

12 files

2.1.5

12 files

2.1.4

12 files

2.1.3

12 files

2.1.2

12 files

2.1.1

12 files

2.1.0

12 files

2.0.4

12 files

2.0.3

12 files

2.0.2

12 files

2.0.1

12 files

2.0.0

9 files

1.1.1

2 files

1.1.0

2 files

1.0.42

2 files

1.0.41

2 files

1.0.40

2 files

1.0.39

2 files

1.0.38

2 files

1.0.37

2 files

1.0.36

2 files

1.0.35

2 files

1.0.34

2 files

1.0.33

2 files

1.0.32

2 files

1.0.31

2 files

1.0.30

1 file

1.0.29

1 file

1.0.28

1 file

1.0.27

1 file

1.0.26

1 file

1.0.25

1 file

1.0.23

1 file

1.0.22

1 file

1.0.21

1 file

1.0.20

1 file

1.0.19

1 file

1.0.18

1 file

1.0.16

1 file

1.0.15

1 file

1.0.14

1 file

1.0.13

1 file

1.0.12

1 file

1.0.11

1 file

1.0.10

1 file

1.0.9

1 file

1.0.8

1 file

1.0.7

1 file

1.0.6

1 file

1.0.5

1 file

1.0.4

1 file

1.0.3

1 file

1.0.2

1 file

1.0.1

1 file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page