Project Maturity ESEE

ci Documentation Status

Concept • Installation • Example • Testing • Documentation

Polytope is a library for extracting complex data from datacubes. It provides an API for non-orthogonal access to data, where the stencil used to extract data from the datacube can be any arbitrary n-dimensional polygon (called a polytope). This can be used to efficiently extract complex features from a datacube, such as polygon regions or spatio-temporal paths.

Polytope is designed to extend different datacube backends:

  • XArray dataarrays
  • FDB object stores (through the GribJump software)

Polytope supports datacubes which have branching, non-uniform indexing, and even cyclic axes. If the datacube backend supports byte-addressability and efficient random access (either in-memory or direct from storage), Polytope can be used to dramatically decrease overall I/O load.

[!IMPORTANT] This software is Incubating and subject to ECMWF's guidelines on Software Maturity.

Concept

Polytope is designed to enable extraction of arbitrary extraction of data from a datacube. Instead of the typical range-based bounding-box approach, Polytope can extract any shape of data from a datacube using a "polytope" (n-dimensional polygon) stencil.

Polytope Concept

The Polytope algorithm can for example be used to extract:

  • 2D cut-outs, such as country cut-outs, from a datacube

    Greece cut-out

  • timeseries from a datacube

    Timeseries

  • more complicated spatio-temporal paths, such as flight paths, from a datacube

    Flight path

  • and many more high-dimensional shapes in arbitrary dimensions...

For more information about the Polytope algorithm, refer to our paper. If this project is useful for your work, please consider citing this paper.

Installation

Install the polytope software with Python 3 (>=3.7) from GitHub directly with the command

python3 -m pip install git+ssh://git@github.com/ecmwf/polytope.git@develop

or from PyPI with the command

python3 -m pip install polytope-python

Example

Here is a step-by-step example of how to use this software.

  1. In this example, we first specify the data which will be in our Xarray datacube. Note that the data here comes from the GRIB file called "winds.grib", which is 3-dimensional with dimensions: step, latitude and longitude.

        import xarray as xr

    array = xr.open_dataset("winds.grib", engine="cfgrib")

    We then construct the Polytope object, passing in some additional metadata describing properties of the longitude axis.

        options = {"longitude": {"cyclic": [0, 360.0]}}

    from polytope_feature.polytope import Polytope

    p = Polytope(datacube=array, axis_options=options)

  2. Next, we create a request shape to extract from the datacube.
    In this example, we want to extract a simple 2D box in latitude and longitude at step 0. We thus create the two relevant shapes we need to build this 3-dimensional object,

        import numpy as np
    from polytope_feature.shapes import Box, Select

    box = Box(["latitude", "longitude"], [0, 0], [1, 1])
    step_point = Select("step", [np.timedelta64(0, "s")])

    which we then incorporate into a Polytope request.

        from polytope_feature.polytope import Request

    request = Request(box, step_point)

  3. Finally, extract the request from the datacube.

        result = p.retrieve(request)

    The result is stored as an IndexTree containing the retrieved data organised hierarchically with axis indices for each point.

        result.pprint()
    

    Output IndexTree: 

        ↳root=None
            ↳step=0 days 00:00:00
                    ↳latitude=0.0
                            ↳longitude=0.0
                            ↳longitude=1.0
                    ↳latitude=1.0
                            ↳longitude=0.0
                            ↳longitude=1.0

Testing

Additional Dependencies

The Polytope tests and examples require additional Python packages compared to the main Polytope algorithm. The additional dependencies are provided in the requirements_test.txt and requirements_examples.txt files, which can respectively be found in the tests and examples folders. Moreover, Polytope's tests and examples also require the installation of eccodes and GDAL. It is possible to install both of these dependencies using either a package manager or manually.

Contributing

The main repository is hosted on GitHub; testing, bug reports and contributions are highly welcomed and appreciated. Please see the Contributing document for the best way to help.

Main contributors:

See also the contributors for a more complete list.

License

Copyright 2021 European Centre for Medium-Range Weather Forecasts (ECMWF)

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at [http://www.apache.org/licenses/LICENSE-2.0](http://www.apache.org/licenses/LICENSE-2.0).

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

In applying this licence, ECMWF does not waive the privileges and immunities
granted to it by virtue of its status as an intergovernmental organisation nor
does it submit to any jurisdiction.

Citing

If this software is useful in your work, please consider citing our paper as

Leuridan, M., Hawkes, J., Smart, S., Danovaro, E., & Quintino, T. (2025, November). Polytope: An Algorithm for Efficient Feature Extraction on Hypercubes. In Journal of Big Data (pp. 1-25).

Other papers include:

Leuridan, M., Bradley, C., Hawkes, J., Quintino, T., & Schultz, M. (2025, June). Performance Analysis of an Efficient Algorithm for Feature Extraction from Large Scale Meteorological Data Stores. In Proceedings of the Platform for Advanced Scientific Computing Conference (pp. 1-9).

Acknowledgements

Past and current funding and support for Polytope is listed in the adjoining Acknowledgements.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2.1.19)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for ecmwf from gravatar.com ecmwf

Credits

Author: European Centre for Medium-Range Weather Forecasts (ECMWF)

Maintainer: James Hawkes

License

Apache Software License (Apache License Version 2.0)

Requires

Python >3.8

Provides Extra

tests unstructured catalogue

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

polytope_python-2.1.18.tar.gz (14.3 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

polytope_python-2.1.18-cp312-cp312-manylinux_2_28_x86_64.whl (566.3 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64

polytope_python-2.1.18-cp312-cp312-macosx_11_0_arm64.whl (518.0 kB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

polytope_python-2.1.18-cp311-cp311-manylinux_2_28_x86_64.whl (571.5 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64

polytope_python-2.1.18-cp311-cp311-macosx_11_0_arm64.whl (522.9 kB view details)

Uploaded CPython 3.11macOS 11.0+ ARM64

polytope_python-2.1.18-cp310-cp310-manylinux_2_28_x86_64.whl (570.0 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64

polytope_python-2.1.18-cp310-cp310-macosx_11_0_arm64.whl (522.8 kB view details)

Uploaded CPython 3.10macOS 11.0+ ARM64

polytope_python-2.1.18-cp39-cp39-manylinux_2_28_x86_64.whl (574.1 kB view details)

Uploaded CPython 3.9manylinux: glibc 2.28+ x86-64

polytope_python-2.1.18-cp39-cp39-macosx_11_0_arm64.whl (525.8 kB view details)

Uploaded CPython 3.9macOS 11.0+ ARM64

File details

Details for the file polytope_python-2.1.18.tar.gz.

File metadata

  • Download URL: polytope_python-2.1.18.tar.gz
  • Upload date:
  • Size: 14.3 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.14.6

File hashes

Hashes for polytope_python-2.1.18.tar.gz
Algorithm Hash digest
SHA256 f3141d0390a96ff5a72a6365a2dc3feab637e9fe58f3ca8df706f9efadec8065
MD5 2116ea58e55499ecd22a7e52d13d0e61
BLAKE2b-256 9d85a1ccf832c8bba616d46caf95ad82ba71e05697c6ba27c719969f56838c11

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.18-cp312-cp312-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.18-cp312-cp312-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 ea22bb0d0e32948478a1d8c6d0877ae389b6e36b95f1608d37651784cb8df253
MD5 2cde532294d9372a7cca13a691509827
BLAKE2b-256 6fb016dd17541decfd3593b02f0d0fdc582e25fcf8dfbf45a65398fc9246480e

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.18-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.18-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 9eb9c0975168ae3a39a38ee4711c2a640e09a3de05d968469a9926ae0bb4e0cf
MD5 e256a530cdb5810441d13cb024b1141e
BLAKE2b-256 871206def97c15c89cb019c86685bd489a2021ad22e90fa0d0218bbca8d0a197

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.18-cp311-cp311-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.18-cp311-cp311-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 9dc90419a846f7985436154c8288e4c7508bdb8e4ac67a4ceae1aba8b7e1ce72
MD5 f49be9268dd35689e01205503a4fa060
BLAKE2b-256 6a29a8d002f86d4efdd14d251354b87235ad8bd76cdff119ac41d429e6f0292b

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.18-cp311-cp311-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.18-cp311-cp311-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 261bf2642f52812fca598d946d8600a5dd8def447ce70e786c42b1f7713dbfdc
MD5 01f39b9f20dcf5625901ea0aee91df0a
BLAKE2b-256 89e3ab83c10142cd0768a7b054dcc780b350f4b1004c5ebed46ae0280a82b728

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.18-cp310-cp310-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.18-cp310-cp310-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 992dac37fa32cd4c3aa643c02a9b532f00739bb1a2d935afbef00e7f2a58e6d7
MD5 6fd4e4825f202846443752a10c2aeeea
BLAKE2b-256 ed21bbea9c83ca6400c78becd622e75ec13ba826529e64568551f90ebada35f1

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.18-cp310-cp310-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.18-cp310-cp310-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2a0c7c59af2eb048d83ca210ddae8602ff7f35933c579fce8ad9f8cdc9ffb1e8
MD5 92cd07a8390f345dc460b8f5b2f1b5b6
BLAKE2b-256 a924dfe35c96b49b6117847b06ec80a6722ab92bb290139181db6dd0c1bb855b

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.18-cp39-cp39-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.18-cp39-cp39-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 738e313f9c4c984f360a5b41b58ee851133a16eca3fb592c5d69008b1d918321
MD5 76ffedf7bbcbec8082d1c0fb6158441a
BLAKE2b-256 b5d480a293955a274a8cedbbcfc64857c112df943c654e41819028b34d78acfe

See more details on using hashes here.

File details

Details for the file polytope_python-2.1.18-cp39-cp39-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polytope_python-2.1.18-cp39-cp39-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 3c91158e934772eec3822c44d4fe8bf0d96e4182f6c53e3e8a801d0b677daeb2
MD5 c422a5cd0561bb5ed39a018d0d1dd19a
BLAKE2b-256 ba8d59ef9187e98d086272908b3126901c89c0bfdcacd3655c92871c426725de

See more details on using hashes here.

Release history Release notifications | RSS feed

2.1.19

9 files

This release

2.1.18 This release

9 files

2.1.17

10 files

2.1.15

12 files

2.1.13

12 files

2.1.12

12 files

2.1.10

12 files

2.1.9

12 files

2.1.8

12 files

2.1.7

12 files

2.1.6

12 files

2.1.5

12 files

2.1.4

12 files

2.1.3

12 files

2.1.2

12 files

2.1.1

12 files

2.1.0

12 files

2.0.4

12 files

2.0.3

12 files

2.0.2

12 files

2.0.1

12 files

2.0.0

9 files

1.1.1

2 files

1.1.0

2 files

1.0.42

2 files

1.0.41

2 files

1.0.40

2 files

1.0.39

2 files

1.0.38

2 files

1.0.37

2 files

1.0.36

2 files

1.0.35

2 files

1.0.34

2 files

1.0.33

2 files

1.0.32

2 files

1.0.31

2 files

1.0.30

1 file

1.0.29

1 file

1.0.28

1 file

1.0.27

1 file

1.0.26

1 file

1.0.25

1 file

1.0.23

1 file

1.0.22

1 file

1.0.21

1 file

1.0.20

1 file

1.0.19

1 file

1.0.18

1 file

1.0.16

1 file

1.0.15

1 file

1.0.14

1 file

1.0.13

1 file

1.0.12

1 file

1.0.11

1 file

1.0.10

1 file

1.0.9

1 file

1.0.8

1 file

1.0.7

1 file

1.0.6

1 file

1.0.5

1 file

1.0.4

1 file

1.0.3

1 file

1.0.2

1 file

1.0.1

1 file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page