Skip to main content

pypi ci mit

Seamlessly compress VTK datasets using Zstandard.

Read in VTK datasets 37x faster, write 14x faster, all while using 28% less space versus VTK’s modern XML format.

Read/Write Speedup and Compression Ratios

Read/Write Speedup and Compression Ratios

File Type / Method

Write Speed

Compression Ratio

Notes

Legacy VTK (.vtk)

465 MB/s

0.88

Significant overhead

VTK XML, none

256 MB/s

0.70

Significant overhead

VTK XML, zlib

105 MB/s

2.52

VTK Default

VTK XML, lz4

401 MB/s

1.47

VTK XML, lzma

9.93 MB/s

3.10

VTK HDF (.vtkhdf), lvl0

1733 MB/s

0.93

No compression

VTK HDF (.vtkhdf), lvl4

137 MB/s

2.37

Default compression

pyvista-zstd (.pv), lvl3

711 MB/s

3.02

Threads = 0

pyvista-zstd (.pv), lvl3

1845 MB/s

3.02

Threads = 4

pyvista-zstd (.pv), lvl22

15.8 MB/s

3.79

All threads (-1)

Usage

Install with:

pip install pyvista-zstd

The package is a C++ library with a Python wrapper around it, so an install is either a prebuilt wheel or a compile. Wheels are published for:

Linux

x86_64 and aarch64 (manylinux2014 and manylinux_2_28)

macOS

x86_64 and arm64

Windows

AMD64

Everywhere else – musl-based Linux, Windows on ARM – pip falls back to the source distribution and builds the core during the install, which needs a C++17 compiler on the machine. CMake does not: it comes from the build requirements. A machine without a compiler fails the install rather than producing a package that imports and cannot read anything.

Compatible with all VTK dataset types. Uses PyVista under the hood.

import pyvista_zstd

# create and write out
ds = pv.Sphere()
pyvista_zstd.write(ds, "dataset.pv")

# read in and show these are identical
ds_in = pyvista_zstd.read("dataset.pv")
assert ds == ds_in

For cell arrays where every cell contains the same number of points, pyvista-zstd stores the common cell width in the dataset metadata and omits the redundant offsets array. This applies to triangles, quads, tetrahedra, and any other fixed-width cell topology. Mixed-width cell arrays retain their explicit offsets.

Alternative VTK example

import vtk
import pyvista_zstd

# create dataset using VTK source
sphere_source = vtk.vtkSphereSource()
sphere_source.SetRadius(1.0)
sphere_source.SetThetaResolution(32)
sphere_source.SetPhiResolution(32)
sphere_source.Update()

vtk_ds = sphere_source.GetOutput()

# read back
pyvista_zstd.write(vtk_ds, "sphere.pv")
ds_in = pyvista_zstd.read("sphere.pv")

PyVista Integration

When pyvista-zstd is installed, it automatically registers with PyVista’s reader registry. This means pv.read() handles .pv files directly:

import pyvista as pv

mesh = pv.read("dataset.pv")

No additional imports needed. This works via PyVista’s pyvista.readers entry point group, so the registration happens at install time.

The C library

The reader and writer are a C++ core with a pure C ABI, pvzstd, declared in cpp/include/pvzstd/pvzstd.h. The Python package is one consumer of it, bound with ctypes; a C or C++ consumer – or a WebAssembly build – can link the same library directly, and it depends on nothing but zstd.

find_package(pvzstd CONFIG REQUIRED)
target_link_libraries(my_app PRIVATE pvzstd::pvzstd)

See The C library for the build options, both ways of consuming the CMake package, the ABI-version contract, and a worked example. The on-disk format is specified in doc/format/container-v2.md.

Rational

VTK’s XML writer is flexible and supports most datasets, but its compression is limited to a single thread, has only a subset of compression algorithms, and the XML format adds significant overhead.

To demonstrate this, the following example writes out a single file without compression. This example requires pyvista>=0.47.0 for the compression parameter.

>>> import numpy as np
>>> import pyvista as pv
>>> ugrid = pv.ImageData(dimensions=(200, 200, 200)).to_tetrahedra()
>>> ugrid["pdata"] = np.random.random(ugrid.n_points)
>>> ugrid["cdata"] = np.random.random(ugrid.n_cells)
>>> nbytes = (
...     ugrid.points.nbytes
...     + ugrid.cell_connectivity.nbytes
...     + ugrid.offset.nbytes
...     + ugrid.celltypes.nbytes
...     + ugrid["pdata"].nbytes
...     + ugrid["cdata"].nbytes
... )
>>> print(f"Size in memory: {nbytes / 1024**2:.2f} MB")

Size in memory: 1993.89 MB
Save using VTK XML format

>>> from pathlib import Path
>>> import time
>>> tmp_path = Path("/tmp/ds.vtu")
>>> tstart = time.time()
>>> ugrid.save(tmp_path, compression=None)
>>> print(f"Written without compression in {time.time() - tstart:.2f} seconds")
>>> nbytes_disk = tmp_path.stat().st_size
>>> print(f"  File size:            {nbytes_disk / 1024**2:.2f} MB")
>>> print(f"  Compression Ratio:    {nbytes / nbytes_disk}")
>>> print()

Written without compression in 7.93 seconds
File size:            2858.94 MB
Compression Ratio:    0.6974239255525742

This amounts to around a 43% overhead using VTK’s XML writer. Using the default compression we can get the file size down to 791 MB, but it takes 19 seconds to compress.

>>> tstart = time.time()
>>> ugrid.save(tmp_path, compression='zlib')  # default
>>> print(f"Compressed in {time.time() - tstart:.2f} seconds")
>>> nbytes_disk = tmp_path.stat().st_size
>>> print(f"  File size:            {nbytes_disk / 1024**2:.2f} MB")
>>> print(f"  Compression Ratio:    {nbytes / nbytes_disk}")
>>> print()

Compressed in 18.83 seconds
File size:            791.05 MB
Compression Ratio:    2.5205590295735663

Clearly there’s room for improvement here as this amounts to a compression rate of 105.89 MB/s.

VTK Compression with Zstandard: pyvista-zstd

This library, pyvista-zstd, writes out VTK datasets with minimal overhead and uses Zstandard for compression. Moreover, it’s been implemented with multi-threading support for both read and write operations.

Let’s compress that file again but this time using pyvista-zstd:

>>> import pyvista_zstd
>>> tmp_path = Path("/tmp/ds.pv")
>>> tstart = time.time()
>>> pyvista_zstd.write(ugrid, tmp_path)
>>> print(f"Compressed pyvista_zstd in {time.time() - tstart:.2f} seconds")
>>> nbytes_disk = tmp_path.stat().st_size
>>> print(f"  File size:            {nbytes_disk / 1024**2:.2f} MB")
>>> print(f"  Compression Ratio:    {nbytes / nbytes_disk}")

Compressed pyvista_zstd in 0.92 seconds
Threads:              -1
File size:            660.41 MB
Compression Ratio:    3.019175309922273

This gives us a write performance of 2167 MB/s using the default number of threads and compression level, resulting in a 20x speedup in write performance versus VTK’s XML writer. This speedup is most noticeable for larger files:

Speedup versus VTK’s XML

Speedup versus VTK’s XML

Even when disabling multi-threading we can still achieve excellent performance:

>>> tstart = time.time()
>>> pyvista_zstd.write(ugrid, tmp_path, n_threads=0)
>>> print(f"Compressed pyvista_zstd in {time.time() - tstart:.2f} seconds")
>>> nbytes_disk = tmp_path.stat().st_size
>>> print(f"  File size:            {nbytes_disk / 1024**2:.2f} MB")
>>> print(f"  Compression Ratio:    {nbytes / nbytes_disk}")

Compressed pyvista_zstd in 2.91 seconds
Threads:              0
File size:            660.47 MB
Compression Ratio:    3.0188911592355683

This amounts to a single-core compression rate of 685.18 MB/s, which is in agreement with Zstandard’s benchmarks.

Note that the benefit of threading drops off rapidly past 8 threads, though part of this is due to the performance versus efficiency cores of the CPU used for benchmarking (see below).

Read/Write Speed versus Number of Threads

Read/Write Speed versus Number of Threads


Reading in the dataset is also fast. Comparing with VTK’s XML reader using defaults:

Read VTK XML

>>> print(f"Read VTK XML:")
>>> timeit pv.read("/tmp/ds.vtu")
6.22 s ± 9.21 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Read zstd

>>> print(f"Read zstd:")
>>> timeit pyvista_zstd.read("/tmp/ds.pv")
563 ms ± 7.96 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

This is an 11x speedup for this dataset versus VTK’s XML, and it’s still fast even with multi-threading disabled:

>>> timeit pyvista_zstd.read("/tmp/ds.pv", n_threads=0)
1.11 s ± 4.51 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

This amounts to 1796 MB/s for a single core, which is also in agreement with Zstandard’s benchmarks.

Additionally, you can control Zstandard’s compression level by setting level=. A quick benchmark for this dataset indicates the defaults give a reasonable performance versus size tradeoff:

Read/Write Speed versus Compression Level

Read/Write Speed versus Compression Level

Note that both pyvista-zstd and VTK’s XML default compression give relatively constant compression ratios for this dataset across varying file sizes:

Compression Ratio versus VTK’s XML

Compression Ratio versus VTK’s XML

These benchmarks were performed on an i9-14900KF running the Linux kernel 6.12.41 using zstandard==0.24.0 from PyPI. Storage was a 2TB Samsung 990 Pro without LUKS mounted at /tmp.

Additional Information

The benchmarks/ directory contains additional benchmarks using many datasets, including all applicable datasets in pyvista.examples (see PyVista Dataset Gallery).

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pyvista_zstd-0.4.1.tar.gz (1.2 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

pyvista_zstd-0.4.1-py3-none-win_amd64.whl (263.4 kB view details)

Uploaded Python 3Windows x86-64

pyvista_zstd-0.4.1-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl (431.2 kB view details)

Uploaded Python 3manylinux: glibc 2.24+ x86-64manylinux: glibc 2.28+ x86-64

pyvista_zstd-0.4.1-py3-none-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl (414.7 kB view details)

Uploaded Python 3manylinux: glibc 2.24+ ARM64manylinux: glibc 2.28+ ARM64

pyvista_zstd-0.4.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (447.7 kB view details)

Uploaded Python 3manylinux: glibc 2.17+ x86-64

pyvista_zstd-0.4.1-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl (439.1 kB view details)

Uploaded Python 3manylinux: glibc 2.17+ ARM64

pyvista_zstd-0.4.1-py3-none-macosx_11_0_arm64.whl (300.3 kB view details)

Uploaded Python 3macOS 11.0+ ARM64

pyvista_zstd-0.4.1-py3-none-macosx_10_15_x86_64.whl (366.4 kB view details)

Uploaded Python 3macOS 10.15+ x86-64

File details

Details for the file pyvista_zstd-0.4.1.tar.gz.

File metadata

  • Download URL: pyvista_zstd-0.4.1.tar.gz
  • Upload date:
  • Size: 1.2 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pyvista_zstd-0.4.1.tar.gz
Algorithm Hash digest
SHA256 7a9bcc2c988760b2ce2d12852814350e02ac3ba7e420fcbe5cec785766431ddb
MD5 bc2744b422692af58da985ebbd61c7f7
BLAKE2b-256 9d87aa0fa5e13f258ddb53b34c5f5470f64017af584d1e7bbb0b5a2fc82b456b

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyvista_zstd-0.4.1.tar.gz:

Publisher: ci_cd.yml on pyvista/pyvista-zstd

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pyvista_zstd-0.4.1-py3-none-win_amd64.whl.

File metadata

  • Download URL: pyvista_zstd-0.4.1-py3-none-win_amd64.whl
  • Upload date:
  • Size: 263.4 kB
  • Tags: Python 3, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for pyvista_zstd-0.4.1-py3-none-win_amd64.whl
Algorithm Hash digest
SHA256 4ae800150e0d9823bf796c567b7e980ac662e4b49bf98ccf71ce4711064e623e
MD5 3aba56837942420a4c3815a0091a2ac8
BLAKE2b-256 80a272300d3cf7949c915403284ad2d1f22bb19e329074da6f37b7dbf4179d09

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyvista_zstd-0.4.1-py3-none-win_amd64.whl:

Publisher: ci_cd.yml on pyvista/pyvista-zstd

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pyvista_zstd-0.4.1-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pyvista_zstd-0.4.1-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 a74ba5bcc6aaa85acea5628282c0e374c7852b61fee0e3a53f8de390e1fb5f9e
MD5 b34a4cc3879b083411a0ac33a8c4fffe
BLAKE2b-256 d0e2645a31eeb9ca943ed1bfdef4ddce6855b44bb6f258e343a1d846ca2b1de7

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyvista_zstd-0.4.1-py3-none-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl:

Publisher: ci_cd.yml on pyvista/pyvista-zstd

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pyvista_zstd-0.4.1-py3-none-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl.

File metadata

File hashes

Hashes for pyvista_zstd-0.4.1-py3-none-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl
Algorithm Hash digest
SHA256 9a13fd0c7a9178eb869e0325c7a405fdef9a6d698b79551e91d3096fa41bc773
MD5 5fa842aab576cd3c1c2da6a99b824dfa
BLAKE2b-256 cdc8d8d701315d24715700f10dafa4b84b86975b137da0df6e53b1e8122a342f

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyvista_zstd-0.4.1-py3-none-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl:

Publisher: ci_cd.yml on pyvista/pyvista-zstd

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pyvista_zstd-0.4.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl.

File metadata

File hashes

Hashes for pyvista_zstd-0.4.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Algorithm Hash digest
SHA256 6cf833b52ef735489b8b906c13ee68cb170de2f3ea4b9459dee70d70fad225fb
MD5 6954a207e06092a15dded183720c15d1
BLAKE2b-256 4bfff99f4b2cbc73276241daffb5856eb7a87eb15b1ffa9aa950fa0a0169f9e5

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyvista_zstd-0.4.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl:

Publisher: ci_cd.yml on pyvista/pyvista-zstd

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pyvista_zstd-0.4.1-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl.

File metadata

File hashes

Hashes for pyvista_zstd-0.4.1-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
Algorithm Hash digest
SHA256 c1da7d3ea6508010e472d5fef2c64de57b7446d9120326d75bde78aae8f0af02
MD5 97d994117fc898adf97731d3ff7de0b2
BLAKE2b-256 2fbe6a8e89eb44eea3d0c5c5b239883faa834437010192065616a694fb0fb0a5

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyvista_zstd-0.4.1-py3-none-manylinux2014_aarch64.manylinux_2_17_aarch64.whl:

Publisher: ci_cd.yml on pyvista/pyvista-zstd

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pyvista_zstd-0.4.1-py3-none-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for pyvista_zstd-0.4.1-py3-none-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 cfd66aeeec3ab7e95232a9a78617978c05c18d152e8950a0cc6055f5365d61e9
MD5 44164ed9a553ca65308f7cd1b9d689ce
BLAKE2b-256 96ccb0fab7051de4b205d0c50c8be61e5fee7d8e8e5a681bfe4249c11282e966

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyvista_zstd-0.4.1-py3-none-macosx_11_0_arm64.whl:

Publisher: ci_cd.yml on pyvista/pyvista-zstd

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file pyvista_zstd-0.4.1-py3-none-macosx_10_15_x86_64.whl.

File metadata

File hashes

Hashes for pyvista_zstd-0.4.1-py3-none-macosx_10_15_x86_64.whl
Algorithm Hash digest
SHA256 0aa9e593cba155e8c07e2e5ff377b02936f736975f767ea29e7051f08eb79459
MD5 110ed3e6adf3663b042df7f7175f101e
BLAKE2b-256 d544205afaa672eeb60dfeb5238be5b1824193dcd8f63911488cb3e4acf6f057

See more details on using hashes here.

Provenance

The following attestation bundles were made for pyvista_zstd-0.4.1-py3-none-macosx_10_15_x86_64.whl:

Publisher: ci_cd.yml on pyvista/pyvista-zstd

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.4.1 This release

8 files

0.4.0

6 files

0.3.1

2 files

0.3.0

2 files

0.2.4

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page