pyVIDE

pyVIDE is a python re-implementation of the VIDE / ZOBOV void finder intended for simulation boxes, relying only on minimal packages for an easy setup and use.

The overall goal is to allow the identification of voids that are identical to VIDE's, but in a simpler environment: pip install, numpy and scipy, with no compiler and no vendored qhull. On the reference catalog (1 214 925 halos in a 640 Mpc/h box) pyVIDE reproduces the corresponding VIDE catalog column for column, at every stage and at five different merging thresholds. The same comparison was repeated on a 3.8-million-halo sample from the same simulation, as well as on a test run with over 60 million tracers. The result was the same voids, the same members, and every void property agreeing to the precision that VIDE prints. See Verification for details.

Which mode to use

identify_voids has one required argument, VIDE_mode. 'pyvide' is the mode for new catalogs: it runs VIDE's algorithm on the exact positions you give it (double precision), keeps every tracer, and sizes its own buffer automatically, which saves memory and runtime. 'VIDE_halos' and 'VIDE_direct' reproduce an existing VIDE run to the last bit, rounding and dropped tracers included.

On three real catalogs, from 1.2 million to over 60 million tracers, at least 97 % of the voids were identical between 'pyvide' and 'VIDE_halos' down to their last tracer, with the same core position and with centres and radii equal to the precision VIDE prints. The same holds for the merged catalogs. In the remaining voids a tracer or two sat on the ridge to a neighbouring void and switched membership to the other void. On the largest test sample a few dozen voids out of several hundred thousand existed in only one of the two runs, and were part of a neighbouring void in the other. The VIDE_mode section below and docs/FIDELITY.md describe the details.

---

Documentation

This README provides an extended overview, with additional details, short guides and specific explanations in the linked files below.

document	what is in it
docs/GETTING_STARTED.md	from a catalog of positions to a finished void catalog, then the everyday workflows, including the use of some of the new features
docs/IDENTIFY_VOIDS.md	description of every argument of identify_voids, and of the density-field mode
docs/CATALOG_COLUMNS.md	every catalog column and void property, what it means, what units it is in
docs/FILES.md	what a run writes: every file, every key, outputs, overwrite
docs/FOR_VIDE_USERS.md	coming from VIDE: which mode, which catalog variant, what maps to what
docs/WEIGHTS_CEILING_AND_DUPLICATES.md	tracer weights, the density ceiling, and what duplicateMode='merge' keeps
docs/PERFORMANCE.md	sizing buffer and numDivisions, runtimes, memory, the warnings you may see
docs/ALGORITHM.md	how the void finder works, stage by stage
docs/FIDELITY.md	what "reproduces VIDE" means and how it was verified
docs/DEVELOPMENT_NOTES.md	how some of those answers in FIDELITY.md were achieved
LICENSE, CITATION.cff	GPL v2, and how to cite pyVIDE with the ZOBOV and VIDE papers

example_run.py is a complete runnable session on a synthetic box. Edit one marked block and it becomes your actual pyVIDE run. pyvide.explain('buffer') prints the documentation of any argument from inside python.

---

Install

git clone https://github.com/nicosmo/pyVIDE.git
cd pyVIDE
pip install -e .        # numpy >= 1.20, scipy >= 1.8, python 3.9 - 3.13

Optional extra packages: h5py for HDF5 output and pytest for the test suite, but in principle pyVIDE needs only numpy and scipy. Nothing is compiled: no qhull build, no numba, no GSL, no netCDF.

pyVIDE is pure python, so pip is optional. Download or clone the repository and either start python from its root or add that folder to your path:

import sys
sys.path.insert(0, "/path/to/pyvide")   # the folder containing pyvide/
import pyvide

Package versions

pyVIDE needs python 3.9, numpy 1.20 and scipy 1.8 or newer. This is the oldest combination it has actually been run on, and the test suite also runs on python 3.13 with current numpy and scipy. The scipy floor is exact rather than cautious: QhullError only became importable from scipy.spatial in 1.8, so on 1.7 the first run fails.

---

Quick start

import numpy as np
import pyvide

positions = np.loadtxt("halos.txt", usecols=(1, 2, 3))   # (N, 3), Mpc/h

catalog = pyvide.identify_voids(
    saveDir="voids",          # output folder, created if missing
    saveName="run1",          # prefix of every output file
    boxLen=640.0,             # scalar = cubic box; (Lx, Ly, Lz) is also possible
    positions=positions,      # the tracers the voids are found in
    VIDE_mode="pyvide",       # required: 'pyvide' for a new catalog, see details below
    numDivisions=2,           # sub-boxes per axis to decrease memory use
)

print(catalog.numVoids, catalog.radius.mean())

The run also wrote voids/run1_*, so the catalog can be read back in any later session. None of the calls below runs the finder again:

# saveDir="voids" and saveName="run1", the same two names as above
catalog = pyvide.load_catalog("voids", "run1", loadMembership=True)

# members() takes a void ID, not a row number, and returns the row numbers of
# the void's tracers in the tracer file (run1_tracers.npz: the kept tracers,
# in input order).  loadMembership=True reads the lists up front; without it,
# members() reads them from run1_void_details.npz on its first call.
maxVoid = int(catalog.voidID[np.argmax(catalog.radius)])
inVoid = pyvide.members(catalog, maxVoid)
print(inVoid.size, "tracers in void", maxVoid)

# one of VIDE's eight catalog variants
vide_catalog = pyvide.filter_catalog(catalog, "untrimmed_all")

# creating a catalog with a different merging threshold, no re-tessellation required
merged_catalog = pyvide.rebuild_catalog("voids", "run1", mergingThreshold=0.2)

A second run under the same saveName refuses to overwrite the first unless you pass overwrite=True. rebuild_catalog and rerun_watershed write new files under a name of their own rather than overwrite the run they read. Every argument of identify_voids is documented in docs/IDENTIFY_VOIDS.md, and docs/GETTING_STARTED.md walks through these workflows in more detail and with additional examples.

---

VIDE_mode: required, no default value

VIDE pushes positions through a text file and a float32 chain before anything is tessellated, and you have to say which behaviour you want:

value	what it does	when to use it
'VIDE_halos'	the full VIDE chain: %e rounding to seven significant digits of the positions, then float32	exactly reproducing a VIDE --halos run
'VIDE_direct'	the float32 chain without the text step	VIDE's plain matter mode (prepared without --halos, i.e. usually on dark matter tracers)
'pyvide'	the same void finding algorithm in double precision: no text round trip, no float32, no tracer dropped at the box edge, and on top of that a buffer sized from the catalog, correct numDivisions=1, and a fix for tracers sitting exactly on a sub-box face	the default choice, unless you are reproducing a VIDE run

The mode sets the precision of the whole run. 'VIDE_halos' and 'VIDE_direct' compute in float32 wherever VIDE does, because that is what makes the two catalogs identical, and they keep VIDE's fixed buffer and its box cut, including the tracers that this cut silently drops. 'pyvide' keeps the algorithm and removes those losses: float64 from the input positions to the catalog, no tracer dropped at the box edge, a buffer dynamically sized from the catalog (depending on the tracer density), and periodic images even with one sub-box per axis. Its catalog is not bit-identical to VIDE's, but it is the same catalog for every practical purpose: on the reference sample 5424 of the 5483 voids hold exactly the same tracers, the rest differ by a tracer or two on the ridge between two voids, and the merged catalogs are the same void for void. The test run with over 60 million tracers gives the same picture, with about two per cent of the voids experiencing changes. docs/FIDELITY.md has these measurements in more detail. Use 'VIDE_halos' or 'VIDE_direct' to reproduce or compare with a VIDE run, and 'pyvide' for everything else. docs/IDENTIFY_VOIDS.md lists every difference.

With VIDE_mode='VIDE_halos' or 'VIDE_direct', a periodic cubic box and no new features in use, pyVIDE reproduces VIDE's arithmetic and not just its algorithm, including the stage-1 quantization chain (%e rounding, float32 storage, the box cut, the tracer-dropping rules), the same Voronoi volumes, down to vorvol's float runsum, and more. See docs/FIDELITY.md for the full list.

Additions in pyVIDE

Every item below is a new or extended feature. Leave it unused or switch it off and the arithmetic above is untouched.

Merging, and the void hierarchy: Think of the tracer density as a mountainous landscape. Each zone is a valley around one local minimum. Raising the merging threshold is like raising a water level: two valleys join when the lowest ridge between them is below the threshold. A void is one core plus every valley that joined it before something stopped the flooding. Because a large valley swallows small ones, small voids end up inside larger ones, and that containment is the hierarchy: parentID names the smallest void containing a given one, treeLevel counts how many enclose it, while children() and descendants() list what is inside. At VIDE's default threshold of 1e-9 nothing effectively joins and the hierarchy is flat, meaning that each zone is a unique void. In contrast, a threshold of 0 does not mean merging is prevented, but instead merges everything into one hierarchy. pyVIDE stores the whole landscape of ridges in _merge_events.npz.

Weights: A weight tells the watershed how much density a tracer represents. A tracer of weight w fills its Voronoi cell with w units instead of one, so a heavy halo, a luminous galaxy, or a tracer that represents several unobserved ones makes its neighbourhood denser, while a low-weight one makes it emptier. The watershed runs on that weighted density, i.e. the original density multiplied by w/mean(w). Volumes, radii, centres and shapes stay geometric, with the weighted volumes reported alongside as voidVolWeighted, zoneVolWeighted and radiusWeighted. Equal weights give the unweighted catalog bit for bit, while unequal weights provide potentially different voids. See docs/WEIGHTS_CEILING_AND_DUPLICATES.md for more information.

A density ceiling: maxCellDensity sets the highest density a cell may have and still belong to a void, in units of the mean density. Cells above it belong to no void, so the voids no longer fill the box: the densest structures are left out as the walls between them. Merging cannot cross a ridge made of such cells, whatever the chosen merging threshold. It is a watershed-stage setting like the weights, so rerun_watershed can change it on a run made with saveIntermediate=True, while rebuild_catalog keeps the run's ceiling. See docs/WEIGHTS_CEILING_AND_DUPLICATES.md for more information.

A second void centre: besides the volume-weighted macrocenter, every tracer catalog has a circumcenter: the centre of the empty sphere through a void's core and three of its neighbours, chosen emptiest first so that all four are neighbours of one another, with that sphere's radius recorded as circumcenterRadius. See docs/CATALOG_COLUMNS.md.

Non-cubic boxes and per-axis periodicity: boxLen=(Lx, Ly, Lz) and periodicBox=(True, True, False) or 'xy'. The tessellation keeps the box's real aspect ratio, because squashing it into a cube would change which tracers are neighbours. numDivisions is per-axis too, so e.g., numDivisions=(2, 4, 2) keeps the sub-boxes cubic in a 100x200x100 box.

Walled boxes: Declare an axis non-periodic and its two faces become walls. A tracer whose Voronoi cell would cross a wall and reach outside the box is removed from the density graph, because on that side the cell is bounded by nothing real and its true volume is unknown. The tracers next to it keep their exact cells. Every void that contains these cells and therefore borders the removed layer gets boundaryFlag=True, because such a void was probably cut short by the box edge. cat.select(~cat.boundaryFlag) keeps only the voids that never reach this layer. Flagged voids stay in the catalog with all their properties, and wallDist gives their distance to the nearest wall. In its survey mode, VIDE scatters random mock particles along the edge and removes whatever is adjacent to them. For simulation boxes it has no wall treatment at all, so walled runs are a new pyVIDE feature with no VIDE counterpart. pyVIDE asks the question geometrically, so there is no mock density to choose. See docs/ALGORITHM.md.

numDivisions=1 in 'pyvide': VIDE's buffer construction generates no periodic images when one sub-box spans an entire axis, so the cells near that axis's faces come out wrong; the VIDE modes reproduce that and warn when one division is used. 'pyvide' adds the images explicitly for correct periodicity and gives the same voids as numDivisions=2 (checked by tests/test_single_division.py).

Coincident/duplicate tracers: Two or more tracers at the same position have no Voronoi cells. By default the run refuses and prints them. With duplicateMode='merge', which is not the default, each such group becomes one tracer carrying the group's summed weight, which is just the number of tracers in the group on an unweighted run. This leaves the density field as it was, and every merged-away tracer stays in the tracer file.

Density-field mode: identify_voids_from_field(saveDir, saveName, boxLen, field) runs the same zone building, watershed, hierarchy and property code on a grid: cells are voxels, adjacency is the lattice, densities are the field values themselves. No tessellation required and no corresponding VIDE_mode. Because of grid effects, trust the shapes and central densities only for voids that span many voxels.

Rebuilding catalogs, and changing weights or the density ceiling without re-tessellating: rebuild_catalog produces a full catalog at any other merging threshold from the stored zone graph, with no new tessellation at all, so it is far quicker than a fresh run and the result is verified identical to a complete rerun. With saveIntermediate=True the Delaunay graph is cached as well, and rerun_watershed then redoes the zones too, so weights and the density ceiling can be changed without tessellating again. Both take the same outputs and saveHDF5 default settings a fresh run takes, always write under their own file name, and record which original run they came from.

Smaller things: Child lists in the hierarchy (numDescendants, children(), descendants()); outputs='voids' for people who run the finder thousands of times and only ever read the actual void catalogs; array-or-path inputs ('run.npz:pos'), with the file recorded in the catalog, for the weights of a rerun as well; a readable end-of-run summary that catalog.summary() reprints; a log whose first line says which run it belongs to; build_tree, a KD-tree that knows which axes of the box are periodic and which not; optional HDF5 output; and a changeable guardResolution, which controls the shell of dummy tracers that closes each sub-box after the buffer.

---

What pyVIDE changes, and why

Each of these is a deliberate difference, and each is documented where it matters. docs/FOR_VIDE_USERS.md has the full list.

• Nothing is deleted by default: Every density basin becomes a catalog row and is flagged. VIDE's text catalog omits single-tracer zones. minRadius=None (no cut) is the new default, while minRadius=-1 reproduces VIDE's default cut at the mean tracer separation.
• mergingThreshold and maxCentralDen are separate: VIDE used the same value for both roles.
• volumeMethod='fast' is the default: The same Voronoi volumes without qhull's two calls per tracer, about 2-3x quicker on the tessellation, and it produces the same void catalog; 'exact' is one keyword away for per-tracer comparisons against a published vol_*.dat.
• Deterministic wall detection: as described above.
• The redshift-space displacement uses the correct formula by default: VIDE's is v·E(z)/100, which is not the comoving displacement v(1+z)/(100E); the two agree at z = 0, differ by about a per cent at z = 0.25 in the reference run's cosmology (Ω_M = 0.272; about 3 % at Ω_M = 0.3), but diverge more above z ≈ 0.5. VIDE's convention stays available as doRSD='VIDE' and is then reproduced exactly. See docs/FIDELITY.md for more information.
• isLeaf becomes isTopLevel, and the meaning flips: VIDE's isLeaf is true when a void has a parent, so it marks the voids that sit inside another one, which is the opposite of what "leaf" normally means in a tree. pyVIDE stores isTopLevel = (parentID == -1). When porting an analysis, wherever VIDE's isLeaf is true, pyVIDE's isTopLevel is false, and the other way round: a void VIDE calls a leaf is a void that sits inside another one.
• Small fixes: minRadius is changeable, output filenames are not length limited, and there is no compiler requirement.

---

Before you plot anything

Three definitions in this catalog are easy to misread. None of them are bugs, and all three are VIDE's own definitions rather than anything pyVIDE introduced. The void properties themselves are defined in the three papers under References: effective radius, volume-weighted centre, ellipticity and the void hierarchy in Sutter et al. 2015 (section 3), density contrast and the probability of a void being spurious in Neyrinck 2008, and the circumcentre in Nadathur & Hotchkiss 2015. All other columns are described in docs/CATALOG_COLUMNS.md.

what	why	what to do
np.mean(cat.ellipticity) is NaN	a one-member void has an all-zero inertia tensor, so the ratio of eigenvalues is 0/0. VIDE's catalogs contain no such voids because jozov2 leaves single-tracer zones out. pyVIDE keeps them	cat.ellipticity[cat.shapeReliable], the flag being numPart >= 4
centralDen is 0 for most voids	the sphere it counts in is one sixty-fourth of the void's volume and only members count, so a typical void of ~200 tracers expects about three tracers inside it and very often has none	cat.numCentral is the raw count; cat.centralDen[cat.numCentral >= 5] is where the column is a measurement
voidID is not the row number	on a box with walls, the zones cut off by a wall keep their zone IDs but are not voids, so they get no catalog row; and filtering removes rows without renumbering the IDs that remain. VIDE behaves the same way	cat.rows_of(ids) converts. Columns are indexed by row; members() and children() take IDs

---

Verification

pyVIDE's VIDE modes, 'VIDE_halos' and 'VIDE_direct', were compared against multiple frozen VIDE runs with a dedicated stage-by-stage harness, covering the prepared input, the quantized positions, the sub-box decomposition, the Delaunay graph, the Voronoi volumes, the full catalog and the redshift-space chain. Every stage passes, at three sizes:

sample	what was compared	result
1 214 925 halos, 640 Mpc/h (the reference catalog)	every stage, at five merging thresholds	every column exact but ellipticity, see below
3.8 million halos, same simulation	the Voronoi volumes and the full catalog, both volume methods	every column exact but the two below; the two volume methods are byte-for-byte identical
the same 3.8-million sample in redshift space (doRSD='VIDE')	the displacement chain and the full catalog	every one of 3 750 899 stored line-of-sight coordinates bit-identical; every column exact
over 60 million tracers (a test run)	the kept tracers, the zones, and every void's core, members and zones	identical; volumes, contrasts, radii and centres match to the precision VIDE prints, and VIDE's default catalog is the same set of voids

The reference sample, VIDE's output for it, pyVIDE's catalog and the harness are in the Zenodo dataset record (10.5281/zenodo.22736465), so the first row can be rerun by anyone. How the 'pyvide' mode compares with those catalogs is in Which mode to use above and in docs/FIDELITY.md.

Two columns generally disagree, both understood, both bounded, and neither coming from pyVIDE's own arithmetic: ellipticity differs in its sixth decimal for about eighty of some five thousand voids, because VIDE diagonalises the shape tensor with GSL and pyVIDE with LAPACK, the disagreement underneath is of order 1e-7; and prob, a three-digit printf of the exactly reproduced densCon, differs for one row of the fourteen thousand in the larger sample, where a rounding boundary falls between the two C libraries. docs/FIDELITY.md has the detail, the reference data, and what you can and cannot check yourself.

What ships with the code is a frozen change detector: tests/data/regression_box.npz holds a 100 000-tracer synthetic catalog and tests/data/regression_reference.npz the catalog pyVIDE 1.0.0 produces from it. pytest tests replays the whole pipeline and compares every column, dtype included. It cannot tell you the answer is right; only the VIDE comparison does that, but it tells you the answer has not changed, on any machine, in under a minute.

---

Not implemented

pyVIDE is a void finder for simulation boxes. These parts of VIDE are out of scope rather than overlooked:

• observation and survey mode, sky coordinates, healpix masks, boundary mocks, selection functions, redshift cuts. The survey-edge mechanism has been adapted for non-periodic boxes, without the random particles;
• lightcone placement, and subsampling of tracers (subsampling needs to be done yourself before running pyVIDE);
• --joggleParticles, which perturbs every particle to dodge a qhull degeneracy; duplicateMode addresses the same problem by naming the tracers that have it;
• snapshot readers, pyVIDE takes arrays, or a path to .npy/.npz files instead;
• VIDE's own output formats, and its voidUtil/apTools analysis package, which read those formats and start from a finished catalog;
• per-void RA/Dec/redshift columns, which are not meaningful for a periodic box.

---

References

• Neyrinck, M. C. 2008, ZOBOV: a parameter-free void-finding algorithm, MNRAS 386, 2101, arXiv:0712.3049
• Sutter, P. M. et al. 2015, VIDE: The Void IDentification and Examination toolkit, A&C 9, 1, arXiv:1406.1191
• Nadathur, S. & Hotchkiss, S. 2015, The nature of voids - I. Watershed void finders and their connection with theoretical models, MNRAS 454, 2228, arXiv:1504.06510

pyVIDE reproduces VIDE master, commit 8329b2c9 (see pyvide.VIDE_REFERENCE).

Citing pyVIDE

Please cite the ZOBOV and VIDE papers above alongside pyVIDE itself:

@software{pyVIDE,
  author  = {Schuster, Nico},
  title   = {pyVIDE},
  version = {1.0.0},
  year    = {2026},
  doi     = {10.5281/zenodo.22737440},
  url     = {https://github.com/nicosmo/pyVIDE}
}

If you use the verification catalogs, please cite the reference data record, 10.5281/zenodo.22736465, and the Magneticum papers listed in docs/FIDELITY.md.

Acknowledgements

pyVIDE is a re-implementation of VIDE and ZOBOV, and above all I thank their authors, Guilhem Lavaux, Paul Sutter and Mark Neyrinck. The source code they released made it possible to reproduce the finder to the last bit, and it remains the reference for everything this package does. I also thank the further VIDE contributors, Alice Pisani, Ben Wandelt, Nico Hamaus, Paul Zivick and Qingqing Mao, as well as Giovanni Verza for his contribution to VIDE on the weights implementation.

I am grateful to Andrés Salcedo, Carlos Correa, Giulia Degni, Julien Zoubian, Kai Lehman, Katayoon Ghaemi, Leander Thiele, Marie-Claude Cousinou, Nathan Findlay, Pierre Boccard, Sesh Nadathur, Simone Sartori, Sofia Contarini and, beyond their work on VIDE itself, to Alice Pisani, Giovanni Verza, Nico Hamaus and Paul Sutter, for many conversations about void finding and about working with VIDE. Several features of pyVIDE, and much of its documentation, grew out of those discussions.

The two 640 Mpc/h samples were drawn from the public galaxy catalog of Magneticum Box2b/hr at z = 0.252, available at magneticum.org/data.html. I thank the Magneticum team, and in particular Klaus Dolag, for making them public.

pyVIDE was developed with substantial use of Anthropic's Claude as a coding and drafting assistant, across the implementation, the test suite and this documentation. I designed pyVIDE, set the goal of bit-identical output and the fidelity policy that follows from it, decided what it adds to VIDE and how, reviewed the code and this documentation, and ran the verification against VIDE. Every claim about agreement with VIDE comes from the comparison harness described in docs/FIDELITY.md, run against VIDE's own outputs, and not from simply comparing the code.

License and attribution

pyVIDE is distributed under the GNU General Public License, version 2 (see LICENSE).

pyVIDE is a from-scratch python implementation of the algorithms in VIDE, the Void IDentification and Examination toolkit, Copyright (C) 2010-2025 Guilhem Lavaux and 2011-2014 P. M. Sutter, whose source files are distributed under the GNU GPL version 2. It was written by reading that source closely; the project's stated goal is bit-identical output. VIDE's void finder is in turn built on ZOBOV by Mark Neyrinck, free software redistributable as long as ZOBOV and its author are acknowledged, which this section and the references above gratefully do.