Out-of-Core DataFrames to visualize and explore big tabular datasets
What is Vaex?
Vaex is a high performance Python library for lazy Out-of-Core DataFrames
(similar to Pandas), to visualize and explore big tabular datasets. It
calculates statistics such as mean, sum, count, standard deviation etc, on an
N-dimensional grid for more than a billion (
10^9) samples/rows per
second. Visualization is done using histograms, density plots and 3d
volume rendering, allowing interactive exploration of big data. Vaex uses
memory mapping, zero memory copy policy and lazy computations for best
performance (no memory wasted).
Instant opening of Huge data files (memory mapping)
Read the documentation on how to efficiently convert your data from CSV files, Pandas DataFrames, or other sources.
Lazy streaming from S3 supported in combination with memory mapping.
Don't waste memory or time with feature engineering, we (lazily) transform your data when needed.
Filtering and evaluating expressions will not waste memory by making copies; the data is kept untouched on disk, and will be streamed only when needed. Delay the time before you need a cluster.
Fast groupby / aggregations
Vaex implements parallelized, highly performant
groupby operations, especially when using categories (>1 billion/second).
Fast and efficient join
Vaex doesn't copy/materialize the 'right' table when joining, saving gigabytes of memory. With subsecond joining on a billion rows, it's pretty fast!
- Remote DataFrames (documentation coming soon)
- Integration into Jupyter and Voila for interactive notebooks and dashboards
- Machine Learning without (explicit) pipelines
Learn how to use Vaex efficiently
Release history Release notifications | RSS feed
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size vaex-3.0.0-py3-none-any.whl (3.9 kB)||File type Wheel||Python version py3||Upload date||Hashes View|
|Filename, size vaex-3.0.0.tar.gz (3.8 kB)||File type Source||Python version None||Upload date||Hashes View|