No project description provided
Project description
Polars
Blazingly fast DataFrames in Rust & Python
Polars is a blazingly fast DataFrames library implemented in Rust using Apache Arrow as memory model.
- Lazy | eager execution
- Multi-threaded
- SIMD
- Query optimization
- Powerful expression API
- Rust | Python | ...
To learn more, read the User Guide.
Rust users read this!
Polars cannot deploy a new version to crates.io
until a new arrow release is issued. Arrow's release cycle takes 3/4
months which is a lot slower than I'd like to release. If it has been a while since a release is issued, it is recommended
to use the current master
branch instead of the published version on crates.io
.
You can add the master like this:
polars = {version="0.13.0", git = "https://github.com/ritchie46/polars" }
Or by fixing to a specific version:
polars = {version="0.13.0", git = "https://github.com/ritchie46/polars", rev = "<optional git tag>" }
Rust version
Required Rust version >=1.51
Python users read this!
Polars is currently transitioning from py-polars
to polars
. Some docs may still refer the old name.
Install the latest polars version with:
$ pip3 install polars
Documentation
Want to know about all the features Polars support? Read the docs!
Rust
Python
- installation guide:
$ pip3 install polars
- User Guide
- Reference guide
Performance
Polars is written to be performant, and it is! But don't take my word for it, take a look at the results in h2oai's db-benchmark.
Cargo Features
Additional cargo features:
temporal (default)
- Conversions between Chrono and Polars for temporal data
simd (nightly)
- SIMD operations
parquet
- Read Apache Parquet format
json
- Json serialization
ipc
- Arrow's IPC format serialization
random
- Generate array's with randomly sampled values
ndarray
- Convert from
DataFrame
tondarray
- Convert from
lazy
- Lazy api
strings
- String utilities for
Utf8Chunked
- String utilities for
object
- Support for generic ChunkedArray's called
ObjectChunked<T>
(generic overT
). These will downcastable from Series through the Any trait.
- Support for generic ChunkedArray's called
[plain_fmt | pretty_fmt]
(mutually exclusive)- one of them should be chosen to fmt DataFrames.
pretty_fmt
can deal with overflowing cells and looks nicer but has more dependencies.plain_fmt (default)
is plain formatting.
- one of them should be chosen to fmt DataFrames.
Contribution
Want to contribute? Read our contribution guideline.
ENV vars
POLARS_PAR_SORT_BOUND
-> Sets the lower bound of rows at which Polars will use a parallel sorting algorithm. Default is 1M rows.POLARS_FMT_MAX_COLS
-> maximum number of columns shown when formatting DataFrames.POLARS_FMT_MAX_ROWS
-> maximum number of rows shown when formatting DataFrames.POLARS_TABLE_WIDTH
-> width of the tables used during DataFrame formatting.POLARS_MAX_THREADS
-> maximum number of threads used in join algorithm. Default is unbounded.POLARS_VERBOSE
-> print logging info to stderr
[Python] compile py-polars from source
If you want a bleeding edge release or maximal performance you should compile py-polars from source.
This can be done by going through the following steps in sequence:
- install the latest rust compiler
$ pip3 install maturin
- Choose any of:
- Very long compile times, fastest binary:
$ cd py-polars && maturin develop --rustc-extra-args="-C target-cpu=native" --release
- Shorter compile times, fast binary:
$ cd py-polars && maturin develop --rustc-extra-args="-C codegen-units=16 lto=no target-cpu=native" --release
Note that the Rust crate implementing the Python bindings is called py-polars
to distinguish from the wrapped
Rust crate polars
itself. However, both the Python package and the Python module are named polars
, so you
can pip install polars
and import polars
(previously, these were called py-polars
and pypolars
).
Acknowledgements
Development of Polars is proudly powered by
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distributions
Hashes for polars-0.7.16-cp36-abi3-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 4829507e6c44ac2143d1287c6bdbfba5338206f55ee871db2714a6ba5a195c02 |
|
MD5 | 76be6889fe22260b11d308d330a93c02 |
|
BLAKE2b-256 | 608e2e19ab4130b7c9c33e1459300793f569c185169cfe064a844af38e0f7a76 |
Hashes for polars-0.7.16-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 877676674ebc0169530f42ccf272fa7da692683e9aa173ce8cf162a1ca5400df |
|
MD5 | 50179bdb0a34a94dd2c4f99cdd72bc2f |
|
BLAKE2b-256 | dd211f21f874fcda9c3672f0f5857a380b62e277272f327eddf57dfddffcf124 |
Hashes for polars-0.7.16-cp36-abi3-macosx_10_7_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c714051c71fd01fbfee56bdc7960f0b49951f29519f2fe62ab452c7b6fcbad5c |
|
MD5 | 3c20943fea7a033408eae907fed42bd0 |
|
BLAKE2b-256 | 587a41e5d43746831f70e6a6eb6ff0813d5e5dae77b5aa79ef457568253f2061 |