This plugin provides stable hashing functionality across different polars versions.
📖 Documentation — every expression, its input and output types, and its arguments.
Examples
Cryptographic Hashers
import polars as pl
import polars_hash as plh
df = pl.DataFrame({
"foo":["hello_world"]
})
result = df.select(plh.col('foo').chash.sha256())
print(result)
┌──────────────────────────────────────────────────────────────────┐
│ foo │
│ --- │
│ str │
╞══════════════════════════════════════════════════════════════════╡
│ 35072c1ae546350e0bfa7ab11d49dc6f129e72ccd57ec7eb671225bbd197c8f1 │
└──────────────────────────────────────────────────────────────────┘
Non-cryptographic Hashers
df = pl.DataFrame({
"foo":["hello_world"]
})
result = df.select(plh.col('foo').nchash.wyhash())
print(result)
┌──────────────────────┐
│ foo │
│ --- │
│ u64 │
╞══════════════════════╡
│ 16737367591072095403 │
└──────────────────────┘
result = df.select(plh.col('foo').nchash.farmhash64())
print(result)
┌──────────────────────┐
│ foo │
│ --- │
│ u64 │
╞══════════════════════╡
│ 15605398435621216523 │
└──────────────────────┘
result = df.select(plh.col('foo').nchash.farmhash32())
print(result)
┌────────────┐
│ foo │
│ --- │
│ u32 │
╞════════════╡
│ 1719156559 │
└────────────┘
result = df.select(plh.col('foo').nchash.cityhash128())
print(result)
┌─────────────────────────────────────────┐
│ foo │
│ --- │
│ u128 │
╞═════════════════════════════════════════╡
│ 133423608296839006301901834072762183026 │
└─────────────────────────────────────────┘
result = df.select(plh.col('foo').nchash.gxhash64())
print(result)
┌─────────────────────┐
│ foo │
│ --- │
│ u64 │
╞═════════════════════╡
│ 2180020304351407825 │
└─────────────────────┘
cityhash32() and cityhash64() return the values printed above for farmhash32()
and farmhash64(). That is expected: FarmHash reuses CityHash for short input, and
hello_world is 11 bytes. See
the CityHash reference.
The GxHash expressions need a CPU with AES instructions and have no software fallback.
Every x86, x86-64 and aarch64 wheel is built for them; there are no linux-armv7 or
linux-ppc64le wheels from 0.8.0 on, because GxHash cannot be built for either. See
the GxHash reference.
Geo Hashers
df = pl.DataFrame(
{"coord": [{"longitude": -120.6623, "latitude": 35.3003}]},
schema={
"coord": pl.Struct(
[pl.Field("longitude", pl.Float64), pl.Field("latitude", pl.Float64)]
),
},
)
df.with_columns(
plh.col('coord').geohash.from_coords().alias('geohash')
)
shape: (1, 2)
┌─────────────────────┬──────────────┐
│ coord ┆ geohash │
│ --- ┆ --- │
│ struct[2] ┆ str │
╞═════════════════════╪══════════════╡
│ {-120.6623,35.3003} ┆ 9q60y60rhsgg │
└─────────────────────┴──────────────┘
pl.select(pl.lit('9q60y60rhs').geohash.to_coords().alias('coordinates'))
shape: (1, 1)
┌───────────────────────┐
│ coordinates │
│ --- │
│ struct[2] │
╞═══════════════════════╡
│ {-120.6623,35.300298} │
└───────────────────────┘
H3 Spatial Index
df = pl.DataFrame(
{"coord": [{"longitude": -120.6623, "latitude": 35.3003}]},
schema={
"coord": pl.Struct(
[pl.Field("longitude", pl.Float64), pl.Field("latitude", pl.Float64)]
),
},
)
df.with_columns(
plh.col('coord').h3.from_coords().alias('h3')
)
shape: (1, 2)
┌─────────────────────┬─────────────────┐
│ coord ┆ h3 │
│ --- ┆ --- │
│ struct[2] ┆ str │
╞═════════════════════╪═════════════════╡
│ {-120.6623,35.3003} ┆ 8c29adc423821ff │
└─────────────────────┴─────────────────┘
Time Hasher
Bins timestamps into variable-precision sliding windows of time, so rows that fall in the same window share a hash. Timestamps must lie between 1970-01-01 and 2098-01-01. A higher precision means a shorter window: 10 covers about 4 seconds, 8 about 4 minutes.
Precision may be 1 to 32, but past about 18 the hash stops changing for present-day timestamps and the extra characters are padding. The exact point depends on the date: timestamps close to 1970 keep splitting to about 21, far-future ones run out sooner.
from datetime import datetime
df = pl.DataFrame({"datetime": [datetime(2017, 2, 21, 20, 15, 13)]})
df.with_columns(
plh.col('datetime').timehash.from_datetime().alias('timehash')
)
shape: (1, 2)
┌─────────────────────┬────────────┐
│ datetime ┆ timehash │
│ --- ┆ --- │
│ datetime[μs] ┆ str │
╞═════════════════════╪════════════╡
│ 2017-02-21 20:15:13 ┆ afcccc0e1b │
└─────────────────────┴────────────┘
pl.select(pl.lit('afcccc0e1b').timehash.to_datetime().alias('datetime'))
shape: (1, 1)
┌────────────────────────────────┐
│ datetime │
│ --- │
│ datetime[μs, UTC] │
╞════════════════════════════════╡
│ 2017-02-21 20:15:11.292315 UTC │
└────────────────────────────────┘
pl.select(pl.lit('afcccc0e1b').timehash.neighbors().alias('neighbors'))
shape: (1, 1)
┌─────────────────────────────┐
│ neighbors │
│ --- │
│ struct[2] │
╞═════════════════════════════╡
│ {"afcccc0e1a","afcccc0e1c"} │
└─────────────────────────────┘
Create hash from multiple columns
df = pl.DataFrame({"foo": ["hello_world"], "bar": ["today"]})
result = df.select(plh.concat_str("foo", "bar").chash.sha256())
Hash a whole row
hash_rows gives each row bytes that no other row can make, for all column types.
Any hasher then reads those bytes.
df = pl.DataFrame(
{"foo": ["hello_world"], "bar": [42], "baz": [[1, 2, 3]], "qux": [{"a": 1}]}
)
df.select(plh.hash_rows(pl.all()).chash.sha2_256())
shape: (1, 1)
┌──────────────────────────────────────────────────────────────────┐
│ foo │
│ --- │
│ str │
╞══════════════════════════════════════════════════════════════════╡
│ 9055866af8d3c113e0a8fdb729ce8e6fa67ed5f6f51efa8235a588e88ea972f4 │
└──────────────────────────────────────────────────────────────────┘
The encoder reads the meaning of a value, not the polars storage of it. An Int32 and
the Int64 next to it make the same hash. A Datetime in milliseconds and the same
time in nanoseconds also make the same hash, and a Categorical makes the hash of its
string. The encoder does not read the column names. Therefore a new name keeps the
hash, but a new order does not. The
reference
gives all the rules and the byte layout of version 1, which does not change.
Metadata
Release files for polars-hash 0.9.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| polars_hash-0.9.3.tar.gz | 100.9 kB | Without provenance | Details for polars_hash-0.9.3.tar.gz |
Built distributions (wheels)
Total release size: 49.6 MB
Release files / polars_hash-0.9.3.tar.gz
| Download URL | polars_hash-0.9.3.tar.gz |
|---|---|
| Size | 100.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
91b935bd259125f2b2ddfc505d4e0cf4c9aaece84eddb3123ffa7ecf9aae17f1
|
|
BLAKE2b-256 checksum How to use checksums |
fa384cd0ad5383abc0d426833a3fd094552fef38cffb77a7fbb10d7ed7dbce96
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / polars_hash-0.9.3-cp310-abi3-win_amd64.whl
| Download URL | polars_hash-0.9.3-cp310-abi3-win_amd64.whl |
|---|---|
| Size | 7.5 MB |
| Tags | CPython 3.10 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
e3f937aef3cf7aeed21929763a93d02c8d5ae148ca824d4b1fcc48b4a6a55372
|
|
BLAKE2b-256 checksum How to use checksums |
517f8f51e99f8aca04fb9fcf26af66d5bbc45791fdcedacc52470f97c148bd4f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / polars_hash-0.9.3-cp310-abi3-win32.whl
| Download URL | polars_hash-0.9.3-cp310-abi3-win32.whl |
|---|---|
| Size | 6.7 MB |
| Tags | CPython 3.10 Windows x86-32 abi3 |
|
SHA-256 checksum How to use checksums |
f3ecc5ea90af6b69f3787a1aa959b166ebc27eee5055578df5f00bc7f987604d
|
|
BLAKE2b-256 checksum How to use checksums |
1804be600fabefb9b8e31f37ec0eb6247f6c18657b702cf3703216ecfd879bea
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / polars_hash-0.9.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | polars_hash-0.9.3-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 7.3 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
859deaaa1307a032f20b054ed605995a3d8fa627667f4a5d69ceb4ba096a28d2
|
|
BLAKE2b-256 checksum How to use checksums |
bf623ee86a12c93148abc022d7a6a569f09f0d82606eb4829b0669019b737e5c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / polars_hash-0.9.3-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | polars_hash-0.9.3-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 6.6 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
126658656629c8e6a1548ef969656adf2162e95e9528bb73a6b8a07a22e61a9c
|
|
BLAKE2b-256 checksum How to use checksums |
a6baae3e5f272f6e24483b1e60f28dbfe2f6b46c6d0400d7af099bcbcf48f708
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / polars_hash-0.9.3-cp310-abi3-manylinux_2_12_i686.manylinux2010_i686.whl
| Download URL | polars_hash-0.9.3-cp310-abi3-manylinux_2_12_i686.manylinux2010_i686.whl |
|---|---|
| Size | 7.9 MB |
| Tags | CPython 3.10 Linux glibc 2.12+ x86-32 abi3 |
|
SHA-256 checksum How to use checksums |
83db78a51ec4ffa241c6c700d9334393f65204a09278aefe4faa25798bfbf20b
|
|
BLAKE2b-256 checksum How to use checksums |
6f895dd6097e5997fa6071425b908c140f5884ad061633f4eefa4e308926c85b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / polars_hash-0.9.3-cp310-abi3-macosx_11_0_arm64.whl
| Download URL | polars_hash-0.9.3-cp310-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 6.3 MB |
| Tags | CPython 3.10 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
2052de662cab8af3a355fbe558250ed2d9ea5e44c61003e8eb40345991d84de8
|
|
BLAKE2b-256 checksum How to use checksums |
cb1da1c26d8caee4aecbd2534a28ef2117a199dc34e0ea3375d216f1fccb793c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Release files / polars_hash-0.9.3-cp310-abi3-macosx_10_12_x86_64.whl
| Download URL | polars_hash-0.9.3-cp310-abi3-macosx_10_12_x86_64.whl |
|---|---|
| Size | 7.1 MB |
| Tags | CPython 3.10 abi3 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
cbdcacdad679ced1c68de990d2506474fa8ed79ebe9afebbe2e0bd6667464a60
|
|
BLAKE2b-256 checksum How to use checksums |
f27a69aca782cb075f952243daefec47dc529d65658fd278ae19af4f77f3379f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.15.0
|
Security for polars-hash 0.9.3
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.