Skip to main content

Polars IPTools

Polars IPTools is a Rust-based extension to accelerates IP address manipulation and enrichment in Polars dataframes. This library includes various utility functions for working with IPv4 and IPv6 addresses and geoip and anonymization/proxy enrichment using MaxMind databases.

Documentation PyPI - Python Version uv ruff ty License: MIT Claude Gemini

Install

pip install polars-iptools

Examples

Simple enrichments

IPTools' Rust implementation gives you speedy answers to basic IP questions like "is this a private IP?"

>>> import polars as pl
>>> import polars_iptools as ip
>>> df = pl.DataFrame({'ip': ['8.8.8.8', '2606:4700::1111', '192.168.100.100', '172.21.1.1', '172.34.5.5', 'a.b.c.d']})
>>> df.with_columns(ip.is_private(pl.col('ip')).alias('is_private'))
shape: (6, 2)
┌─────────────────┬────────────┐
 ip               is_private 
 ---              ---        
 str              bool       
╞═════════════════╪════════════╡
 8.8.8.8          false      
 2606:4700::1111  false      
 192.168.100.100  true       
 172.21.1.1       true       
 172.34.5.5       false      
 a.b.c.d          false      
└─────────────────┴────────────┘

IP extension types

IPTools provides two Arrow extension types for storing IP addresses efficiently. IPv4 uses 4-byte UInt32 storage; IPAddress uses 16-byte binary and handles both IPv4 and IPv6. Both types survive Parquet and IPC round-trips with dtype preserved.

>>> import polars as pl
>>> import polars_iptools as ip

>>> df = pl.DataFrame({"ip": ["8.8.8.8", "2606:4700::1111", "192.168.1.1", "invalid"]})
>>> df.with_columns(ip.to_address("ip"))
shape: (4, 2)
┌─────────────────┬─────────────────┐
 ip               ip              
 ---              ---             
 str              ip_addr         
╞═════════════════╪═════════════════╡
 8.8.8.8          8.8.8.8         
 2606:4700::1111  2606:4700::1111 
 192.168.1.1      192.168.1.1     
 invalid          null            
└─────────────────┴─────────────────┘

>>> # Write typed column to Parquet — dtype is preserved on read
>>> typed = df.with_columns(ip.to_address("ip"))
>>> typed.write_parquet("/tmp/ips.parquet")
>>> pl.read_parquet("/tmp/ips.parquet").dtypes
[String, IPAddress]

is_in but for network ranges

Pandas and Polars have is_in functions to perform membership lookups. IPTools extends this to enable IP address membership in IP networks. This function works seamlessly with both IPv4 and IPv6 addresses and converts the specified networks into a Level-Compressed trie (LC-Trie) for fast, efficient lookups.

>>> import polars as pl
>>> import polars_iptools as ip
>>> df = pl.DataFrame({'ip': ['8.8.8.8', '1.1.1.1', '2606:4700::1111']})
>>> networks = ['8.8.8.0/24', '2606:4700::/32']
>>> df.with_columns(ip.is_in(pl.col('ip'), networks).alias('is_in'))
shape: (3, 2)
┌─────────────────┬───────┐
 ip               is_in 
 ---              ---   
 str              bool  
╞═════════════════╪═══════╡
 8.8.8.8          true  
 1.1.1.1          false 
 2606:4700::1111  true  
└─────────────────┴───────┘

GeoIP enrichment

Using MaxMind's GeoLite2-ASN.mmdb and GeoLite2-City.mmdb databases, IPTools provides offline enrichment of network ownership and geolocation.

ip.geoip.full returns a Polars struct containing all available metadata parameters. If you just want the ASN and AS organization, you can use ip.geoip.asn.

>>> import polars as pl
>>> import polars_iptools as ip

>>> df = pl.DataFrame({"ip":["8.8.8.8", "192.168.1.1", "2606:4700::1111", "999.abc.def.123"]})
>>> df.with_columns([ip.geoip.full(pl.col("ip")).alias("geoip")])

shape: (4, 2)
┌─────────────────┬─────────────────────────────────┐
 ip               geoip                           
 ---              ---                             
 str              struct[11]                      
╞═════════════════╪═════════════════════════════════╡
 8.8.8.8          {15169,"GOOGLE","","NA","","", 
 192.168.1.1      {0,"","","","","","","",0.0,0. 
 2606:4700::1111  {13335,"CLOUDFLARENET","","","… │
 999.abc.def.123  {null,null,null,null,null,null 
└─────────────────┴─────────────────────────────────┘

>>> df.with_columns([ip.geoip.asn(pl.col("ip")).alias("asn")])
shape: (4, 2)
┌─────────────────┬───────────────────────┐
 ip               asn                   
 ---              ---                   
 str              str                   
╞═════════════════╪═══════════════════════╡
 8.8.8.8          AS15169 GOOGLE        
 192.168.1.1                            
 2606:4700::1111  AS13335 CLOUDFLARENET 
 999.abc.def.123                        
└─────────────────┴───────────────────────┘

Spur enrichment

Spur is a commercial service that provides "data to detect VPNs, residential proxies, and bots". One of its offerings is a Maxmind mmdb format of at most 2,000,000 "busiest" Anonymous or Anonymous+Residential ips.

ip.spur.full returns a Polars struct containing all available metadata parameters.

>>> import polars as pl
>>> import polars_iptools as ip

>>> df = pl.DataFrame({"ip":["8.8.8.8", "192.168.1.1", "999.abc.def.123"]})
>>> df.with_columns([ip.spur.full(pl.col("ip")).alias("spur")])

shape: (3, 2)
┌─────────────────┬─────────────────────────────────┐
 ip               spur                            
 ---              ---                             
 str              struct[7]                       
╞═════════════════╪═════════════════════════════════╡
 8.8.8.8          {0.0,"","","","","",null}       
 192.168.1.1      {0.0,"","","","","",null}       
 999.abc.def.123  {null,null,null,null,null,null 
└─────────────────┴─────────────────────────────────┘

Environment Configuration

IPTools uses two MaxMind databases: GeoLite2-ASN.mmdb and GeoLite2-City.mmdb. You only need these files if you call the geoip functions.

Set the MAXMIND_MMDB_DIR environment variable to tell the extension where these files are located.

export MAXMIND_MMDB_DIR=/path/to/your/mmdb/files
# or Windows users
set MAXMIND_MMDB_DIR=c:\path\to\your\mmdb\files

If the environment is not set, polars_iptools will check two other common locations (on Mac/Linux):

/usr/local/share/GeoIP
/opt/homebrew/var/GeoIP

Spur Environment

If you're a Spur customer, export the feed as spur.mmdb and specify its location using SPUR_MMDB_DIR environment variable.

export SPUR_MMDB_DIR=/path/to/spur/mmdb
# or Windows users
set SPUR_MMDB_DIR=c:\path\to\spur\mmdb

Credit

Developing this extension was super easy by following Marco Gorelli's tutorial and cookiecutter template.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

polars_iptools-0.2.2.tar.gz (180.8 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

polars_iptools-0.2.2-cp310-abi3-win_amd64.whl (6.2 MB view details)

Uploaded CPython 3.10+Windows x86-64

polars_iptools-0.2.2-cp310-abi3-win32.whl (5.6 MB view details)

Uploaded CPython 3.10+Windows x86

polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_x86_64.whl (5.8 MB view details)

Uploaded CPython 3.10+musllinux: musl 1.2+ x86-64

polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_armv7l.whl (5.6 MB view details)

Uploaded CPython 3.10+musllinux: musl 1.2+ ARMv7l

polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_aarch64.whl (5.3 MB view details)

Uploaded CPython 3.10+musllinux: musl 1.2+ ARM64

polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.6 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl (5.3 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARMv7l

polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (5.1 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

polars_iptools-0.2.2-cp310-abi3-macosx_11_0_arm64.whl (5.0 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

polars_iptools-0.2.2-cp310-abi3-macosx_10_12_x86_64.whl (5.4 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file polars_iptools-0.2.2.tar.gz.

File metadata

  • Download URL: polars_iptools-0.2.2.tar.gz
  • Upload date:
  • Size: 180.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for polars_iptools-0.2.2.tar.gz
Algorithm Hash digest
SHA256 8d4dc458008ea45ae95d7253c007b900acc595c941072fc4400090228baf221c
MD5 29057b9d810242e51695d408e52be7ef
BLAKE2b-256 51882ee2036a69f03b9d24a2039589d7bf6985ff6045c958b0a441e257b3e565

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2.tar.gz:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-win_amd64.whl.

File metadata

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 c099dad0dac568965db89ddd1e7fe6226cc34d4052aec858a5d9b99feae674b2
MD5 016e8884d67dab913f4e7dc0d752ad57
BLAKE2b-256 11a4444ac6cd6638767d3d9de3c84153b244a6e8dbe5758df4e1bc3de39fa022

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-win_amd64.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-win32.whl.

File metadata

  • Download URL: polars_iptools-0.2.2-cp310-abi3-win32.whl
  • Upload date:
  • Size: 5.6 MB
  • Tags: CPython 3.10+, Windows x86
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-win32.whl
Algorithm Hash digest
SHA256 9755e808090a13a360383f59a7571ade7728cdc20d99720e3d1b4b54afedc789
MD5 f94b2725e1f67ae20e845da22e88502c
BLAKE2b-256 895033b436f024bc8654d60c70119bc9e5e197f2fd679be6d71d50d08382f226

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-win32.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_x86_64.whl.

File metadata

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_x86_64.whl
Algorithm Hash digest
SHA256 8bcf64c708728600892a3e12d0afc0209e144a1da178a678004458719151d391
MD5 92c4c2c4ebb4840ab65873bf7ff4ac76
BLAKE2b-256 bfddc0f47a409282499efa87e258b08df466e7b39261757f828b1be735e0ea35

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_x86_64.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_armv7l.whl.

File metadata

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_armv7l.whl
Algorithm Hash digest
SHA256 5cd514844f3885d87e43859743c067a5cad351402f47684c2e9f1db7e9b81c7a
MD5 301ec184f8e4a8f15788bcd3fbebaa67
BLAKE2b-256 799a4967f9aa0fdaef84c6ecb3c76a107f108c48f4fdb2b8f68ccce0ee155d82

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_armv7l.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_aarch64.whl.

File metadata

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_aarch64.whl
Algorithm Hash digest
SHA256 ea9cd127c41e2d083d0025c6f26247a66bc6690148879e401acdd35c027459b4
MD5 35ec4d2028c55df690314f74c2ab1ea0
BLAKE2b-256 ae8d96ef5d834bcaafbe7752a5b54a43ab42c044f51cb3d678cc4ad69ae5b5d7

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-musllinux_1_2_aarch64.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 03c9640e1f014e9570ae2be02f36fba1835a0fde07b43433541dd3a32c568edc
MD5 049eec4f12291871d0efa6bbae53d297
BLAKE2b-256 a732c59188821b10d797295d4305c99e0356f9cc14b0a374f6e24b31ce8ce52c

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl.

File metadata

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Algorithm Hash digest
SHA256 d4806aea0e5e8a32655f931eb43cec0d4b023025c7590845915bb1c9f166f0f8
MD5 bb90490538a4910efee3107947c51d36
BLAKE2b-256 5dff312a5100d3f9a6df997e6fcf407008d456beab0712a2075a79ba232c31c6

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 b21a776809d8a446a4e4eaf70513741eeaa70e07924aef1a8fd0b0d63d60688c
MD5 c98830d351a581c2327e4c0f4ab6a441
BLAKE2b-256 c8dfdd259810a9aff0a8df3b5d40d8147d1d7fbb9222d52b64336f48c67b53d9

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 2dd4f8ee76936cef371c13d161d08e6a6ff86fbe19a696315484238019ba9bcc
MD5 398016a2fc3f84b3700c5c9a165c0b57
BLAKE2b-256 127433da43447a2cbe8186fcf8fff5de51df8b29ab2aad196e7b945dbd08593f

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file polars_iptools-0.2.2-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for polars_iptools-0.2.2-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 ad7950b03f27da91304db7ceae0f54df73d2b2306aba78f9266c3066acdf0919
MD5 969dfff4a4d6062f4660d4dcbc7af527
BLAKE2b-256 35a179da5b05945f3e09217e325881ce61d83248283c8573fed21c66ff965b82

See more details on using hashes here.

Provenance

The following attestation bundles were made for polars_iptools-0.2.2-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: publish_to_pypi.yml on erichutchins/polars_iptools

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.2.2 This release

11 files

0.2.1

11 files

0.2.0

11 files

0.1.10

11 files

0.1.9

6 files

0.1.8

6 files

0.1.7

6 files

0.1.6

6 files

0.1.5

6 files

0.1.2

6 files

0.1.0

6 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page