PolarsGrouper

PolarsGrouper is a Rust-based extension for Polars that provides efficient graph analysis capabilities, with a focus on component grouping and network analysis.

Core Features

Component Grouping

• super_merger: Easy-to-use wrapper for grouping connected components
• super_merger_weighted: Component grouping with weight thresholds
• Efficient implementation using Rust and Polars
• Works with both eager and lazy Polars DataFrames

Hierarchy / BOM Explosion

• hierarchy_totals, hierarchy_levels, hierarchy_paths: explode a parent → child edge list (bill of materials, chart of accounts, WBS) into its transitive closure, with quantities rolled up along every path
• Replaces WITH RECURSIVE / CONNECT BY queries and for-loops, and runs inside lazy queries

Additional Graph Analytics

• Shortest Path Analysis: Find shortest paths between nodes
• PageRank: Calculate node importance scores
• Betweenness Centrality: Identify key bridge nodes
• Association Rules: Discover item relationships and patterns

Installation

pip install polars-grouper

# For development:
python -m venv .venv
source .venv/bin/activate
maturin develop

Usage Examples

Basic Component Grouping

The core functionality uses super_merger to identify connected components:

import polars as pl
from polars_grouper import super_merger

df = pl.DataFrame({
    "from": ["A", "B", "C", "D", "E", "F"],
    "to": ["B", "C", "A", "E", "F", "D"],
    "value": [1, 2, 3, 4, 5, 6]
})

result = super_merger(df, "from", "to")
print(result)

Weighted Component Grouping

For cases where edge weights matter:

from polars_grouper import super_merger_weighted

df = pl.DataFrame({
    "from": ["A", "B", "C", "D", "E"],
    "to": ["B", "C", "D", "E", "A"],
    "weight": [0.9, 0.2, 0.05, 0.8, 0.3]
})

result = super_merger_weighted(
    df, 
    "from", 
    "to", 
    "weight",
    weight_threshold=0.3
)
print(result)

Hierarchy / BOM Explosion

Resolve a bill of materials in one lazy expression. Quantities multiply along a path (4 wheels × 5 screws) and add up across paths (+ 20 screws directly on the car):

import polars as pl
from polars_grouper import hierarchy_totals, hierarchy_levels, hierarchy_paths

bom = pl.LazyFrame({
    "parent": ["car", "car", "car", "wheel", "wheel", "wheel", "rim"],
    "child": ["wheel", "steering_wheel", "screw", "tyre", "rim", "screw", "iron"],
    "qty": [4.0, 1.0, 20.0, 1.0, 1.0, 5.0, 2.5],
})

totals = bom.select(hierarchy_totals("parent", "child", "qty").alias("bom")).unnest("bom")

# Raw materials needed for one car
totals.filter(pl.col("ancestor") == "car", pl.col("is_leaf")).collect()
# ┌──────────┬────────────────┬───────┬──────────┬─────────┐
# │ ancestor ┆ descendant     ┆ level ┆ quantity ┆ is_leaf │
# ╞══════════╪════════════════╪═══════╪══════════╪═════════╡
# │ car      ┆ steering_wheel ┆ 1     ┆ 1.0      ┆ true    │
# │ car      ┆ screw          ┆ 1     ┆ 40.0     ┆ true    │
# │ car      ┆ tyre           ┆ 2     ┆ 4.0      ┆ true    │
# │ car      ┆ iron           ┆ 3     ┆ 10.0     ┆ true    │
# └──────────┴────────────────┴───────┴──────────┴─────────┘

Three functions, one per level of detail. They share column names, so each is an aggregation of the next:

All three take top_level_only (explode finished products only), include_self (add a level-0 row per node) and max_depth. A cycle in the data raises an error that names it.

The same works for any hierarchy. To roll up general-ledger balances along a chart of accounts, link every account to itself and to everything below it, then aggregate:

closure = accounts.select(
    hierarchy_totals("parent_account", "account", include_self=True).alias("closure")
).unnest("closure")

balances = (
    closure.join(transactions, left_on="descendant", right_on="account")
    .group_by("ancestor")
    .agg(pl.col("amount").sum())
)

Additional Graph Analytics

Shortest Path Analysis

Find shortest paths between nodes:

from polars_grouper import calculate_shortest_path

df = pl.DataFrame({
    "from": ["A", "A", "B", "C"],
    "to": ["B", "C", "C", "D"],
    "weight": [1.0, 2.0, 1.0, 1.5]
})

paths = df.select(
    calculate_shortest_path(
        pl.col("from"),
        pl.col("to"),
        pl.col("weight"),
        directed=False
    ).alias("paths")
).unnest("paths")

PageRank Calculation

Calculate node importance:

from polars_grouper import page_rank

df = pl.DataFrame({
    "from": ["A", "A", "B", "C", "D"],
    "to": ["B", "C", "C", "A", "B"]
})

rankings = df.select(
    page_rank(
        pl.col("from"),
        pl.col("to"),
        damping_factor=0.85
    ).alias("pagerank")
).unnest("pagerank")

Association Rule Mining

Discover item relationships:

from polars_grouper import graph_association_rules

transactions = pl.DataFrame({
    "transaction_id": [1, 1, 1, 2, 2, 3],
    "item_id": ["A", "B", "C", "B", "D", "A"],
    "frequency": [1, 2, 1, 1, 1, 1]
})

rules = transactions.select(
    graph_association_rules(
        pl.col("transaction_id"),
        pl.col("item_id"),
        pl.col("frequency"),
        min_support=0.1
    ).alias("rules")
).unnest("rules")

Betweenness Centrality

Identify bridge nodes:

from polars_grouper import betweenness_centrality

df = pl.DataFrame({
    "from": ["A", "A", "B", "C", "D", "E"],
    "to": ["B", "C", "C", "D", "E", "A"]
})

centrality = df.select(
    betweenness_centrality(
        pl.col("from"),
        pl.col("to"),
        normalized=True
    ).alias("centrality")
).unnest("centrality")

Performance

The library is implemented in Rust for high performance:

• Efficient memory usage
• Fast computation for large graphs
• Seamless integration with Polars' lazy evaluation

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.