Ursa — Polars-shaped dataframes for graph data

Status: v0.1 in progress. An in-memory, single-machine graph analytics library with a dataframe-first API — what Polars is to tabular data, for graphs. The engine foundation is in place: real algorithm kernels, and collect() executing as one DataFusion plan with graph ops as first-class logical nodes. The full design lives in docs/SPEC.md; this README describes what is actually built right now and how the pieces fit.

Ursa is a Rust core (Apache Arrow throughout), a DataFusion query engine with graph operators as first-class plan nodes, and a fluent, Polars-shaped Python expression API. There is no Graph object — an EdgeFrame is the graph, and every operation returns a frame.

import ursa as ur

edges = ur.scan_edges("web-google.csv", src="FromNodeId", dst="ToNodeId")
top = (
    edges.nodes()
    .with_columns(
        pagerank  = ur.pagerank(edges, damping=0.85),
        in_degree = ur.degree(edges, direction="in"),
    )
    .sort("pagerank", descending=True)
    .head(20)
    .collect()          # runs as one DataFusion plan; results are Arrow
)

What works today

The architecture, crate boundaries, and load-bearing seams are all in place, and the parts that prove the design is sound are real and tested end-to-end.

Layer State
ursa-core — CSR topology index + kernels ✅ Real & unit-tested. Dense u32 indexing, lazy-transpose CSR with the edge_ids permutation, and working degree / pagerank (pull-based) / connected_components (union-find) / triangle_count / clustering_coefficient (sorted-adjacency intersection) / bfs (frontier) / closeness / betweenness (Brandes, with source sampling) / label_propagation / louvain (modularity) kernels, each with a weighted variant.
ursa-plan — DataFusion engine ✅ Unified plan. Each collect() is one DataFusion LogicalPlan — Limit → Sort → Filter → GraphAlgorithmNode — where GraphAlgorithmNode is a real UserDefinedLogicalNode lowered to GraphAlgorithmExec by our own ExtensionPlanner. Graph ops are first-class citizens of the plan (not orchestrated from outside), which is where future optimizer rules register. A DataFusion scan reads Parquet/CSV edge/node files, local or from object storage (s3:// / gs:// / az://), with the column projection pushed into the file.
ursa-py — PyO3 bindings ✅ Wired. Arrow in/out zero-copy (PyCapsule), GIL released during compute.
Python dialect + collect() ✅ Live & executing. The Polars-shaped expression/plan builder, plus collect() for a standalone algorithm, a composed with_columns(...).filter(...).sort(...).head(n).select(...) pipeline, node-attribute enrichment (in-memory or scan_nodes file-backed tables joined by id, ur.col("attr") usable in filter/sort; with_columns stays additive and select(...) narrows the output Polars-faithfully — which drives a projection pushdown so a scan_nodes file reads only the columns the plan proves it needs), neighbors().agg() over numeric and string attributes, the traversals hop() and shortest_path() (first-class HopNode/ShortestPathNode returning EdgeFrames) plus random_walk() (a RandomWalkNode returning a (walk_id, step, node) frame), and the whole-graph stats describe() / density() / avg_path_length() / diameter(). Over in-memory or scan_edges/scan_nodes sources — local files or object storage (s3:// / gs:// / az://, with storage_options={...}). 
import ursa as ur

# The EdgeFrame *is* the graph. Build one from native Python data — a list of
# row dicts, a dict of columns, a polars/pandas DataFrame, or pyarrow. No
# `import pyarrow` required; `src`/`dst` name the endpoint columns:
edges = ur.EdgeFrame(
    [{"s": 1, "d": 0}, {"s": 2, "d": 0}, {"s": 3, "d": 0}, {"s": 0, "d": 1}],
    src="s",
    dst="d",
)
# equivalently: ur.EdgeFrame({"s": [1, 2, 3, 0], "d": [0, 0, 0, 1]}, src="s", dst="d")

# Composed pipeline — runs through DataFusion end to end:
(
    edges.nodes()
    .with_columns(pr=ur.pagerank(edges), indeg=ur.degree(edges, direction="in"))
    .filter(ur.col("indeg") > 0)
    .sort("pr", descending=True)
    .head(10)
    .collect()
    .to_polars()
)

# ...or straight from a file (Parquet/CSV, projection pushed into the scan):
ur.pagerank(ur.scan_edges("edges.parquet", src="s", dst="d")).collect().to_polars()

# However the frame is built — constructor, from_polars/from_pandas/from_arrow,
# or scan_edges — it behaves identically from here on.

# Node ids may be int64 (the fast path) or strings (e.g. UUIDs) — auto-detected
# from the column type; results come back keyed by the original ids:
str_edges = ur.EdgeFrame({"s": ["u1", "u1", "u2"], "d": ["u2", "u3", "u3"]}, src="s", dst="d")
ur.pagerank(str_edges).collect().to_polars()   # id column is Utf8

NodeFrame(data, id=...) is the matching constructor for an attribute table (joined to the graph by id); a NodeFrame is optional — nodes are implicit in the edges (edges.nodes()), and attributes are a separate table you attach when you have them.

Architecture

ursa/
├── ursa-core/    # Topology (CSR) + algorithm kernels. Pure Rust: arrow + rayon.
│                 # NO DataFusion dependency. Independently testable.
├── ursa-plan/    # DataFusion extensions: custom logical node + ExecutionPlan
│                 # (-> ursa-core), the query builder, scan/session plumbing.
│                 # The ONE seam where our dialect lowers to DataFusion.
│                 # (optimizer rules + object_store: future work)
├── ursa-py/      # PyO3 bindings. Thin: plan builders, collect(), Arrow FFI.
└── python/ursa/  # Python package: dialect, frames, IO, graph verbs, stats.

Governing rule: Arrow at the boundaries, index in the middle. Every kernel takes Arrow columns plus a shared topology index in, and hands Arrow arrays out.

Why DataFusion (not the Polars crates): extensibility is the designed use case — graph ops must be first-class citizens of one query plan, not coordinated by a "traffic cop" around a closed planner. The accepted cost is that we own a Polars-shaped expression frontend; it is deliberately quarantined at one seam — python/ursa/_expr.py builds the dialect, and ursa-plan/src/query.rs lowers it (today a small JSON column IR + comparison filters) to a DataFusion plan.

Develop

The Python side is managed with uv; linting and formatting use ruff and type-checking uses ty.

# Rust core: real kernels, fast to build/test (arrow + rayon only)
cargo test -p ursa-core

# Whole workspace (compiles DataFusion; slower)
cargo check

# Python: uv creates the venv, builds the maturin extension, and installs the
# dev dependency group in one step.
uv sync

uv run pytest                     # pure-Python tests + native-kernel tests
uv run ruff check .               # lint
uv run ruff format .              # format
uv run ty check                   # type-check

uv run rebuilds the native extension as needed, so editing Rust and re-running uv run pytest picks up the change. Requirements: Python ≥ 3.10 and uv; the Rust toolchain is pinned in rust-toolchain.toml (rustup installs it automatically), so local cargo clippy uses the exact same lint set as CI.

Roadmap

The engine foundation is in place — every collect() is one DataFusion plan with custom graph logical nodes — and many features have fanned out on top of it: node-valued algorithms (pagerank, degree, connected_components, triangle_count, clustering_coefficient, closeness, betweenness, label_propagation, louvain), composed pipelines, scan_edges/scan_nodes sources, node-attribute enrichment (in-memory or file-backed tables joined by id), neighbors().agg() over numeric and string attributes, the traversals hop() and shortest_path() (each its own first-class logical node returning an EdgeFrame, on a shared single-source BFS kernel family) plus random_walk(), the eager whole-graph stats density / avg_path_length / diameter and the one-row describe, object-storage scans (s3:// / gs:// / az:// via object_store, with storage_options), and sink_parquet/sink_csv egress.

Weighted algorithms are live across the board: weight= is a per-operation expression over edge columns (weight=ur.col("amount") * ur.col("fx")), evaluated to an f64 per edge and gathered per CSR slot via the edge_ids permutation. Weighted PageRank, shortest_path (Dijkstra), closeness, betweenness (Dijkstra-Brandes), and louvain all ship.

A few relational verbs are modelled in the plan (so they compose and show in .explain()) but are not yet executable and raise a clear error when collected: sample, group_by().agg, join, and schema.

Next, in rough priority order:

  1. Scale-oriented kernel refinements: the direction-optimizing (top-down/bottom-up) BFS switch, and delta-stepping for the (already shipping, Dijkstra-based) weighted SSSP.
  2. Optimizer rules — push node-set filters before traversal, fuse neighbors().agg into a segmented CSR reduction. (The topology index is now built once and shared across ops over a frame — the index-preservation contract — which is the seam these rules register on.)
  3. Breadth — a published docs site, and broadening the benchmark coverage. The cross-library benchmark flywheel is live in benchmarks/ — a typer/rich CLI that races Ursa against NetworkX, rustworkx, and igraph across nine algorithms, separates cold end-to-end from warm kernel time, cross-checks every result against the NetworkX oracle, and writes raw rows to Parquet so leaderboards and gap-hunting are re-renderable downstream steps. (String/UUID node ids alongside int64 are already supported, auto-detected from the column type.)

License

MIT OR Apache-2.0.

Project links

Data verified by PyPI on
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for cl_dixon from gravatar.com cl_dixon

Credits

Author: Ursa contributors

GitHub Statistics

Data verified by PyPI on 
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT OR Apache-2.0

Requires

Python >=3.10

Provides Extra

polars

Tags

graph analytics arrow datafusion polars dataframe

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ursa_graph-0.2.0.tar.gz (148.6 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ursa_graph-0.2.0-cp310-abi3-win_amd64.whl (30.1 MB view details)

Uploaded CPython 3.10+Windows x86-64

ursa_graph-0.2.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (30.1 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ursa_graph-0.2.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (26.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

ursa_graph-0.2.0-cp310-abi3-macosx_11_0_arm64.whl (27.1 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

ursa_graph-0.2.0-cp310-abi3-macosx_10_12_x86_64.whl (29.1 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file ursa_graph-0.2.0.tar.gz.

File metadata

  • Download URL: ursa_graph-0.2.0.tar.gz
  • Upload date:
  • Size: 148.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ursa_graph-0.2.0.tar.gz
Algorithm Hash digest
SHA256 eb3c441452da6384a5a44a6c587abeb48131cd62d2f8042f3d5a5e390ea0737e
MD5 cb4c3a69efc0d2f4472a9ca8a8209eac
BLAKE2b-256 4ade8eeb53767b83b2e3f21a04c15cd4d69051ed65a69ddc0f110b4a8a1ec55a

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.2.0.tar.gz:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.2.0-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: ursa_graph-0.2.0-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 30.1 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ursa_graph-0.2.0-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 025e24f66aa548336eca396762034256cd045229143e4af3649c9d7629a4da61
MD5 a7324c88067076315dcd31a16ea9f069
BLAKE2b-256 7d0d46f26bfd4a58d04e429e1d19b6e97c7ec62bf7de31ba9e08a9572298f257

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.2.0-cp310-abi3-win_amd64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.2.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.2.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 82fee5b7778b5094c131a9f3514ba3525795226fbcf86864ddc4c579519e034b
MD5 098ab6e438892dcf0838bb49f63b7256
BLAKE2b-256 68bb1af69dcd3b0424013cd8e64503c985f8178e8e70bf866871459815ae77e5

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.2.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.2.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.2.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 f5228c2fe22cad156ccbbdace3ed37c7478e7208cd32980cd05320b9140d0fc9
MD5 0f37606a9f6218bcc9567958fd65f66b
BLAKE2b-256 fd9c2e134887b2822b1bcda7d05b0ac272060526e1d459d2a4e8f078662ca42f

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.2.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.2.0-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.2.0-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 52ab1369fad2574a94296826b217ac4332282ed5c0d31cca48e01f2e379aca22
MD5 3e313c4184428f7ea666e43cc5998650
BLAKE2b-256 87efd693d7f74422bab1ad7ee683d2a96dce07f23e806ab99f8ca3f5834cc7bf

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.2.0-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.2.0-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.2.0-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 55e635fe73f717a07a181d0b50c897a7cc9f8509b5d8e2fb00267a6c0184c983
MD5 606377feb22bc7ada3e0239ed6a64400
BLAKE2b-256 b92b2596dd06a6d0ea3bf0c2103e86aacba357b3de48df1d3df1dbc7efbfd3db

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.2.0-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page