Ursa — Polars-shaped dataframes for graph data

Status: v0.1 in progress. An in-memory, single-machine graph analytics library with a dataframe-first API — what Polars is to tabular data, for graphs. The engine foundation is in place: real algorithm kernels, and collect() executing as one DataFusion plan with graph ops as first-class logical nodes. The full design lives in docs/SPEC.md; this README describes what is actually built right now and how the pieces fit.

Ursa is a Rust core (Apache Arrow throughout), a DataFusion query engine with graph operators as first-class plan nodes, and a fluent, Polars-shaped Python expression API. There is no Graph object — an EdgeFrame is the graph, and every operation returns a frame.

import ursa as ur

edges = ur.scan_edges("web-google.csv", src="FromNodeId", dst="ToNodeId")
top = (
    edges.nodes()
    .with_columns(
        pagerank  = ur.pagerank(edges, damping=0.85),
        in_degree = ur.degree(edges, direction="in"),
    )
    .sort("pagerank", descending=True)
    .head(20)
    .collect()          # runs as one DataFusion plan; results are Arrow
)

What works today

The architecture, crate boundaries, and load-bearing seams are all in place, and the parts that prove the design is sound are real and tested end-to-end.

Layer State
ursa-core — CSR topology index + kernels ✅ Real & unit-tested. Dense u32 indexing, lazy-transpose CSR with the edge_ids permutation, and working degree / pagerank (pull-based) / connected_components (union-find) / triangle_count / clustering_coefficient (sorted-adjacency intersection) / bfs (frontier) / closeness / betweenness (Brandes, with source sampling) / label_propagation / louvain (modularity) kernels. Weighted variants are the remaining kernel work.
ursa-plan — DataFusion engine ✅ Unified plan. Each collect() is one DataFusion LogicalPlan — Limit → Sort → Filter → GraphAlgorithmNode — where GraphAlgorithmNode is a real UserDefinedLogicalNode lowered to GraphAlgorithmExec by our own ExtensionPlanner. Graph ops are first-class citizens of the plan (not orchestrated from outside), which is where future optimizer rules register. A DataFusion scan reads Parquet/CSV edge/node files, local or from object storage (s3:// / gs:// / az://), with the column projection pushed into the file.
ursa-py — PyO3 bindings ✅ Wired. Arrow in/out zero-copy (PyCapsule), GIL released during compute.
Python dialect + collect() ✅ Live & executing. The Polars-shaped expression/plan builder, plus collect() for a standalone algorithm, a composed with_columns(...).filter(...).sort(...).head(n) pipeline, node-attribute enrichment (in-memory or scan_nodes file-backed tables joined by id, ur.col("attr") usable in filter/sort), neighbors().agg() over numeric and string attributes, the traversals hop() and shortest_path() (first-class HopNode/ShortestPathNode returning EdgeFrames) plus random_walk() (a RandomWalkNode returning a (walk_id, step, node) frame), and the whole-graph stats describe() / density() / avg_path_length() / diameter(). Over in-memory or scan_edges/scan_nodes sources — local files or object storage (s3:// / gs:// / az://, with storage_options={...}). 
import ursa as ur, pyarrow as pa

edges = ur.from_arrow(pa.table({"s": [1, 2, 3, 0], "d": [0, 0, 0, 1]}), src="s", dst="d")

# Composed pipeline — runs through DataFusion end to end:
(
    edges.nodes()
    .with_columns(pr=ur.pagerank(edges), indeg=ur.degree(edges, direction="in"))
    .filter(ur.col("indeg") > 0)
    .sort("pr", descending=True)
    .head(10)
    .collect()
    .to_polars()
)

# ...or straight from a file (Parquet/CSV, projection pushed into the scan):
ur.pagerank(ur.scan_edges("edges.parquet", src="s", dst="d")).collect().to_polars()

# Node ids may be int64 (the fast path) or strings (e.g. UUIDs) — auto-detected
# from the column type; results come back keyed by the original ids:
str_edges = ur.from_arrow(pa.table({"s": ["u1", "u1", "u2"], "d": ["u2", "u3", "u3"]}), src="s", dst="d")
ur.pagerank(str_edges).collect().to_polars()   # id column is Utf8

Architecture

ursa/
├── ursa-core/    # Topology (CSR) + algorithm kernels. Pure Rust: arrow + rayon.
│                 # NO DataFusion dependency. Independently testable.
├── ursa-plan/    # DataFusion extensions: custom logical node + ExecutionPlan
│                 # (-> ursa-core), the query builder, scan/session plumbing.
│                 # The ONE seam where our dialect lowers to DataFusion.
│                 # (optimizer rules + object_store: future work)
├── ursa-py/      # PyO3 bindings. Thin: plan builders, collect(), Arrow FFI.
└── python/ursa/  # Python package: dialect, frames, IO, graph verbs, stats.

Governing rule: Arrow at the boundaries, index in the middle. Every kernel takes Arrow columns plus a shared topology index in, and hands Arrow arrays out.

Why DataFusion (not the Polars crates): extensibility is the designed use case — graph ops must be first-class citizens of one query plan, not coordinated by a "traffic cop" around a closed planner. The accepted cost is that we own a Polars-shaped expression frontend; it is deliberately quarantined at one seam — python/ursa/_expr.py builds the dialect, and ursa-plan/src/query.rs lowers it (today a small JSON column IR + comparison filters) to a DataFusion plan.

Develop

The Python side is managed with uv; linting and formatting use ruff and type-checking uses ty.

# Rust core: real kernels, fast to build/test (arrow + rayon only)
cargo test -p ursa-core

# Whole workspace (compiles DataFusion; slower)
cargo check

# Python: uv creates the venv, builds the maturin extension, and installs the
# dev dependency group in one step.
uv sync

uv run pytest                     # pure-Python tests + native-kernel tests
uv run ruff check .               # lint
uv run ruff format .              # format
uv run ty check                   # type-check

uv run rebuilds the native extension as needed, so editing Rust and re-running uv run pytest picks up the change. Requirements: Python ≥ 3.10 and uv; the Rust toolchain is pinned in rust-toolchain.toml (rustup installs it automatically), so local cargo clippy uses the exact same lint set as CI.

Roadmap

The engine foundation is in place — every collect() is one DataFusion plan with custom graph logical nodes — and many features have fanned out on top of it: node-valued algorithms (pagerank, degree, connected_components, triangle_count, clustering_coefficient, closeness, betweenness, label_propagation, louvain), composed pipelines, scan_edges/scan_nodes sources, node-attribute enrichment (in-memory or file-backed tables joined by id), neighbors().agg() over numeric and string attributes, the traversals hop() and shortest_path() (each its own first-class logical node returning an EdgeFrame, on a shared single-source BFS kernel family) plus random_walk(), the eager whole-graph stats density / avg_path_length / diameter and the one-row describe, object-storage scans (s3:// / gs:// / az:// via object_store, with storage_options), and sink_parquet/sink_csv egress.

Weighted algorithms are live across the board: weight= is a per-operation expression over edge columns (weight=ur.col("amount") * ur.col("fx")), evaluated to an f64 per edge and gathered per CSR slot via the edge_ids permutation. Weighted PageRank, shortest_path (Dijkstra), closeness, betweenness (Dijkstra-Brandes), and louvain all ship.

Next, in rough priority order:

  1. The direction-optimizing (top-down/bottom-up) BFS switch and weighted SSSP via delta-stepping, for scale.
  2. Optimizer rules — push node-set filters before traversal, fuse neighbors().agg into a segmented CSR reduction. (The topology index is now built once and shared across ops over a frame — the index-preservation contract — which is the seam these rules register on.)
  3. Breadth — benchmarks vs NetworkX/rustworkx/igraph, a published docs site. (String/UUID node ids alongside int64 are already supported, auto-detected from the column type.)

License

MIT OR Apache-2.0.

Project links

Data verified by PyPI on
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.2.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for cl_dixon from gravatar.com cl_dixon

Credits

Author: Ursa contributors

GitHub Statistics

Data verified by PyPI on 
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT OR Apache-2.0

Requires

Python >=3.10

Provides Extra

polars

Tags

graph analytics arrow datafusion polars dataframe

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ursa_graph-0.1.1.tar.gz (117.1 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ursa_graph-0.1.1-cp310-abi3-win_amd64.whl (20.4 MB view details)

Uploaded CPython 3.10+Windows x86-64

ursa_graph-0.1.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (20.8 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ursa_graph-0.1.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (18.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

ursa_graph-0.1.1-cp310-abi3-macosx_11_0_arm64.whl (18.7 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

ursa_graph-0.1.1-cp310-abi3-macosx_10_12_x86_64.whl (20.0 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file ursa_graph-0.1.1.tar.gz.

File metadata

  • Download URL: ursa_graph-0.1.1.tar.gz
  • Upload date:
  • Size: 117.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for ursa_graph-0.1.1.tar.gz
Algorithm Hash digest
SHA256 a1a5df634f59ff70f0008764d7d7626e7661beb2b026d8f5720dbd5f11cbb654
MD5 8e087b9692940dc7824ba7611bfa5e7f
BLAKE2b-256 acec10db50bf8e5d1130e247703a547f1f4f4241c4a6b4c550e771699861de43

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.1.tar.gz:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.1.1-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: ursa_graph-0.1.1-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.4 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for ursa_graph-0.1.1-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 8c17e67bf15a511234a1d32baf5e56ab8b6406d547f4f03ff552509eaa2b7c51
MD5 b2a9c62d25f672242f6ecb12217f66c2
BLAKE2b-256 596add9065881682a7f8adc67a89e991e00b09fde65ceab5c82acbd501d30790

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.1-cp310-abi3-win_amd64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.1.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.1.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 2efc3afd1e19bbdf6c14b4e1f16a6fc897f812cc56edb0ee270b67a046e1e382
MD5 471ca24ef33d290231a2b10c2b7cff40
BLAKE2b-256 6aa949a9b1d03a8fea2eb0302f3ebe1086a7e14ca709b481471458f31dbc40d4

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.1-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.1.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.1.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 419206cabcea8b3981fe21a25bfd526ae8afe4fcb4fcb307f1b6c59cb223a494
MD5 d97fef2fcba5939c3bb09cf4e44f3efe
BLAKE2b-256 2745bdebe8fba90417c6894ad22eb16a9530e72b6f4bc99bce9341e407cb564e

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.1-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.1.1-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.1.1-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 730fdad46a99d5292dcaf8f3586235ee3dfb4c7aecac7fcd5a5c5c1357f6a539
MD5 7b255834e4e23bfd3ec15e3c5a9fa9c3
BLAKE2b-256 9e880690424a178057ef214fab84ffb6b74b4958ce911fb38cf4d59e47664b54

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.1-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.1.1-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.1.1-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 33656ac3828aeb832f856d77b9f645853b33d6bb3343f922658cdd0cfd1a49d7
MD5 0efa712c4f0906410a483b3ed69abdb7
BLAKE2b-256 0497088cd4315fbf76fe728c4d8b05a3cda200ec0968a0df8dadde77ac2d5607

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.1-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page