Ursa — Polars-shaped dataframes for graph data

Status: v0.1 in progress. An in-memory, single-machine graph analytics library with a dataframe-first API — what Polars is to tabular data, for graphs. The engine foundation is in place: real algorithm kernels, and collect() executing as one DataFusion plan with graph ops as first-class logical nodes. The full design lives in docs/SPEC.md; this README describes what is actually built right now and how the pieces fit.

Ursa is a Rust core (Apache Arrow throughout), a DataFusion query engine with graph operators as first-class plan nodes, and a fluent, Polars-shaped Python expression API. There is no Graph object — an EdgeFrame is the graph, and every operation returns a frame.

import ursa as ur

edges = ur.scan_edges("web-google.csv", src="FromNodeId", dst="ToNodeId")
top = (
    edges.nodes()
    .with_columns(
        pagerank  = ur.pagerank(edges, damping=0.85),
        in_degree = ur.degree(edges, direction="in"),
    )
    .sort("pagerank", descending=True)
    .head(20)
    .collect()          # runs as one DataFusion plan; results are Arrow
)

What works today

The architecture, crate boundaries, and load-bearing seams are all in place, and the parts that prove the design is sound are real and tested end-to-end.

Layer State
ursa-core — CSR topology index + kernels ✅ Real & unit-tested. Dense u32 indexing, lazy-transpose CSR with the edge_ids permutation, and working degree / pagerank (pull-based) / connected_components (union-find) / triangle_count / clustering_coefficient (sorted-adjacency intersection) / bfs (frontier) / closeness / betweenness (Brandes, with source sampling) / label_propagation / louvain (modularity) kernels. Weighted variants are the remaining kernel work.
ursa-plan — DataFusion engine ✅ Unified plan. Each collect() is one DataFusion LogicalPlan — Limit → Sort → Filter → GraphAlgorithmNode — where GraphAlgorithmNode is a real UserDefinedLogicalNode lowered to GraphAlgorithmExec by our own ExtensionPlanner. Graph ops are first-class citizens of the plan (not orchestrated from outside), which is where future optimizer rules register. A DataFusion scan reads Parquet/CSV edge/node files, local or from object storage (s3:// / gs:// / az://), with the column projection pushed into the file.
ursa-py — PyO3 bindings ✅ Wired. Arrow in/out zero-copy (PyCapsule), GIL released during compute.
Python dialect + collect() ✅ Live & executing. The Polars-shaped expression/plan builder, plus collect() for a standalone algorithm, a composed with_columns(...).filter(...).sort(...).head(n) pipeline, node-attribute enrichment (in-memory or scan_nodes file-backed tables joined by id, ur.col("attr") usable in filter/sort), neighbors().agg() over numeric and string attributes, the traversals hop() and shortest_path() (first-class HopNode/ShortestPathNode returning EdgeFrames) plus random_walk() (a RandomWalkNode returning a (walk_id, step, node) frame), and the whole-graph stats describe() / density() / avg_path_length() / diameter(). Over in-memory or scan_edges/scan_nodes sources — local files or object storage (s3:// / gs:// / az://, with storage_options={...}). 
import ursa as ur, pyarrow as pa

edges = ur.from_arrow(pa.table({"s": [1, 2, 3, 0], "d": [0, 0, 0, 1]}), src="s", dst="d")

# Composed pipeline — runs through DataFusion end to end:
(
    edges.nodes()
    .with_columns(pr=ur.pagerank(edges), indeg=ur.degree(edges, direction="in"))
    .filter(ur.col("indeg") > 0)
    .sort("pr", descending=True)
    .head(10)
    .collect()
    .to_polars()
)

# ...or straight from a file (Parquet/CSV, projection pushed into the scan):
ur.pagerank(ur.scan_edges("edges.parquet", src="s", dst="d")).collect().to_polars()

# Node ids may be int64 (the fast path) or strings (e.g. UUIDs) — auto-detected
# from the column type; results come back keyed by the original ids:
str_edges = ur.from_arrow(pa.table({"s": ["u1", "u1", "u2"], "d": ["u2", "u3", "u3"]}), src="s", dst="d")
ur.pagerank(str_edges).collect().to_polars()   # id column is Utf8

Architecture

ursa/
├── ursa-core/    # Topology (CSR) + algorithm kernels. Pure Rust: arrow + rayon.
│                 # NO DataFusion dependency. Independently testable.
├── ursa-plan/    # DataFusion extensions: custom logical node + ExecutionPlan
│                 # (-> ursa-core), the query builder, scan/session plumbing.
│                 # The ONE seam where our dialect lowers to DataFusion.
│                 # (optimizer rules + object_store: future work)
├── ursa-py/      # PyO3 bindings. Thin: plan builders, collect(), Arrow FFI.
└── python/ursa/  # Python package: dialect, frames, IO, graph verbs, stats.

Governing rule: Arrow at the boundaries, index in the middle. Every kernel takes Arrow columns plus a shared topology index in, and hands Arrow arrays out.

Why DataFusion (not the Polars crates): extensibility is the designed use case — graph ops must be first-class citizens of one query plan, not coordinated by a "traffic cop" around a closed planner. The accepted cost is that we own a Polars-shaped expression frontend; it is deliberately quarantined at one seam — python/ursa/_expr.py builds the dialect, and ursa-plan/src/query.rs lowers it (today a small JSON column IR + comparison filters) to a DataFusion plan.

Develop

The Python side is managed with uv; linting and formatting use ruff and type-checking uses ty.

# Rust core: real kernels, fast to build/test (arrow + rayon only)
cargo test -p ursa-core

# Whole workspace (compiles DataFusion; slower)
cargo check

# Python: uv creates the venv, builds the maturin extension, and installs the
# dev dependency group in one step.
uv sync

uv run pytest                     # pure-Python tests + native-kernel tests
uv run ruff check .               # lint
uv run ruff format .              # format
uv run ty check                   # type-check

uv run rebuilds the native extension as needed, so editing Rust and re-running uv run pytest picks up the change. Requirements: Python ≥ 3.10 and uv; the Rust toolchain is pinned in rust-toolchain.toml (rustup installs it automatically), so local cargo clippy uses the exact same lint set as CI.

Roadmap

The engine foundation is in place — every collect() is one DataFusion plan with custom graph logical nodes — and many features have fanned out on top of it: node-valued algorithms (pagerank, degree, connected_components, triangle_count, clustering_coefficient, closeness, betweenness, label_propagation, louvain), composed pipelines, scan_edges/scan_nodes sources, node-attribute enrichment (in-memory or file-backed tables joined by id), neighbors().agg() over numeric and string attributes, the traversals hop() and shortest_path() (each its own first-class logical node returning an EdgeFrame, on a shared single-source BFS kernel family) plus random_walk(), the eager whole-graph stats density / avg_path_length / diameter and the one-row describe, object-storage scans (s3:// / gs:// / az:// via object_store, with storage_options), and sink_parquet/sink_csv egress.

Weighted algorithms are live across the board: weight= is a per-operation expression over edge columns (weight=ur.col("amount") * ur.col("fx")), evaluated to an f64 per edge and gathered per CSR slot via the edge_ids permutation. Weighted PageRank, shortest_path (Dijkstra), closeness, betweenness (Dijkstra-Brandes), and louvain all ship.

Next, in rough priority order:

  1. The direction-optimizing (top-down/bottom-up) BFS switch and weighted SSSP via delta-stepping, for scale.
  2. Optimizer rules — push node-set filters before traversal, fuse neighbors().agg into a segmented CSR reduction. (The topology index is now built once and shared across ops over a frame — the index-preservation contract — which is the seam these rules register on.)
  3. Breadth — benchmarks vs NetworkX/rustworkx/igraph, a published docs site. (String/UUID node ids alongside int64 are already supported, auto-detected from the column type.)

License

MIT OR Apache-2.0.

Project links

Data verified by PyPI on
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.2.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for cl_dixon from gravatar.com cl_dixon

Credits

Author: Ursa contributors

GitHub Statistics

Data verified by PyPI on 
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT OR Apache-2.0

Requires

Python >=3.10

Provides Extra

polars

Tags

graph analytics arrow datafusion polars dataframe

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

ursa_graph-0.1.0-cp310-abi3-win_amd64.whl (20.4 MB view details)

Uploaded CPython 3.10+Windows x86-64

ursa_graph-0.1.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (20.8 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ x86-64

ursa_graph-0.1.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (18.0 MB view details)

Uploaded CPython 3.10+manylinux: glibc 2.17+ ARM64

ursa_graph-0.1.0-cp310-abi3-macosx_11_0_arm64.whl (18.7 MB view details)

Uploaded CPython 3.10+macOS 11.0+ ARM64

ursa_graph-0.1.0-cp310-abi3-macosx_10_12_x86_64.whl (20.0 MB view details)

Uploaded CPython 3.10+macOS 10.12+ x86-64

File details

Details for the file ursa_graph-0.1.0-cp310-abi3-win_amd64.whl.

File metadata

  • Download URL: ursa_graph-0.1.0-cp310-abi3-win_amd64.whl
  • Upload date:
  • Size: 20.4 MB
  • Tags: CPython 3.10+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.14

File hashes

Hashes for ursa_graph-0.1.0-cp310-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 c98f8ff8c08a81aefe98f18bc415f95fd75ba42fbe4d39801937625afa8b4655
MD5 5afe456caeb1ad45f545a5f179e87f45
BLAKE2b-256 6a5cbcd85af1258c06bdc22b62850e1b521bb95bbc2b12497fdf4155bc78690e

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.0-cp310-abi3-win_amd64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.1.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.1.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 6a2a99aa3cac208af5f6f7be3ca8bf522419d066b286495eaae326bd7a1155f4
MD5 3db171dcd7fdc1000e01a46c91327a86
BLAKE2b-256 66d25371db73b8da23996eef73289b89754efe6e3f2829c743e6125371c78b24

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.1.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.1.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 b7225ca95fff3e87987c89d2c311bc037ff79427f7bb93d45b5d7951e1b3e4d7
MD5 136857d320f7ef3aa198f0ab88fea0d3
BLAKE2b-256 0db8c62348d210c6c3f5434fbdc32739f865d0eb1cb6164b23dcf3065a180e8c

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.0-cp310-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.1.0-cp310-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.1.0-cp310-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 ebc8c6aed07e3f1ac80af666187fe8ffd2d23078292e7e35b0e58b0f575b6bd5
MD5 0083406671835501edb5aa8a0931079a
BLAKE2b-256 008071b5c79d4b771197cc831777f7a392cf411254d4bff6c25d86061866e0c8

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.0-cp310-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ursa_graph-0.1.0-cp310-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for ursa_graph-0.1.0-cp310-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 ce446823ebafcfc901a60a43e232cd16dd07b9cbcc8b86897f0e911e6a43402c
MD5 e7c8c16c3b36048ff4b063e9883fc876
BLAKE2b-256 1c184d394287ccb25c81b8dffcea0d3fdb75536875572bdae09f29772a98a982

See more details on using hashes here.

Provenance

The following attestation bundles were made for ursa_graph-0.1.0-cp310-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on cldixon/ursa

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page