Skip to main content

Orso

Orso is a shared DataFrame library for Opteryx and Mabel.

PyPI Latest Release Downloads codecov Documentation

Overview

Orso is not intended to compete with Polars or Pandas (or your favorite bear DataFrame technology), instead it is developed as a common layer for Mabel and Opteryx.

Key Use Cases:

  • In Opteryx, Orso provides most of the database Cursor functionality
  • In Mabel, Orso provides the data schema and validation functionality

Orso DataFrames are row-based, driven by their initial target use-case as the WAL for Mabel and Cursor for Opteryx. Each row in an Orso DataFrame can be quickly converted to a Tuple of values, a Dictionary, or a byte representation.

Installation

Install Orso from PyPI:

pip install orso

Quick Start

Creating a DataFrame

import orso

# Create from list of dictionaries
df = orso.DataFrame([
    {'name': 'Alice', 'age': 30, 'city': 'New York'},
    {'name': 'Bob', 'age': 25, 'city': 'San Francisco'},
    {'name': 'Charlie', 'age': 35, 'city': 'Chicago'}
])

print(f"Created DataFrame with {df.rowcount} rows and {df.columncount} columns")

Displaying Data

# Display the DataFrame
print(df.display())

# Convert to different formats
arrow_table = df.arrow()  # PyArrow Table
pandas_df = df.pandas()   # Pandas DataFrame

Working with Schema

# Access column names
print("Columns:", df.column_names)

# Access schema information  
print("Schema:", df.schema)

Converting Between Formats

# From PyArrow
import pyarrow as pa
arrow_table = pa.table({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})
orso_df = orso.DataFrame.from_arrow(arrow_table)

# To Pandas
pandas_df = orso_df.pandas()

# To and from draken (opteryx-core's columnar engine), when draken is importable
morsel = orso_df.to_draken()
orso_df = orso.DataFrame.from_draken(morsel)

Features

  • Lightweight: Minimal overhead for tabular data operations
  • Row-based: Optimized for row-oriented operations
  • Interoperable: Easy conversion to/from PyArrow, Pandas
  • Schema-aware: Built-in data validation and type checking
  • Fast serialization: Efficient conversion to bytes, tuples, and dictionaries

API Reference

DataFrame Class

The main DataFrame class provides the following key methods:

  • DataFrame(dictionaries=None, *, rows=None, schema=None) - Constructor
  • display(limit=5, colorize=True, show_types=True) - Pretty print the DataFrame
  • arrow(size=None) - Convert to PyArrow Table
  • pandas(size=None) - Convert to Pandas DataFrame
  • from_arrow(tables) - Create DataFrame from PyArrow Table(s)
  • fetchall() - Get all rows as list of Row objects
  • collect() - Materialize the DataFrame
  • append(other) - Append another DataFrame
  • distinct() - Get unique rows

Properties

  • rowcount - Number of rows
  • columncount - Number of columns
  • column_names - List of column names
  • schema - Schema information

Development

Building from Source

# Clone the repository
git clone https://github.com/mabel-dev/orso.git
cd orso

# Install dependencies
pip install -r requirements.txt
pip install -r tests/requirements.txt

# Build the native extension (needs a C++20 compiler; nanobind comes from pip)
make compile

# Run tests
make test

Contributing

Orso is part of the Mabel ecosystem. Contributions are welcome! Please ensure:

  1. All tests pass: make test
  2. Code follows the project style: make lint
  3. New features include appropriate tests
  4. Documentation is updated for API changes

Performance Benchmarking

Orso has a micro-benchmark harness in benchmarks/ covering construction, append, Row, Arrow conversion, iteration, display and type parsing. Each case is warmed up and timed over several rounds; compare the minimum against a saved baseline:

# Run the full suite (or a subset with -k <substring>)
python benchmarks/bench.py

# Record a baseline, change code, rebuild the extensions, then compare
python benchmarks/bench.py -o benchmarks/baseline_0.0.246.json
python setup.py build_ext --inplace
python benchmarks/bench.py -c benchmarks/baseline_0.0.246.json

make bench and make bench-baseline wrap the same commands.

Building from source

make compile builds the native layer (orso._native, a nanobind extension over draken headers vendored into native/). There is no Cython any more. The vendored headers are copied from an opteryx-core checkout beside this one with make sync-draken and verified with make check-draken; never edit them in place. See DESIGN.md.

License

License

Orso is licensed under Apache 2.0 unless explicitly indicated otherwise.

Status

Status

Orso is in beta. Beta means different things to different people, to us, being beta means:

  • Interfaces are generally stable but may still have breaking changes
  • Unit tests are not reliable enough to capture breaks to functionality
  • Bugs are likely to exist in edge cases
  • Code may not be tuned for performance

As such, we really don't recommend using Orso in critical applications.

  • Opteryx - SQL query engine for data files
  • Mabel - Data processing framework

Metadata

Release files for orso 0.1.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for orso 0.1.5
File Size Uploaded Provenance Actions
orso-0.1.5.tar.gz 439.5 kB Without provenance Details for orso-0.1.5.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for orso 0.1.5
File Actions
orso-0.1.5-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.14 CPython 3.14 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details for orso-0.1.5-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.5-cp314-cp314-macosx_10_15_universal2.whl CPython 3.14 CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.5-cp314-cp314-macosx_10_15_universal2.whl
orso-0.1.5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.13 CPython 3.13 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details for orso-0.1.5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.5-cp313-cp313-macosx_10_15_universal2.whl CPython 3.13 CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.5-cp313-cp313-macosx_10_15_universal2.whl
orso-0.1.5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.5-cp312-cp312-macosx_10_15_universal2.whl CPython 3.12 CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.5-cp312-cp312-macosx_10_15_universal2.whl
orso-0.1.5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.5-cp311-cp311-macosx_10_15_universal2.whl CPython 3.11 CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.5-cp311-cp311-macosx_10_15_universal2.whl
orso-0.1.5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details for orso-0.1.5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.5-cp310-cp310-macosx_10_15_universal2.whl CPython 3.10 CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.5-cp310-cp310-macosx_10_15_universal2.whl

Total release size: 13.2 MB

Release files / orso-0.1.5.tar.gz

Download URL orso-0.1.5.tar.gz
Size 439.5 kB
Tags Source
SHA-256 checksum
How to use checksums
b4c5c96c1afac728c4638c2b26f2a53dfe605e239dbaf705e511274142383a8e
BLAKE2b-256 checksum
How to use checksums
252fc3be165bc4e64763ec2d9e5ffdc3ddb7e10deab420a4506deb90885c2da8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.5-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.14 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
1abd12878c1ad8abb1fe8bcf3431e4979a09e37a396ca24fe646c3b94a62be9a
BLAKE2b-256 checksum
How to use checksums
c3bc7b8c20ff7acd01f420f6101592e23ab7c36b458d7780e338107f6b0615b5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp314-cp314-macosx_10_15_universal2.whl

Download URL orso-0.1.5-cp314-cp314-macosx_10_15_universal2.whl
Size 572.2 kB
Tags CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
7fc044bed8b4bac96943baec2b8e71adee0c25c03bcbcedf98ecdd61775e51e5
BLAKE2b-256 checksum
How to use checksums
db8a999d8df6a4e688251815e9720e459cd50c1e673b35bc9dd13cd9eab22960
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.5-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.13 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
5ed42b08d5e5bf8acbb10e4874bfaa175fd041f60fb8d86bf365f06cf9db4942
BLAKE2b-256 checksum
How to use checksums
458ba4a0f295b67961d722e6505f8def1f11fecf5b442420cfae344fa9621a8d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp313-cp313-macosx_10_15_universal2.whl

Download URL orso-0.1.5-cp313-cp313-macosx_10_15_universal2.whl
Size 572.0 kB
Tags CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
f92ef478c8fc669e469e0fd3ed9c4b9d8cddc3e1d76d9abb5073a802d2c70c9a
BLAKE2b-256 checksum
How to use checksums
7f82c7cbc6f18ec5568453eeec79bd6f4915228b5d8a324116fd2664c040b610
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.5-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.12 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
8989c12d8bb8cef38626c8ef8073ac010f291cf5fd3fb41624a72d02da09be47
BLAKE2b-256 checksum
How to use checksums
a9f01bb038325021fe683b8faf61be926c8f7cf434c2c4a8200096a33e2fb41d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp312-cp312-macosx_10_15_universal2.whl

Download URL orso-0.1.5-cp312-cp312-macosx_10_15_universal2.whl
Size 571.9 kB
Tags CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
a8339d71de5cd5777a46b9cd7f22bc9a4aa4567fbe1b7c13ffa28733fd9ee1e2
BLAKE2b-256 checksum
How to use checksums
5b084f4f1f2145191651234ac79646dfc6eb38e80fb7cf71335e96a01c52eb5e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.5-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.11 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
dd4a315091b42734f9f895926b7331943eb9fbd9f427c6bf48db2e59d4c81b3b
BLAKE2b-256 checksum
How to use checksums
b0c3ec36318ea8f45c5fda023372be7d842215617c67fe7d6eb83fbea7621735
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp311-cp311-macosx_10_15_universal2.whl

Download URL orso-0.1.5-cp311-cp311-macosx_10_15_universal2.whl
Size 567.3 kB
Tags CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
78e0030f308334cb5f683a4ed55a20aadaf5da56bea62a876b3d4b26d0f9ba91
BLAKE2b-256 checksum
How to use checksums
3b3610014de1f65c70788ea3c39b0b0b2d9afd8f39c3fa53f6ef2869f6503d1e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.10 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
1f829d3aa8f968c1932ed19f74a125b99bfddcb8a67988adb92e3f7825082dc1
BLAKE2b-256 checksum
How to use checksums
08245e4b0f8963798ac4cd5490287f4488cba1db3aa478c7624e8d3bc4308693
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.5-cp310-cp310-macosx_10_15_universal2.whl

Download URL orso-0.1.5-cp310-cp310-macosx_10_15_universal2.whl
Size 568.1 kB
Tags CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
7a471bd48650f4e3775c5695df487a15414bb321db524e0c8b8be19fd0375838
BLAKE2b-256 checksum
How to use checksums
5e69a2524a2eb7767d06f0e3a991a42a91518a99316cad185773094893b1b327
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for orso 0.1.5

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.5 This release

11 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page