Skip to main content

Orso

Orso is a shared DataFrame library for Opteryx and Mabel.

PyPI Latest Release Downloads codecov Documentation

Overview

Orso is not intended to compete with Polars or Pandas (or your favorite bear DataFrame technology), instead it is developed as a common layer for Mabel and Opteryx.

Key Use Cases:

  • In Opteryx, Orso provides most of the database Cursor functionality
  • In Mabel, Orso provides the data schema and validation functionality

Orso DataFrames are row-based, driven by their initial target use-case as the WAL for Mabel and Cursor for Opteryx. Each row in an Orso DataFrame can be quickly converted to a Tuple of values, a Dictionary, or a byte representation.

Installation

Install Orso from PyPI:

pip install orso

Quick Start

Creating a DataFrame

import orso

# Create from list of dictionaries
df = orso.DataFrame([
    {'name': 'Alice', 'age': 30, 'city': 'New York'},
    {'name': 'Bob', 'age': 25, 'city': 'San Francisco'},
    {'name': 'Charlie', 'age': 35, 'city': 'Chicago'}
])

print(f"Created DataFrame with {df.rowcount} rows and {df.columncount} columns")

Displaying Data

# Display the DataFrame
print(df.display())

# Convert to different formats
arrow_table = df.arrow()  # PyArrow Table
pandas_df = df.pandas()   # Pandas DataFrame

Working with Schema

# Access column names
print("Columns:", df.column_names)

# Access schema information  
print("Schema:", df.schema)

Converting Between Formats

# From PyArrow
import pyarrow as pa
arrow_table = pa.table({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})
orso_df = orso.DataFrame.from_arrow(arrow_table)

# To Pandas
pandas_df = orso_df.pandas()

# To and from draken (opteryx-core's columnar engine), when draken is importable
morsel = orso_df.to_draken()
orso_df = orso.DataFrame.from_draken(morsel)

Features

  • Lightweight: Minimal overhead for tabular data operations
  • Row-based: Optimized for row-oriented operations
  • Interoperable: Easy conversion to/from PyArrow, Pandas
  • Schema-aware: Built-in data validation and type checking
  • Fast serialization: Efficient conversion to bytes, tuples, and dictionaries

API Reference

DataFrame Class

The main DataFrame class provides the following key methods:

  • DataFrame(dictionaries=None, *, rows=None, schema=None) - Constructor
  • display(limit=5, colorize=True, show_types=True) - Pretty print the DataFrame
  • arrow(size=None) - Convert to PyArrow Table
  • pandas(size=None) - Convert to Pandas DataFrame
  • from_arrow(tables) - Create DataFrame from PyArrow Table(s)
  • fetchall() - Get all rows as list of Row objects
  • collect() - Materialize the DataFrame
  • append(other) - Append another DataFrame
  • distinct() - Get unique rows

Properties

  • rowcount - Number of rows
  • columncount - Number of columns
  • column_names - List of column names
  • schema - Schema information

Development

Building from Source

# Clone the repository
git clone https://github.com/mabel-dev/orso.git
cd orso

# Install dependencies
pip install -r requirements.txt
pip install -r tests/requirements.txt

# Build the native extension (needs a C++20 compiler; nanobind comes from pip)
make compile

# Run tests
make test

Contributing

Orso is part of the Mabel ecosystem. Contributions are welcome! Please ensure:

  1. All tests pass: make test
  2. Code follows the project style: make lint
  3. New features include appropriate tests
  4. Documentation is updated for API changes

Performance Benchmarking

Orso has a micro-benchmark harness in benchmarks/ covering construction, append, Row, Arrow conversion, iteration, display and type parsing. Each case is warmed up and timed over several rounds; compare the minimum against a saved baseline:

# Run the full suite (or a subset with -k <substring>)
python benchmarks/bench.py

# Record a baseline, change code, rebuild the extensions, then compare
python benchmarks/bench.py -o benchmarks/baseline_0.0.246.json
python setup.py build_ext --inplace
python benchmarks/bench.py -c benchmarks/baseline_0.0.246.json

make bench and make bench-baseline wrap the same commands.

Building from source

make compile builds the native layer (orso._native, a nanobind extension over draken headers vendored into native/). There is no Cython any more. The vendored headers are copied from an opteryx-core checkout beside this one with make sync-draken and verified with make check-draken; never edit them in place. See DESIGN.md.

License

License

Orso is licensed under Apache 2.0 unless explicitly indicated otherwise.

Status

Status

Orso is in beta. Beta means different things to different people, to us, being beta means:

  • Interfaces are generally stable but may still have breaking changes
  • Unit tests are not reliable enough to capture breaks to functionality
  • Bugs are likely to exist in edge cases
  • Code may not be tuned for performance

As such, we really don't recommend using Orso in critical applications.

  • Opteryx - SQL query engine for data files
  • Mabel - Data processing framework

Metadata

Release files for orso 0.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for orso 0.1.1
File Size Uploaded Provenance Actions
orso-0.1.1.tar.gz 435.9 kB Without provenance Details for orso-0.1.1.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for orso 0.1.1
File Actions
orso-0.1.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.14 CPython 3.14 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.1-cp314-cp314-macosx_10_15_universal2.whl CPython 3.14 CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.1-cp314-cp314-macosx_10_15_universal2.whl
orso-0.1.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.13 CPython 3.13 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.1-cp313-cp313-macosx_10_15_universal2.whl CPython 3.13 CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.1-cp313-cp313-macosx_10_15_universal2.whl
orso-0.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.1-cp312-cp312-macosx_10_15_universal2.whl CPython 3.12 CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.1-cp312-cp312-macosx_10_15_universal2.whl
orso-0.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details for orso-0.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.1-cp311-cp311-macosx_10_15_universal2.whl CPython 3.11 CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.1-cp311-cp311-macosx_10_15_universal2.whl
orso-0.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details for orso-0.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.1-cp310-cp310-macosx_10_15_universal2.whl CPython 3.10 CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.1-cp310-cp310-macosx_10_15_universal2.whl

Total release size: 13.2 MB

Release files / orso-0.1.1.tar.gz

Download URL orso-0.1.1.tar.gz
Size 435.9 kB
Tags Source
SHA-256 checksum
How to use checksums
4b97b279a61351e77c205b4c22490d7742ef86baf8d7eba17ccd02923799bb9b
BLAKE2b-256 checksum
How to use checksums
44746aad3379eb8a73317cb4a0335903d841e5bff33433467530b3d3f91e700c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.1-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.14 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
20abd2cea99bf3bff6c8f4244d1e74610cf639987f528c09247b58fd9dee7f5d
BLAKE2b-256 checksum
How to use checksums
5c45459b2dcf571c5eef9d80956837ddadfb6e3a0c3988b68c606e22d033869b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp314-cp314-macosx_10_15_universal2.whl

Download URL orso-0.1.1-cp314-cp314-macosx_10_15_universal2.whl
Size 570.8 kB
Tags CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
a18a1820460f26607b8463b5331992773ad0b3ac0994c08ad9c578899abf227a
BLAKE2b-256 checksum
How to use checksums
7f7772e7558e62713aef49fe6608604d8f6bc327e1dc1a5ff4bcdc3de3bcf39e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.1-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.13 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
a0463cfcb0a5cd11e84c84dd17a1259379b4c78d1ab76af8f5ef025ac990f0bf
BLAKE2b-256 checksum
How to use checksums
62aa9939fde510df0f2a03cbdb61e44ab5986e4c44f6bd67419ed777bfcfcec3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp313-cp313-macosx_10_15_universal2.whl

Download URL orso-0.1.1-cp313-cp313-macosx_10_15_universal2.whl
Size 570.6 kB
Tags CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
8dc92913c376bf550d1a99203d3d6608a5374d0c0da3a378ff22897655fc0f39
BLAKE2b-256 checksum
How to use checksums
6f25b12b21614b8403941059d6334401190ad9047571538c3791e3d9e95f89be
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.12 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
d2def1a072fd799d03ad3d1b326573563d7c7d1bd791a246200f8b15b7f36c28
BLAKE2b-256 checksum
How to use checksums
bd35a1e7c2248379338498dc93c9ac4db6a421080b06b9b476b14760978881f4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp312-cp312-macosx_10_15_universal2.whl

Download URL orso-0.1.1-cp312-cp312-macosx_10_15_universal2.whl
Size 570.5 kB
Tags CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
37ec4d386771a8feeb66fa01ed2f848b59a6c26c13da4ccc3b8c033cc0ed48c9
BLAKE2b-256 checksum
How to use checksums
394e9f3b8f5858ceca653c005a6cbe5ffa3571180b3f5fc772cf9c8eab4ae591
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.11 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
f67bc0c5db310e51f775c6a5c70bb7441b4940d318464a87736c4b9fa9633869
BLAKE2b-256 checksum
How to use checksums
c3705ea8097c5733f54143835d81b599354e036ea023afeff170f62dd6b9086d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp311-cp311-macosx_10_15_universal2.whl

Download URL orso-0.1.1-cp311-cp311-macosx_10_15_universal2.whl
Size 565.9 kB
Tags CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
0eff5300ba430695ca69303aaa20e1f78db2c1994af8550cd345a7c51bc15d3f
BLAKE2b-256 checksum
How to use checksums
b822ad9bed7b65d716ba78295a205d772f9bf582c2acc0611295cfb807480eb2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.10 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
24389f3a795fd08e4d6bbc2672c0482c2c15f64f1be680768fedbcd6f811b46f
BLAKE2b-256 checksum
How to use checksums
e531f27b9cf3e1916d22cbf4f14ed531a97753a6914b57951f0984f47cae2522
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.1-cp310-cp310-macosx_10_15_universal2.whl

Download URL orso-0.1.1-cp310-cp310-macosx_10_15_universal2.whl
Size 566.7 kB
Tags CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
4c24e806d5634088ac0d6f372bdc3edeb9400e802192fdebbfd3e1b43e130ad7
BLAKE2b-256 checksum
How to use checksums
c3b562a22737d1bb93e2bb1198afed4488a9d634c5f7380d6da392f7be07b48d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for orso 0.1.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.1 This release

11 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page