Skip to main content

Orso

Orso is a shared DataFrame library for Opteryx and Mabel.

PyPI Latest Release Downloads codecov Documentation

Overview

Orso is not intended to compete with Polars or Pandas (or your favorite bear DataFrame technology), instead it is developed as a common layer for Mabel and Opteryx.

Key Use Cases:

  • In Opteryx, Orso provides most of the database Cursor functionality
  • In Mabel, Orso provides the data schema and validation functionality

Orso DataFrames are row-based, driven by their initial target use-case as the WAL for Mabel and Cursor for Opteryx. Each row in an Orso DataFrame can be quickly converted to a Tuple of values, a Dictionary, or a byte representation.

Installation

Install Orso from PyPI:

pip install orso

Quick Start

Creating a DataFrame

import orso

# Create from list of dictionaries
df = orso.DataFrame([
    {'name': 'Alice', 'age': 30, 'city': 'New York'},
    {'name': 'Bob', 'age': 25, 'city': 'San Francisco'},
    {'name': 'Charlie', 'age': 35, 'city': 'Chicago'}
])

print(f"Created DataFrame with {df.rowcount} rows and {df.columncount} columns")

Displaying Data

# Display the DataFrame
print(df.display())

# Convert to different formats
arrow_table = df.arrow()  # PyArrow Table
pandas_df = df.pandas()   # Pandas DataFrame

Working with Schema

# Access column names
print("Columns:", df.column_names)

# Access schema information  
print("Schema:", df.schema)

Converting Between Formats

# From PyArrow
import pyarrow as pa
arrow_table = pa.table({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})
orso_df = orso.DataFrame.from_arrow(arrow_table)

# To Pandas
pandas_df = orso_df.pandas()

# To and from draken (opteryx-core's columnar engine), when draken is importable
morsel = orso_df.to_draken()
orso_df = orso.DataFrame.from_draken(morsel)

Features

  • Lightweight: Minimal overhead for tabular data operations
  • Row-based: Optimized for row-oriented operations
  • Interoperable: Easy conversion to/from PyArrow, Pandas
  • Schema-aware: Built-in data validation and type checking
  • Fast serialization: Efficient conversion to bytes, tuples, and dictionaries

API Reference

DataFrame Class

The main DataFrame class provides the following key methods:

  • DataFrame(dictionaries=None, *, rows=None, schema=None) - Constructor
  • display(limit=5, colorize=True, show_types=True) - Pretty print the DataFrame
  • arrow(size=None) - Convert to PyArrow Table
  • pandas(size=None) - Convert to Pandas DataFrame
  • from_arrow(tables) - Create DataFrame from PyArrow Table(s)
  • fetchall() - Get all rows as list of Row objects
  • collect() - Materialize the DataFrame
  • append(other) - Append another DataFrame
  • distinct() - Get unique rows

Properties

  • rowcount - Number of rows
  • columncount - Number of columns
  • column_names - List of column names
  • schema - Schema information

Development

Building from Source

# Clone the repository
git clone https://github.com/mabel-dev/orso.git
cd orso

# Install dependencies
pip install -r requirements.txt
pip install -r tests/requirements.txt

# Build the native extension (needs a C++20 compiler; nanobind comes from pip)
make compile

# Run tests
make test

Contributing

Orso is part of the Mabel ecosystem. Contributions are welcome! Please ensure:

  1. All tests pass: make test
  2. Code follows the project style: make lint
  3. New features include appropriate tests
  4. Documentation is updated for API changes

Performance Benchmarking

Orso has a micro-benchmark harness in benchmarks/ covering construction, append, Row, Arrow conversion, iteration, display and type parsing. Each case is warmed up and timed over several rounds; compare the minimum against a saved baseline:

# Run the full suite (or a subset with -k <substring>)
python benchmarks/bench.py

# Record a baseline, change code, rebuild the extensions, then compare
python benchmarks/bench.py -o benchmarks/baseline_0.0.246.json
python setup.py build_ext --inplace
python benchmarks/bench.py -c benchmarks/baseline_0.0.246.json

make bench and make bench-baseline wrap the same commands.

Building from source

make compile builds the native layer (orso._native, a nanobind extension over draken headers vendored into native/). There is no Cython any more. The vendored headers are copied from an opteryx-core checkout beside this one with make sync-draken and verified with make check-draken; never edit them in place. See DESIGN.md.

License

License

Orso is licensed under Apache 2.0 unless explicitly indicated otherwise.

Status

Status

Orso is in beta. Beta means different things to different people, to us, being beta means:

  • Interfaces are generally stable but may still have breaking changes
  • Unit tests are not reliable enough to capture breaks to functionality
  • Bugs are likely to exist in edge cases
  • Code may not be tuned for performance

As such, we really don't recommend using Orso in critical applications.

  • Opteryx - SQL query engine for data files
  • Mabel - Data processing framework

Metadata

Release files for orso 0.1.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for orso 0.1.2
File Size Uploaded Provenance Actions
orso-0.1.2.tar.gz 438.3 kB Without provenance Details for orso-0.1.2.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for orso 0.1.2
File Actions
orso-0.1.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.14 CPython 3.14 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.2-cp314-cp314-macosx_10_15_universal2.whl CPython 3.14 CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.2-cp314-cp314-macosx_10_15_universal2.whl
orso-0.1.2-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.13 CPython 3.13 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.2-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.2-cp313-cp313-macosx_10_15_universal2.whl CPython 3.13 CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.2-cp313-cp313-macosx_10_15_universal2.whl
orso-0.1.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.2-cp312-cp312-macosx_10_15_universal2.whl CPython 3.12 CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.2-cp312-cp312-macosx_10_15_universal2.whl
orso-0.1.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.2-cp311-cp311-macosx_10_15_universal2.whl CPython 3.11 CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.2-cp311-cp311-macosx_10_15_universal2.whl
orso-0.1.2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.2-cp310-cp310-macosx_10_15_universal2.whl CPython 3.10 CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.2-cp310-cp310-macosx_10_15_universal2.whl

Total release size: 13.2 MB

Release files / orso-0.1.2.tar.gz

Download URL orso-0.1.2.tar.gz
Size 438.3 kB
Tags Source
SHA-256 checksum
How to use checksums
b64af508e80c07d5f11fa3cdd1faca73e0a5f2f41526083985e52391a0f6cec7
BLAKE2b-256 checksum
How to use checksums
2e0464ee98cafb6de01785302d616dcfa34b39843db5a180dd7f4617a6ae63d8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.2-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.14 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
4da8820df5a50290ae398f279302bbcdb9003e3f8f96ae526064c32a1f52a45f
BLAKE2b-256 checksum
How to use checksums
7ea62196d0c5214ea064e0710631514a16d5f502da3d00ee76e889865ca02698
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp314-cp314-macosx_10_15_universal2.whl

Download URL orso-0.1.2-cp314-cp314-macosx_10_15_universal2.whl
Size 571.6 kB
Tags CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
893500895d7c40b24ca29d258b79dd0227527e3331a87ffa7dbde0a99ce67c44
BLAKE2b-256 checksum
How to use checksums
7657ed06e3bf0c7ce44dbd858b3b7b3ca72a08d2bf18b568e6f2757c0dd304d0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.2-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.13 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
51c6827b674e5dd5615166b6cfced5401257afeaf0de3cd4788dd20062897c60
BLAKE2b-256 checksum
How to use checksums
0cc3831244aab41cc668f549377b5c23eefed32389c42bdbdf44efd4696cc5b6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp313-cp313-macosx_10_15_universal2.whl

Download URL orso-0.1.2-cp313-cp313-macosx_10_15_universal2.whl
Size 571.5 kB
Tags CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
93ab795e060134ffad0c0d1ad3fe2e9e7691b7a55dc99145b0ba17d49c6ff8bd
BLAKE2b-256 checksum
How to use checksums
5562a0c43ab862183f90df218ea5a28183c69543cff8f86adaec887a8e8302c0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.2-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.12 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
e21e3c595706723f9b06def611311ff44af7d7437cf010ce3364c0907bd4dc96
BLAKE2b-256 checksum
How to use checksums
33e3f4be6c18351bf5edfe01dc48e4b37bbf099848b544a010aee941b60c0bcf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp312-cp312-macosx_10_15_universal2.whl

Download URL orso-0.1.2-cp312-cp312-macosx_10_15_universal2.whl
Size 571.4 kB
Tags CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
2847c472a57594bca6582a5cf0121c34d2cd82eb111610c35cd7e0889e4c7a33
BLAKE2b-256 checksum
How to use checksums
8428ca36b51ebc811bc538638e3bd63bfd0af6372aaf4680845a6ef43ca23d69
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.2-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.11 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
df80e2ff7db6c997c10f9fb772321dce6c13103999387c8466450fe38d855093
BLAKE2b-256 checksum
How to use checksums
37125b56a4d5b6aa439cc72b79fda52cea07853a74a7a1abc0ea1f9e3f459b1e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp311-cp311-macosx_10_15_universal2.whl

Download URL orso-0.1.2-cp311-cp311-macosx_10_15_universal2.whl
Size 566.8 kB
Tags CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
1b43b0f70a2a962ba1f6cd97eea553a52e33324446f78a1b18bf3fc0d2974fed
BLAKE2b-256 checksum
How to use checksums
c098a95fcd4151254953ea30d90b76839c5f49bdd0f2210bd3b02e6d31a041f6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.2-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.10 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
82a0da1aa309daa5f46589615e0d6d5c0310c3fae0f841a85993cbd72b745ec2
BLAKE2b-256 checksum
How to use checksums
87f198f972a3ff4eb100197d09f913594713acf001843413d09a2a0990430592
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Release files / orso-0.1.2-cp310-cp310-macosx_10_15_universal2.whl

Download URL orso-0.1.2-cp310-cp310-macosx_10_15_universal2.whl
Size 567.5 kB
Tags CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
a954f1d2e168e8bfe66a4f52215fb73840044210ff51b857238f0543a7408c08
BLAKE2b-256 checksum
How to use checksums
f5a2b77af34550e7538c10a5202513d3b34b45c041715d914edc60c38a751363
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.14

Security for orso 0.1.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.2 This release

11 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page