Skip to main content

Orso

Orso is a shared DataFrame library for Opteryx and Mabel.

PyPI Latest Release Downloads codecov Documentation

Overview

Orso is not intended to compete with Polars or Pandas (or your favorite bear DataFrame technology), instead it is developed as a common layer for Mabel and Opteryx.

Key Use Cases:

  • In Opteryx, Orso provides most of the database Cursor functionality
  • In Mabel, Orso provides the data schema and validation functionality

Orso DataFrames are row-based, driven by their initial target use-case as the WAL for Mabel and Cursor for Opteryx. Each row in an Orso DataFrame can be quickly converted to a Tuple of values, a Dictionary, or a byte representation.

Installation

Install Orso from PyPI:

pip install orso

Quick Start

Creating a DataFrame

import orso

# Create from list of dictionaries
df = orso.DataFrame([
    {'name': 'Alice', 'age': 30, 'city': 'New York'},
    {'name': 'Bob', 'age': 25, 'city': 'San Francisco'},
    {'name': 'Charlie', 'age': 35, 'city': 'Chicago'}
])

print(f"Created DataFrame with {df.rowcount} rows and {df.columncount} columns")

Displaying Data

# Display the DataFrame
print(df.display())

# Convert to different formats
arrow_table = df.arrow()  # PyArrow Table
pandas_df = df.pandas()   # Pandas DataFrame

Working with Schema

# Access column names
print("Columns:", df.column_names)

# Access schema information  
print("Schema:", df.schema)

Converting Between Formats

# From PyArrow
import pyarrow as pa
arrow_table = pa.table({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})
orso_df = orso.DataFrame.from_arrow(arrow_table)

# To Pandas
pandas_df = orso_df.pandas()

# To and from draken (opteryx-core's columnar engine), when draken is importable
morsel = orso_df.to_draken()
orso_df = orso.DataFrame.from_draken(morsel)

Features

  • Lightweight: Minimal overhead for tabular data operations
  • Row-based: Optimized for row-oriented operations
  • Interoperable: Easy conversion to/from PyArrow, Pandas
  • Schema-aware: Built-in data validation and type checking
  • Fast serialization: Efficient conversion to bytes, tuples, and dictionaries

API Reference

DataFrame Class

The main DataFrame class provides the following key methods:

  • DataFrame(dictionaries=None, *, rows=None, schema=None) - Constructor
  • display(limit=5, colorize=True, show_types=True) - Pretty print the DataFrame
  • arrow(size=None) - Convert to PyArrow Table
  • pandas(size=None) - Convert to Pandas DataFrame
  • from_arrow(tables) - Create DataFrame from PyArrow Table(s)
  • fetchall() - Get all rows as list of Row objects
  • collect() - Materialize the DataFrame
  • append(other) - Append another DataFrame
  • distinct() - Get unique rows

Properties

  • rowcount - Number of rows
  • columncount - Number of columns
  • column_names - List of column names
  • schema - Schema information

Development

Building from Source

# Clone the repository
git clone https://github.com/mabel-dev/orso.git
cd orso

# Install dependencies
pip install -r requirements.txt
pip install -r tests/requirements.txt

# Build the native extension (needs a C++20 compiler; nanobind comes from pip)
make compile

# Run tests
make test

Contributing

Orso is part of the Mabel ecosystem. Contributions are welcome! Please ensure:

  1. All tests pass: make test
  2. Code follows the project style: make lint
  3. New features include appropriate tests
  4. Documentation is updated for API changes

Performance Benchmarking

Orso has a micro-benchmark harness in benchmarks/ covering construction, append, Row, Arrow conversion, iteration, display and type parsing. Each case is warmed up and timed over several rounds; compare the minimum against a saved baseline:

# Run the full suite (or a subset with -k <substring>)
python benchmarks/bench.py

# Record a baseline, change code, rebuild the extensions, then compare
python benchmarks/bench.py -o benchmarks/baseline_0.0.246.json
python setup.py build_ext --inplace
python benchmarks/bench.py -c benchmarks/baseline_0.0.246.json

make bench and make bench-baseline wrap the same commands.

Building from source

make compile builds the native layer (orso._native, a nanobind extension over draken headers vendored into native/). There is no Cython any more. The vendored headers are copied from an opteryx-core checkout beside this one with make sync-draken and verified with make check-draken; never edit them in place. See DESIGN.md.

License

License

Orso is licensed under Apache 2.0 unless explicitly indicated otherwise.

Status

Status

Orso is in beta. Beta means different things to different people, to us, being beta means:

  • Interfaces are generally stable but may still have breaking changes
  • Unit tests are not reliable enough to capture breaks to functionality
  • Bugs are likely to exist in edge cases
  • Code may not be tuned for performance

As such, we really don't recommend using Orso in critical applications.

  • Opteryx - SQL query engine for data files
  • Mabel - Data processing framework

Metadata

Release files for orso 0.1.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for orso 0.1.4
File Size Uploaded Provenance Actions
orso-0.1.4.tar.gz 439.0 kB Without provenance Details for orso-0.1.4.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for orso 0.1.4
File Actions
orso-0.1.4-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.14 CPython 3.14 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.4-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.4-cp314-cp314-macosx_10_15_universal2.whl CPython 3.14 CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.4-cp314-cp314-macosx_10_15_universal2.whl
orso-0.1.4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.13 CPython 3.13 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.4-cp313-cp313-macosx_10_15_universal2.whl CPython 3.13 CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.4-cp313-cp313-macosx_10_15_universal2.whl
orso-0.1.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.4-cp312-cp312-macosx_10_15_universal2.whl CPython 3.12 CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.4-cp312-cp312-macosx_10_15_universal2.whl
orso-0.1.4-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details for orso-0.1.4-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.4-cp311-cp311-macosx_10_15_universal2.whl CPython 3.11 CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.4-cp311-cp311-macosx_10_15_universal2.whl
orso-0.1.4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details for orso-0.1.4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.4-cp310-cp310-macosx_10_15_universal2.whl CPython 3.10 CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.4-cp310-cp310-macosx_10_15_universal2.whl

Total release size: 13.2 MB

Release files / orso-0.1.4.tar.gz

Download URL orso-0.1.4.tar.gz
Size 439.0 kB
Tags Source
SHA-256 checksum
How to use checksums
d8a7f348bd65a5345dc683d7c2a5221f2edacc807a51f7e7bee8105d0812e249
BLAKE2b-256 checksum
How to use checksums
371a05a4390d933bdea3ad43a47aa58dbbe824d09f2cdf3704eba83be0c2b8fb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.4-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.14 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
1f6409c4db4a47355decf46432066e393248380608f6d256533ebec18063e910
BLAKE2b-256 checksum
How to use checksums
a6f96fb748f3651407012bc0754cbfe9d309a3b9ed29c324daafa8d8b340bafa
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp314-cp314-macosx_10_15_universal2.whl

Download URL orso-0.1.4-cp314-cp314-macosx_10_15_universal2.whl
Size 571.9 kB
Tags CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
e17dc34ec4a1a96474d1b1510f7c0650be5cfdf723c417c72b2b4b40d7d7f7a9
BLAKE2b-256 checksum
How to use checksums
941d92d09708e3d4f613cd78e3b4b9be39f9c0f2b9a795a7edf7ff1e273842ce
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.4-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.13 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
325b986e047c0c20ab0aab552be8aaec825373bd886b3ca5162c5235bd7cd82b
BLAKE2b-256 checksum
How to use checksums
36460b6cd0b7569e3c0891d7821f45a4019cc3548030c3932eb77f6941bfc0b9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp313-cp313-macosx_10_15_universal2.whl

Download URL orso-0.1.4-cp313-cp313-macosx_10_15_universal2.whl
Size 571.7 kB
Tags CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
2575e25082f8d2b69a3f276e75924e2b8dba632d971a6848fa2f1e7d34428038
BLAKE2b-256 checksum
How to use checksums
d2ec8f68bc04cdfc387f7abffbec029c226c23e8b62a3d98aa8136e4cc369e39
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.4-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.12 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
de59e347b23775f1e47eca573c98a6621ef946b1a26bd6819d5edafdfbff9e1e
BLAKE2b-256 checksum
How to use checksums
00023db5a2dc169624c144c8c9bf68b3606ad10dabdf6e48ea65ad68753ed1a2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp312-cp312-macosx_10_15_universal2.whl

Download URL orso-0.1.4-cp312-cp312-macosx_10_15_universal2.whl
Size 571.6 kB
Tags CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
d16bfa1a9bb810ff277d3a047e8971e2a30b4e60ef10835623862eb475e4692a
BLAKE2b-256 checksum
How to use checksums
17aaa09587e9cc26c7e661226648ae3513398fbf25729de53dbe71b2d98ade86
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.4-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.11 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
34a9801c6c355d47ddb6bbe74d4506a0745793d45f0c80d89f484a7705f5b6ca
BLAKE2b-256 checksum
How to use checksums
030b1237c550fc27b33fbf15d63459c1789b63890c6f2f486d76e226f129d95d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp311-cp311-macosx_10_15_universal2.whl

Download URL orso-0.1.4-cp311-cp311-macosx_10_15_universal2.whl
Size 567.0 kB
Tags CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
7ce74ed65b53315cca96d0f875f8ac8446b546a49bc5e69571383753a2c6e3d6
BLAKE2b-256 checksum
How to use checksums
2e43a3748526b72196ecfd41ad7f39dfc21714bbc93987bd7106ad81bcf55b17
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.4-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.10 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
0dd761eb0abb0ed46c724e98e181326cb95d73483aa0f471bcc310ed9fc848a9
BLAKE2b-256 checksum
How to use checksums
a5c920f81de9d7068ea8fa55cedee5f6309054b549ea1b19c5261aa1fb070db6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.4-cp310-cp310-macosx_10_15_universal2.whl

Download URL orso-0.1.4-cp310-cp310-macosx_10_15_universal2.whl
Size 567.8 kB
Tags CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
608dccbca6727969b06e372e1ab139844f7b71c19680958beac9534a136b6d02
BLAKE2b-256 checksum
How to use checksums
30ed5845dc5f94b6273113912df7d097f5b035ee3dcff41e25e0e8bba34edb4d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for orso 0.1.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.4 This release

11 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page