Skip to main content

Orso

Orso is a shared DataFrame library for Opteryx and Mabel.

PyPI Latest Release Downloads codecov Documentation

Overview

Orso is not intended to compete with Polars or Pandas (or your favorite bear DataFrame technology), instead it is developed as a common layer for Mabel and Opteryx.

Key Use Cases:

  • In Opteryx, Orso provides most of the database Cursor functionality
  • In Mabel, Orso provides the data schema and validation functionality

Orso DataFrames are row-based, driven by their initial target use-case as the WAL for Mabel and Cursor for Opteryx. Each row in an Orso DataFrame can be quickly converted to a Tuple of values, a Dictionary, or a byte representation.

Installation

Install Orso from PyPI:

pip install orso

Quick Start

Creating a DataFrame

import orso

# Create from list of dictionaries
df = orso.DataFrame([
    {'name': 'Alice', 'age': 30, 'city': 'New York'},
    {'name': 'Bob', 'age': 25, 'city': 'San Francisco'},
    {'name': 'Charlie', 'age': 35, 'city': 'Chicago'}
])

print(f"Created DataFrame with {df.rowcount} rows and {df.columncount} columns")

Displaying Data

# Display the DataFrame
print(df.display())

# Convert to different formats
arrow_table = df.arrow()  # PyArrow Table
pandas_df = df.pandas()   # Pandas DataFrame

Working with Schema

# Access column names
print("Columns:", df.column_names)

# Access schema information  
print("Schema:", df.schema)

Converting Between Formats

# From PyArrow
import pyarrow as pa
arrow_table = pa.table({'x': [1, 2, 3], 'y': ['a', 'b', 'c']})
orso_df = orso.DataFrame.from_arrow(arrow_table)

# To Pandas
pandas_df = orso_df.pandas()

# To and from draken (opteryx-core's columnar engine), when draken is importable
morsel = orso_df.to_draken()
orso_df = orso.DataFrame.from_draken(morsel)

Features

  • Lightweight: Minimal overhead for tabular data operations
  • Row-based: Optimized for row-oriented operations
  • Interoperable: Easy conversion to/from PyArrow, Pandas
  • Schema-aware: Built-in data validation and type checking
  • Fast serialization: Efficient conversion to bytes, tuples, and dictionaries

API Reference

DataFrame Class

The main DataFrame class provides the following key methods:

  • DataFrame(dictionaries=None, *, rows=None, schema=None) - Constructor
  • display(limit=5, colorize=True, show_types=True) - Pretty print the DataFrame
  • arrow(size=None) - Convert to PyArrow Table
  • pandas(size=None) - Convert to Pandas DataFrame
  • from_arrow(tables) - Create DataFrame from PyArrow Table(s)
  • fetchall() - Get all rows as list of Row objects
  • collect() - Materialize the DataFrame
  • append(other) - Append another DataFrame
  • distinct() - Get unique rows

Properties

  • rowcount - Number of rows
  • columncount - Number of columns
  • column_names - List of column names
  • schema - Schema information

Development

Building from Source

# Clone the repository
git clone https://github.com/mabel-dev/orso.git
cd orso

# Install dependencies
pip install -r requirements.txt
pip install -r tests/requirements.txt

# Build the native extension (needs a C++20 compiler; nanobind comes from pip)
make compile

# Run tests
make test

Contributing

Orso is part of the Mabel ecosystem. Contributions are welcome! Please ensure:

  1. All tests pass: make test
  2. Code follows the project style: make lint
  3. New features include appropriate tests
  4. Documentation is updated for API changes

Performance Benchmarking

Orso has a micro-benchmark harness in benchmarks/ covering construction, append, Row, Arrow conversion, iteration, display and type parsing. Each case is warmed up and timed over several rounds; compare the minimum against a saved baseline:

# Run the full suite (or a subset with -k <substring>)
python benchmarks/bench.py

# Record a baseline, change code, rebuild the extensions, then compare
python benchmarks/bench.py -o benchmarks/baseline_0.0.246.json
python setup.py build_ext --inplace
python benchmarks/bench.py -c benchmarks/baseline_0.0.246.json

make bench and make bench-baseline wrap the same commands.

Building from source

make compile builds the native layer (orso._native, a nanobind extension over draken headers vendored into native/). There is no Cython any more. The vendored headers are copied from an opteryx-core checkout beside this one with make sync-draken and verified with make check-draken; never edit them in place. See DESIGN.md.

License

License

Orso is licensed under Apache 2.0 unless explicitly indicated otherwise.

Status

Status

Orso is in beta. Beta means different things to different people, to us, being beta means:

  • Interfaces are generally stable but may still have breaking changes
  • Unit tests are not reliable enough to capture breaks to functionality
  • Bugs are likely to exist in edge cases
  • Code may not be tuned for performance

As such, we really don't recommend using Orso in critical applications.

  • Opteryx - SQL query engine for data files
  • Mabel - Data processing framework

Metadata

Release files for orso 0.1.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for orso 0.1.8
File Size Uploaded Provenance Actions
orso-0.1.8.tar.gz 441.6 kB Without provenance Details for orso-0.1.8.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for orso 0.1.8
File Actions
orso-0.1.8-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.14 CPython 3.14 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.8-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.8-cp314-cp314-macosx_10_15_universal2.whl CPython 3.14 CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.8-cp314-cp314-macosx_10_15_universal2.whl
orso-0.1.8-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.13 CPython 3.13 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.8-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.8-cp313-cp313-macosx_10_15_universal2.whl CPython 3.13 CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.8-cp313-cp313-macosx_10_15_universal2.whl
orso-0.1.8-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details for orso-0.1.8-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.8-cp312-cp312-macosx_10_15_universal2.whl CPython 3.12 CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.8-cp312-cp312-macosx_10_15_universal2.whl
orso-0.1.8-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details for orso-0.1.8-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.8-cp311-cp311-macosx_10_15_universal2.whl CPython 3.11 CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.8-cp311-cp311-macosx_10_15_universal2.whl
orso-0.1.8-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details for orso-0.1.8-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
orso-0.1.8-cp310-cp310-macosx_10_15_universal2.whl CPython 3.10 CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64) Details for orso-0.1.8-cp310-cp310-macosx_10_15_universal2.whl

Total release size: 13.2 MB

Release files / orso-0.1.8.tar.gz

Download URL orso-0.1.8.tar.gz
Size 441.6 kB
Tags Source
SHA-256 checksum
How to use checksums
61e8ec9e6bd8e9f53c476df5df2bb9a0cb3bf1049afb2945dccca07a0a416d5d
BLAKE2b-256 checksum
How to use checksums
c91af141aeae4832e57b9552899438004996e6c1c8ba7d2dea4e3798c5e8e579
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.8-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.14 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
b94508572f196026187cd7c305a117c2455a03774ba5686226097abbde4b55d5
BLAKE2b-256 checksum
How to use checksums
064ab40d23beee7f1a228a6f3b234f8cc02a0d547d63436b41b0e8c4d1bba351
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp314-cp314-macosx_10_15_universal2.whl

Download URL orso-0.1.8-cp314-cp314-macosx_10_15_universal2.whl
Size 573.2 kB
Tags CPython 3.14 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
a83bec8ff25f08b9ac34fd09ff8112938b3444122e863709d07ae3660bd503e6
BLAKE2b-256 checksum
How to use checksums
b71707213d056406f3212547f9ba08d1594268b05e4d954101e6fb8a55e582be
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.8-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.13 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
9d6ad3838ed1301520c277e659c1db3f0b2786aaa24988eaea4314409964294a
BLAKE2b-256 checksum
How to use checksums
f68ceb377aec7d424acb0111df6bb9fab8c707b6cf01b64c5bc6838cc75d53b8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp313-cp313-macosx_10_15_universal2.whl

Download URL orso-0.1.8-cp313-cp313-macosx_10_15_universal2.whl
Size 573.1 kB
Tags CPython 3.13 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
6f632ba53c15abc3261223c1886a507446aa22279d6531993c1785a8659775ae
BLAKE2b-256 checksum
How to use checksums
78332994fc27f9f3742d9896390e9da808603e5a2b3da7b121c85eda3c5c702c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.8-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.12 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
6d881576d7636de939ef7bb9de68d62488cbb1e1c031e3c555a5d05b8e026557
BLAKE2b-256 checksum
How to use checksums
5d10185d7bd9ce035963ef4f33ec9856d773eb7262a2e0cbeed288071e4f8626
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp312-cp312-macosx_10_15_universal2.whl

Download URL orso-0.1.8-cp312-cp312-macosx_10_15_universal2.whl
Size 572.9 kB
Tags CPython 3.12 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
7ff6e47c51a5e9786f333c5f70fb1bcc07ed1ce4e3d48199fcf97cf9a7a6c122
BLAKE2b-256 checksum
How to use checksums
8337b1f70e7f9f15518b337e1439b4e031c11aa71e342fa4958f4f59cfad4d68
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.8-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.11 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
84df437858a57a8ccc34d94f0b0571f85027e2431f2d7a74db6037d05d784c6c
BLAKE2b-256 checksum
How to use checksums
09b384ba26b9de59ae0d4fb4189d5529af90b0787d8afc88d8f2721dab92dd70
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp311-cp311-macosx_10_15_universal2.whl

Download URL orso-0.1.8-cp311-cp311-macosx_10_15_universal2.whl
Size 568.4 kB
Tags CPython 3.11 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
98ad4804294fb32302f6aeb7e4f51ed8a737df30db2352fc863329cb63e7d734
BLAKE2b-256 checksum
How to use checksums
78f35bab90d9c801cfad8828b632e25998ec917bffb71a3f74e60ff03a9be59e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL orso-0.1.8-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 2.0 MB
Tags CPython 3.10 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
93351582f870ea1550d7bd77db393df556f2380c765476b938a69c335d26cb64
BLAKE2b-256 checksum
How to use checksums
0108d7fb6d5e91ae9a7f6ce57d148151a3beaccd1c8e80e008fcfc7974347a0a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Release files / orso-0.1.8-cp310-cp310-macosx_10_15_universal2.whl

Download URL orso-0.1.8-cp310-cp310-macosx_10_15_universal2.whl
Size 569.1 kB
Tags CPython 3.10 macOS 10.15+ universal2 (ARM64, x86-64)
SHA-256 checksum
How to use checksums
f89260e71d4d9cb116c6affa8c6fd9e5f22a7c3519ee0d5d1cd57f5f0a6e16c8
BLAKE2b-256 checksum
How to use checksums
b5b65c9200912d70ecafb5cb3fddb83394c9d76c0ace3a1d2240298298f4b12d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.16

Security for orso 0.1.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.8 This release

11 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page