polarway 2.0.0

High-performance Railway-Oriented DataFrame engine with gRPC streaming and Lakehouse support

Polarway Python Client

Python client library for connecting to Polarway gRPC server.

Installation

# Install from source
cd polarway-python
pip install -e .

# Generate protocol buffers
./generate_protos.sh

Quick Start

import polarway as pw

# Connect to server
pw.connect("localhost:50051")

# Read Parquet
df = pw.read_parquet("data.parquet")

# Get info
print(df.shape())  # (1000, 5)
print(df.schema()) # Schema info

# Select columns
df2 = df.select(["col1", "col2"])

# Collect to Arrow Table
table = df2.collect()
print(table)

# Write to Parquet
df2.write_parquet("output.parquet")

Features

• gRPC Interface: Network-capable DataFrame operations
• Arrow IPC: Zero-copy data transfer
• Handle-Based: Efficient server-side operations
• Streaming: Process larger-than-memory datasets

Phase 1 Implementation

Currently implemented:

• ✅ Connection management
• ✅ read_parquet
• ✅ write_parquet
• ✅ select
• ✅ get_schema
• ✅ get_shape
• ✅ collect
• ✅ Handle management (drop, heartbeat)

Coming in Phase 2+:

• filter, group_by, join, sort
• Time-series operations
• Network data sources