pyulysses 1.0.0

Library to manage connection to Dremio via Apache Arrow Flight

PyUlysses

PyUlysses is a Python library for seamless connectivity to Dremio DataHub using Apache Arrow Flight. It provides an intuitive interface for executing SQL queries and managing data operations with built-in support for DuckDB integration.

Architecture

┌─────────────────┐
│  Your Python    │
│  Application    │
└────────┬────────┘
         │
         │ PyUlysses
         ▼
┌─────────────────┐      Arrow Flight      ┌─────────────────┐
│   PyArrow       │◄────────────────────────►│  Dremio Server  │
│   Client        │                          │   (DataHub)     │
└────────┬────────┘                          └─────────────────┘
         │
         │ Zero-copy transfer
         ▼
┌─────────────────┐
│    DuckDB       │
│   Analytics     │
└─────────────────┘

Features

Key Features:

• High-performance Arrow Flight connectivity to Dremio
• Native DuckDB integration for query results
• Automatic retry logic with configurable timeouts
• Robust error handling and logging
• Secure authentication via PAT tokens
• Zero-copy data transfer with Apache Arrow
• Direct integration with analytical workflows

Installation

Virtual Environment (Recommended)

While optional, we strongly recommend using a virtual environment to isolate PyUlysses dependencies from your system Python installation.

Linux/macOS (Cortex)

# Create virtual environment
python3 -m venv venv

# Activate virtual environment
source venv/bin/activate

# Install lib
pip install pyulysses

Windows

# Create virtual environment
python -m venv venv

# Activate virtual environment (Command Prompt)
venv\Scripts\activate.bat

# Or PowerShell
venv\Scripts\Activate.ps1

# Install lib
pip install pyulysses

Quick Start

1. Configure Environment Variables

Create a .env file in your project root:

DREMIO_USERNAME=your_username
DREMIO_ACCESS_TOKEN=your_personal_access_token
DREMIO_HOST=dremio.example.com
DREMIO_PORT=9047

2. Basic Usage

Create a Python code inside the src folder.

from pyulysses.connector.datahub_connector import Client

# Initialize client
client = Client()

# Execute query
result = client.query(""" SELECT * FROM your_table """)

# Display results
print(result)

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

🤖 Developer

Developer	Role
Wallace Camargo	Data Engineer