gapless-deribit-clickhouse

Deribit options data pipeline with ClickHouse storage.

Features

• Historical trades: Backfillable from 2018 via history.deribit.com
• Schema-first: YAML schemas generate types, DDL, and documentation
• Python API: Clean interface for querying options data
• BTC + ETH options: Focused on Deribit options trading data

Installation

pip install gapless-deribit-clickhouse

Quick Start

import gapless_deribit_clickhouse as gdch

# Fetch historical trades
df = gdch.fetch_trades(
    underlying="BTC",
    start="2024-01-01",
    end="2024-01-31",
    option_type="C"  # Calls only
)

# Collect trades to ClickHouse
gdch.collect_trades(
    underlying="BTC",
    start="2024-01-01"
)

Data Sources

Trades Table

Instrument Name Format

{UNDERLYING}-{DDMMMYY}-{STRIKE}-{TYPE}

Examples:

• BTC-27DEC24-100000-C - BTC call, $100k strike, expires Dec 27 2024
• ETH-28MAR25-5000-P - ETH put, $5k strike, expires Mar 28 2025

Configuration

Credentials

Set ClickHouse credentials via one of:

1. Environment variables:

   export CLICKHOUSE_HOST_READONLY=<host>
   export CLICKHOUSE_USER_READONLY=<user>
   export CLICKHOUSE_PASSWORD_READONLY=<password>
   

2. .env file in project root

3. Doppler (recommended for production):

   doppler setup --project gapless-deribit-clickhouse --config prd
   

Development

# Install with dev dependencies
pip install -e ".[dev]"

# Run tests
pytest

# Run linting
ruff check src/

Architecture

+---------------------+
| history.deribit.com |
|  /get_last_trades   |
+---------------------+
          |
          v
+---------------------+
|   TradesCollector   |  Historical
|    (Backfill)       |  Backfill
+---------------------+
          |
          v
+=====================+
|       deribit       |
|   .options_trades   |
+=====================+
          |
          v
+---------------------+
|     Python API      |
|   fetch_trades()    |
|  collect_trades()   |
+---------------------+

ADR: 2025-12-08-clickhouse-naming-convention

License

MIT