libre-alpha 0.2.4

Libre Alpha Python SDK for interacting with gRPC-based backend

Libre Alpha Python SDK

This SDK provides historical upload, historical download, local runner send, and real-time subscription APIs (HTTP + gRPC + WebSocket).

Install

pip install libre-alpha

# Auth (X-API-Key)
export LIBRE_API_KEY="YOUR_API_KEY"

# Optional: record encryption key (used to encrypt uploads for encrypted records, and decrypt encrypted payloads)
export LIBRE_RECORD_ENCRYPTION_KEY="YOUR_RECORD_ENCRYPTION_KEY"

# Optional: local libre_runner gRPC address for send_data()
export LIBRE_RUNNER_HOST="127.0.0.1"
export LIBRE_RUNNER_PORT="8128"

Initialize Client

from libre_alpha.client import LibreAlphaClient

client = LibreAlphaClient(
    api_key="YOUR_API_KEY",            # optional if using env LIBRE_API_KEY
)

## Historical Upload API (Python) - protobuf stream (/api/entries/history/upload)

This is a **historical backfill** uploader that sends protobuf `DataEntry` messages in a length-prefixed stream:

- Request body format: `[4-byte big-endian length][DataEntry][4-byte length][DataEntry]...`
- Backend groups entries **by minute**; this SDK implementation will also **bucket by minute** and send **1 minute per request**.
- Auth: `X-API-Key` (required).
- Data is carried per-symbol in `symbol_values`:
  - For non-encrypted records: `symbol_values[i].values.items = [float, ...]`
  - For encrypted records: `symbol_values[i].encrypted_payload = <bytes>` (AES-256-GCM)

Client method (recommended):

```python
def upload_data(
    record_id: int,
    df: pd.DataFrame,
    encryption_key: str | None = None,
) -> None:
    ...

• Auth is always taken from client.api_key (or env LIBRE_API_KEY when constructing the client).
• encryption_key: record encryption key for encrypting payload when the record is encrypted. If omitted, the SDK uses env LIBRE_RECORD_ENCRYPTION_KEY.

Note: the SDK also exposes a module-level upload_data(...) with the same arguments; client.upload_data(...) delegates to it internally. (We only show the signature once here to avoid confusion.)

DataFrame format:

• Required columns:
  • symbol (str): symbol_id string (e.g. "1", "3"). If you pass non-numeric symbol names, the SDK will best-effort map them to symbol_id via /api/records/user-records (target_symbols).
  • timestamp (datetime64 or unix timestamp; will be normalized to UTC and uploaded in microseconds)
• Feature columns (choose ONE approach):
  • Provide a features column containing a list of floats (or a string like "[1.0, nan, 2.0]"), OR
  • Provide user-defined float columns matching the record schema; the SDK will fetch feature order from /api/records/{id}.
• Strict schema check (enabled):
  • Uploaded feature fields are compared with API schema from /api/records/{id}.
  • Feature columns must be exactly consistent with API (不能少、不能多).
  • If using features column, each row's feature count must exactly equal API feature count.

Upload packing behavior:

• 1 request = 1 minute (backend requirement)
• Within that minute, the SDK aggregates rows by exact timestamp into 1 DataEntry.
• Each aggregated DataEntry carries multiple symbols in symbol_values: [{symbol_id, values.items}, {symbol_id, values.items}, ...]

Example (upload one minute from a previously-downloaded CSV):

import pandas as pd
from libre_alpha.client import LibreAlphaClient

df = pd.read_csv("history_1h_record12_BTC.csv")

# pick the first minute only (backend requires 1 minute per upload)
first_minute = int(df["timestamp"].iloc[0] // 60_000_000)
df1 = df[df["timestamp"].apply(lambda x: int(x // 60_000_000) == first_minute)].copy()
df1 = df1[["symbol", "timestamp", "features"]]

client = LibreAlphaClient(api_key="YOUR_API_KEY")
client.upload_data(
    record_id=12,
    df=df1,
    # encryption_key="YOUR_RECORD_ENCRYPTION_KEY",  # only needed if record is encrypted (or set LIBRE_RECORD_ENCRYPTION_KEY)
)

Historical File Download API (Python) - protobuf (length-prefixed)

def download_historical_data(
    record_id: int,
    symbols: list[str],
    start: datetime | str | int,
    end: datetime | str | int,
    encryption_key: str | None = None,
) -> pandas.DataFrame:
    pass

Parameters:

• record_id: the record id to download
• symbols: list of symbol names (e.g. ["BINANCE_BTCUSDT"]). If you pass [], the SDK will best-effort fetch all symbols from /api/records/user-records (target_symbols).
• encryption_key: optional, record encryption key for decrypting symbol_values[].encrypted_payload

Notes:

• Backend /api/entries/download-links enforces end-start <= 24h (microseconds). The SDK automatically splits long ranges into multiple 24h windows and merges results locally.
• Download worker concurrency is fixed at 100.
• Stream read chunk size is fixed at 1048576 (1 MiB). Each worker thread reuses an HTTP connection pool to the same download host.
• Returned timestamps are in microseconds.
• DataEntry uses repeated SymbolValues symbol_values; each row corresponds to one symbol_id.
• When data is encrypted (symbol_values[].encrypted_payload present), pass encryption_key to decrypt.
• The download progress line shows total bytes and real-time speed (MB/s). Set LIBRE_DOWNLOAD_PROGRESS=0 to disable progress output.

Example

from libre_alpha.client import LibreAlphaClient
import datetime as dt

client = LibreAlphaClient(
    api_key="YOUR_API_KEY",
)

start = dt.datetime(2026, 1, 1, 0, 0, 0, tzinfo=dt.timezone.utc)
end   = dt.datetime(2026, 1, 3, 0, 0, 0, tzinfo=dt.timezone.utc)  # >24h is OK (SDK will split)

df = client.download_historical_data(
    record_id=2,
    symbols=["BINANCE_BTCUSDT"],
    start=start,
    end=end,
    encryption_key="YOUR_RECORD_ENCRYPTION_KEY",  # for decrypting encrypted data
)

print(df.head())
print("rows:", len(df))

Local Runner Send API (Python) - gRPC ProcessJson

send_data(...) is still available in the Python SDK. It sends one JSON payload to a local libre_runner through the gRPC JsonService/ProcessJson RPC.

Client method:

def send_data(
    *,
    data: dict,
    record_id: int,
    timeout: float | None = None,
):
    ...

Parameters:

• data: JSON-serializable payload to send.
• record_id: target backend record id. This is required and is sent in gRPC metadata.
• timeout: optional gRPC timeout in seconds. Defaults to the client timeout.

Notes:

• Transport is gRPC, not HTTP.
• The SDK sends to LIBRE_RUNNER_HOST:LIBRE_RUNNER_PORT (defaults to 127.0.0.1:8128), or to host= / port= passed when constructing LibreAlphaClient.
• The underlying RPC is JsonService/ProcessJson.
• send_data(...) is the current send entrypoint for local runner usage. There is no separately documented send_alpha(...) API in the current SDK implementation.
• send_data(...) can also send alpha-style payloads. What matters is that record_id and the fields inside data match the target record schema.
• Runner determines whether the target record is encrypted. For encrypted records it will upload encrypted_payload; for non-encrypted records it will upload values.
• If you include symbol in data, runner will use it. If omitted, runner may try to infer it from record configuration.

Example: send normal data to local runner

from libre_alpha import LibreAlphaClient

client = LibreAlphaClient(
    api_key="YOUR_API_KEY",
    host="127.0.0.1",
    port=8128,
)

resp = client.send_data(
    record_id=13,
    data={
        "symbol": "BINANCE_BTCUSDT",
        "Price": 150,
        "Volume": 200,
        "timestamp_ms": 1710000000000,
    },
)

print(resp)

Example: send alpha data to local runner

from libre_alpha import LibreAlphaClient

client = LibreAlphaClient(
    api_key="YOUR_API_KEY",
    host="127.0.0.1",
    port=8128,
)

resp = client.send_data(
    record_id=4,
    data={
        "symbol": "BINANCE_BTCUSDT",
        "alpha_value": 0.1234,
        "timestamp_ms": 1710000000000,
    },
)

print(resp)

Real-time Subscribe API (Python) - WebSocket ws/data

from typing import List, Callable
import pandas as pd

def subscribe_data(
    subscriptions: List[dict],
    *,
    on_data_fn: Callable[[str, pd.DataFrame], None],
    encryption_key: str | None = None,
) -> None:
    """Subscribes to live data for multiple records.

    Each subscription item is:
      - record_id: int
      - symbols: optional list[str]. If omitted or empty, subscribe all symbols for that record.
    """

Parameters:

• subscriptions: list of {record_id, symbols?} items. symbols can be symbol names or symbol_id strings. If symbols is omitted or [], the SDK subscribes to all symbols for that record (best-effort via /api/records/{id}).
• on_data_fn: callback invoked upon receiving new data. Takes (symbol: str, df: DataFrame).
• encryption_key: optional, record encryption key for decrypting symbol_values[].encrypted_payload; required if data is encrypted.

Notes:

• Connects to ws/data, sends { wallet_address, record_id } on open. One WS connection per record_id.
• If data is encrypted (symbol_values[].encrypted_payload has value), pass encryption_key or set LIBRE_RECORD_ENCRYPTION_KEY.
• DataFrame schema matches download_historical_data (record_id, symbol, features/expanded feature columns, timestamps).
• Blocking; run in a daemon thread for long-lived subscription.

Example

import threading
import pandas as pd
from libre_alpha import LibreAlphaClient

client = LibreAlphaClient(api_key="YOUR_API_KEY")

def on_data_fn(symbol: str, df: pd.DataFrame):
    # record_id is included in df["record_id"]
    print(f"[{symbol}] record_id={df['record_id'].iloc[0]} rows={len(df)}")

t = threading.Thread(
    target=client.subscribe_data,
    kwargs={
        "subscriptions": [
            {"record_id": 1, "symbols": ["BTCUSDT", "ETHUSDT"]},
            {"record_id": 2},  # symbols omitted => all symbols
        ],
        "on_data_fn": on_data_fn,
        "encryption_key": "YOUR_RECORD_ENCRYPTION_KEY",
    },
    daemon=True,
)
t.start()