lucidlink 0.8.9

Python SDK for LucidLink — programmatic access to LucidLink filespaces

lucidlink

The LucidLink Python SDK lets you programmatically access LucidLink filespaces: reading and writing files, streaming data, integrating with data libraries like Pandas and Dask, and managing external cloud storage via LucidLink Connect (Enterprise add-on).

Key Concepts

• Workspace — your organization's LucidLink account.
• Filespace — a named storage volume within a workspace. Most SDK operations are scoped to a filespace.
• Daemon — the SDK's main entry point. Initializes the LucidLink client and manages its lifecycle.
• Service Account — a non-human identity used to authenticate SDK calls. Tokens look like sa_live:your_key.
• LucidLink Connect — a feature (Enterprise add-on) that surfaces files from external S3 storage inside a filespace without copying them.

Authentication

All SDK operations require a service account token. Create one in the LucidLink web client (requires admin permissions). Service accounts are available on Business and Enterprise plans, as well as during Trial.

credentials = lucidlink.ServiceAccountCredentials(
    token="sa_live:your_key"
)

Features

• Service Account Authentication — authenticate using service account tokens
• File Operations — read, write, create, delete files and directories
• Streaming I/O — full io.RawIOBase interface for standard Python I/O compatibility
• fsspec Integration — compatible with Pandas, Dask, PyArrow, and other data libraries
• LucidLink Connect — manage external S3 data stores and link files into filespaces (Enterprise)

Installation

Requires Python 3.10 or later.

pip install lucidlink

For fsspec integration:

pip install "lucidlink[fsspec]"

Quick Start

import lucidlink

# Create and start daemon
with lucidlink.Daemon() as daemon:
    # Authenticate with service account
    credentials = lucidlink.ServiceAccountCredentials(
        token="sa_live:your_key"
    )
    workspace = daemon.authenticate(credentials)

    # Link to a filespace
    filespace = workspace.link_filespace(name="production-data")

    # List directory
    for entry in filespace.fs.read_dir("/"):
        print(f"{entry.name}: {entry.size} bytes")

    # Write a file
    filespace.fs.write_file("/example.txt", b"Hello from LucidLink!")

    # Read a file
    data = filespace.fs.read_file("/example.txt")
    print(data)

    # Cleanup
    workspace.unlink_filespace()

Streaming File Access

Full io.RawIOBase compatibility for streaming:

# Binary streaming
with filespace.open("/large_file.dat", "rb", buffering=8192) as f:
    for chunk in iter(lambda: f.read(4096), b""):
        process(chunk)

# Text streaming with encoding
with filespace.open("/document.txt", "rt", encoding="utf-8") as f:
    for line in f:
        print(line.strip())

# Byte range reads
with filespace.open("/data.bin", "rb") as f:
    f.seek(1000)
    data = f.read(100)  # Read 100 bytes from offset 1000

fsspec Integration

Access LucidLink files using standard data libraries (requires pip install "lucidlink[fsspec]"):

import pandas as pd

# Read CSV directly from LucidLink
df = pd.read_csv(
    "lucidlink://workspace/filespace/data.csv",
    storage_options={"token": "sa_live:..."}
)

# Write Parquet to LucidLink
df.to_parquet(
    "lucidlink://workspace/filespace/output.parquet",
    storage_options={"token": "sa_live:..."}
)

Direct fsspec usage:

from lucidlink.fsspec import LucidLinkFileSystem

fs = LucidLinkFileSystem(token="sa_live:...", sandboxed=True)

# List, download, upload
entries = fs.ls("lucidlink://workspace/filespace/", detail=True)
fs.get("lucidlink://workspace/filespace/file.txt", "local_file.txt")
fs.put("local_file.txt", "lucidlink://workspace/filespace/uploaded.txt")

# Move/rename (native operation, faster than copy+delete)
fs.mv("lucidlink://workspace/filespace/old.txt",
      "lucidlink://workspace/filespace/new.txt")

fs.close()

Storage Modes

Sandboxed Mode (Default)

Uses a temporary directory that is automatically cleaned up:

daemon = lucidlink.Daemon()  # sandboxed by default

Physical Mode

Uses a persistent .lucid folder:

from lucidlink import Daemon, StorageConfig, StorageMode

# With cleanup on exit
daemon = Daemon(storage=StorageConfig(mode=StorageMode.PHYSICAL))

# Keep files after exit
daemon = Daemon(storage=StorageConfig(
    mode=StorageMode.PHYSICAL,
    persist_on_exit=True,
))

# Custom storage location
daemon = Daemon(storage=StorageConfig(
    mode=StorageMode.PHYSICAL,
    persist_on_exit=True,
    root_path=Path("D:/lucid_data"),
))

License and usage

This Software Development Kit (the “SDK”) is proprietary to LucidLink Corp. and is provided solely to enable you to develop applications and integrations that interact with LucidLink services.

Use of this SDK is governed by the LucidLink Terms and Conditions available at: https://www.lucidlink.com/terms (the “LucidLink Terms”), or, if you or your organization have entered into a separate written agreement with LucidLink governing your use of LucidLink services (for example, a subscription agreement or master services agreement), such separate written agreement.

This SDK (including any associated example or sample code) is not licensed under an open‑source license. All rights are reserved by LucidLink except as expressly granted in the Applicable Terms.

Support

• Support: [e-mail]