lavender-data 0.1.2

Load & evolve datasets efficiently

Load & evolve datasets efficiently

Please visit our docs for more information.
docs.lavenderdata.com

Quick Start

Installation

pip install lavender-data

Start the server

lavender-data server start --init

lavender-data is running on 0.0.0.0:8000
UI is running on http://localhost:3000
API key created: la-...

Save the API key to use it in the next steps.

export LAVENDER_API_URL=http://0.0.0.0:8000
export LAVENDER_API_KEY=la-...

Create an example dataset

lavender-data client \
  datasets create \
  --name my_dataset \
  --uid-column-name id \
  --shardset-location https://docs.lavenderdata.com/example-dataset/images/

Iterate over the dataset

import lavender_data.client as lavender

lavender.init()

iteration = lavender.LavenderDataLoader(
    dataset_name="my_dataset",
    shuffle=True,
    shuffle_block_size=10,
)

for i in iteration:
    print(i["id"])

Please visit our docs for more information.
docs.lavenderdata.com