pytortilla 0.1.0a0

The file format behind TACO.

The Tortilla Reader

What is Tortilla?

A Tortilla is an open-source, cloud-optimized format for storing large collections of files (referred to as "items") in a Binary Large Object (BLOB). It is quite simple, similar to safetensor and puffin, but designed for seamless compatibility with the Geospatial Data Abstraction Library (GDAL) Virtual File System.

Specification

Read the full specification here.

Installation

You can install the Tortilla reader using pip:

pip install pytortilla

Usage

The tortilla reader only support three methods: create, load, and compile.

Create

Create a Tortilla file from a list of local files.

import pytortilla

files = ["path/to/local/A.tif", "path/to/local/B.tif", "path/to/local/C.tif"]

tortilla_file = pytortilla.create(
    files=files,
    output= "path/to/local/demo.tortilla"
)

Load

Load local and remote Tortilla files. For remote files, the server must support the HTTP Range header method.

import pytortilla

# either local or remote you can load a Tortilla file with the same function

## local
tortilla_file = pytortilla.load(
    file="path/to/local/demo.tortilla"
)

## remote
tortilla_file = pytortilla.load(
    file="https://huggingface.co/datasets/tacofoundation/example/resolve/main/soy_una.tortilla"
)

Compile

Create small subsets of the Tortilla file. For instance, if you have a Tortilla file with 1000 items, you can create a new Tortilla file with only the first 100 items.

import pytortilla

# Load a Tortilla file
metadata = pytortilla.load(file="https://huggingface.co/datasets/tacofoundation/example/resolve/main/soy_una.tortilla")


# Create a new Tortilla file
subset = pytortilla.compile(
    dataset=metadata.iloc[40:50],
    output="soy_una_feliz.tortilla"
)