daft-lance 0.2.0

Lance integration for Daft - compaction, indexing, merge columns, and REST operations

daft-lance

Lance integration for Daft. Provides distributed compaction, scalar indexing, column merging, and REST catalog operations for Lance datasets.

Install

pip install daft-lance

For REST catalog support:

pip install daft-lance[rest]

Usage

Compaction

from daft_lance import compact_files

compact_files("s3://bucket/my_dataset")

Scalar Indexing

from daft_lance import create_scalar_index

create_scalar_index("s3://bucket/my_dataset", column="name", index_type="INVERTED")

Column Merging

from daft_lance import merge_columns_df

merge_columns_df(df, "s3://bucket/my_dataset")

REST Catalog

from daft_lance import LanceRestConfig, write_lance_rest

config = LanceRestConfig(base_url="https://api.lancedb.com", api_key="...")
write_lance_rest(mp, config, namespace="default", table_name="my_table")

Development

Requires uv.

uv sync
uv run pytest tests/ -v

License

Apache-2.0