lancecli

Inspect and dump Lance datasets from the command line.

Reads go through pylance only. There is no DuckDB (or pandas) on the data path: column projection, SQL filters, sorts, and row limits are pushed into the Lance scanner. Random row access uses Lance take, not a full scan.

• Install
• Demo dataset
• Quick start
• Commands — worked examples on examples/events.lance
• CLI reference — every flag, format, env var, and exit code
• Cloud storage — env-only S3, then optional flags
• Development
• Releasing

Install

pip install lancecli

That installs the lancecli command. From then on:

lancecli --help
lancecli stat /path/to/dataset.lance

lancecli <command> [flags] <URI>

The URI is a positional argument (local directory, s3://, gs://, or az://). Flags can go before or after it. Every command also has -h / --help with the same flags.

Quick start

After pip install lancecli, point it at your dataset:

lancecli stat /path/to/events.lance
lancecli show /path/to/events.lance -c id,name -n 10

The copy-paste examples below use examples/events.lance from a clone of this repository.

lancecli stat examples/events.lance
lancecli show examples/events.lance
lancecli take --indices 0,2:5,-1 -c id,name,age examples/events.lance
lancecli show -n 5 -c id,age --order-by age:desc examples/events.lance

Commands

stat

First command to run on an unfamiliar dataset. Metadata only — no row scan. Prints rows, deleted, fragment sizes, on-disk data size, version, format, and indexes.

lancecli stat examples/events.lance

╭───────────────────────╮
│ examples/events.lance │
╰─── dataset summary ───╯

 rows        50
 deleted     0  (0.0%)
 columns     5
 fragments   1  (50 rows each)
 data size   3.8 KiB
 version     1  (latest 1)
 format      Lance 2.1
 versions    1
 indices     none

Machine-readable:

lancecli stat examples/events.lance --format json

inspect is the verbose sibling (full schema dump). Prefer stat for a health check.

show

Human-readable preview. Defaults to the first 10 rows. Always combine with -c / --columns on wide tables, and -n if you want more than 10.

lancecli show examples/events.lance -c id,name,age,city

┏━━━━┳━━━━━━━━┳━━━━━┳━━━━━━┓
┃ id ┃ name   ┃ age ┃ city ┃
┡━━━━╇━━━━━━━━╇━━━━━╇━━━━━━┩
│ 0  │ user_0 │ 20  │ TLV  │
│ 1  │ user_1 │ 21  │ NYC  │
│ 2  │ user_2 │ 22  │ BER  │
│ 3  │ user_3 │ 23  │ LON  │
│ 4  │ user_4 │ 24  │ TLV  │
│ 5  │ user_5 │ 25  │ NYC  │
│ 6  │ user_6 │ 26  │ BER  │
│ 7  │ user_7 │ 27  │ LON  │
│ 8  │ user_8 │ 28  │ TLV  │
│ 9  │ user_9 │ 29  │ NYC  │
└────┴────────┴─────┴──────┘

Vectors and long strings are truncated. --full prints them whole:

lancecli show examples/events.lance -n 2 -c id,embedding
lancecli show examples/events.lance -n 1 -c id,embedding --full

┏━━━━┳━━━━━━━━━━━━━━━┓
┃ id ┃ embedding     ┃
┡━━━━╇━━━━━━━━━━━━━━━┩
│ 0  │ [0, 1, …] (4) │
│ 1  │ [4, 5, …] (4) │
└────┴───────────────┘

┏━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ id ┃ embedding            ┃
┡━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ 0  │ [0.0, 1.0, 2.0, 3.0] │
└────┴──────────────────────┘

--format can be table (default), csv, jsonl, or json.

Filter (pushed into the scanner)

--filter is a SQL predicate. Lance evaluates it in the scanner; with a scalar index on that column it can skip fragments instead of reading every row.

lancecli show examples/events.lance -n 20 -c id,name --filter "age > 18"
lancecli show examples/events.lance -c id,city,age --filter "city = 'TLV'"

Sort (--order-by)

On show / csv / jsonl the sort is passed to the Lance scanner (order_by=), together with --filter, -c, and -n. The table is not loaded into Python and then sorted.

--order-by age              # ascending (default)
--order-by age:desc
--order-by age:desc,id      # age descending, then id ascending

Directions: asc / ascending / a, or desc / descending / d.

Five oldest ages in the demo file:

lancecli show examples/events.lance -n 5 -c id,name,age --order-by age:desc

┏━━━━┳━━━━━━━━━┳━━━━━┓
┃ id ┃ name    ┃ age ┃
┡━━━━╇━━━━━━━━━╇━━━━━┩
│ 39 │ user_39 │ 59  │
│ 38 │ user_38 │ 58  │
│ 37 │ user_37 │ 57  │
│ 36 │ user_36 │ 56  │
│ 35 │ user_35 │ 55  │
└────┴─────────┴─────┘

Filter first, then sort the matching rows (both still in the scanner):

lancecli show examples/events.lance -c id,city,age \
  --filter "city = 'TLV'" --order-by age:desc

┏━━━━┳━━━━━━┳━━━━━┓
┃ id ┃ city ┃ age ┃
┡━━━━╇━━━━━━╇━━━━━┩
│ 36 │ TLV  │ 56  │
│ 32 │ TLV  │ 52  │
│ 28 │ TLV  │ 48  │
│ 24 │ TLV  │ 44  │
│ 20 │ TLV  │ 40  │
│ 16 │ TLV  │ 36  │
│ 12 │ TLV  │ 32  │
│ 48 │ TLV  │ 28  │
│ 8  │ TLV  │ 28  │
│ 4  │ TLV  │ 24  │
└────┴──────┴─────┘

(Default -n 10, so this is the ten oldest TLV rows — not every TLV row.)

If stat / indices show a scalar or vector index on a column, --filter (and sometimes --order-by) on that column can use it instead of a full scan.

take

Lance-native random access. take does not scan from the start of the file. It calls ds.take(indices): positional live-row indexes in the opened version (0 is the first live row, -1 is the last).

That is the point versus a Parquet dump: jump to row 100_000 without reading the 99_999 before it.

--indices uses Python list/slice syntax (comma-separated):

lancecli take --indices 0,2:5,-1 -c id,name,age examples/events.lance

┏━━━━┳━━━━━━━━━┳━━━━━┓
┃ id ┃ name    ┃ age ┃
┡━━━━╇━━━━━━━━━╇━━━━━┩
│ 0  │ user_0  │ 20  │
│ 2  │ user_2  │ 22  │
│ 3  │ user_3  │ 23  │
│ 4  │ user_4  │ 24  │
│ 49 │ user_49 │ 29  │
└────┴─────────┴─────┘

Every tenth row:

lancecli take --indices '::10' -c id,name,city examples/events.lance

┏━━━━┳━━━━━━━━━┳━━━━━━┓
┃ id ┃ name    ┃ city ┃
┡━━━━╇━━━━━━━━━╇━━━━━━┩
│ 0  │ user_0  │ TLV  │
│ 10 │ user_10 │ BER  │
│ 20 │ user_20 │ TLV  │
│ 30 │ user_30 │ BER  │
│ 40 │ user_40 │ TLV  │
└────┴─────────┴──────┘

--order-by on take sorts the small table you already fetched, not the whole dataset:

lancecli take --indices 0,10,20,30,40 -c id,age --order-by age:desc examples/events.lance

Indexes are live-row positions in the current version, not the id column. After deletes/compaction, “row 5” is the fifth surviving row, not necessarily id == 5.

sample

Random live-row indexes, then take. Default 10 rows. --seed makes it reproducible.

lancecli sample examples/events.lance -n 3 --seed 42 -c id,name

┏━━━━┳━━━━━━━━━┓
┃ id ┃ name    ┃
┡━━━━╇━━━━━━━━━┩
│ 40 │ user_40 │
│ 7  │ user_7  │
│ 1  │ user_1  │
└────┴─────────┘

With --filter, matching rows are loaded then sampled (still column-projected). --order-by sorts the sample afterwards.

tail

Last N rows (default 10). Uses count_rows plus a scanner offset, so it does not materialize the prefix.

lancecli tail examples/events.lance -n 3 -c id,name,age

┏━━━━┳━━━━━━━━━┳━━━━━┓
┃ id ┃ name    ┃ age ┃
┡━━━━╇━━━━━━━━━╇━━━━━┩
│ 47 │ user_47 │ 27  │
│ 48 │ user_48 │ 28  │
│ 49 │ user_49 │ 29  │
└────┴─────────┴─────┘

--filter applies first: tail --filter "city = 'TLV'" is the last N TLV rows in scan order, not the last N of the whole file.

csv / jsonl

Stream Arrow record batches to stdout. They never load the whole table into memory. Default is every matching row (-n is optional). Prefer jsonl for vectors, lists, and nested types.

lancecli csv examples/events.lance -n 3 -c id,name,age,city

id,name,age,city
0,user_0,20,TLV
1,user_1,21,NYC
2,user_2,22,BER

lancecli jsonl examples/events.lance -n 2 -c id,name,embedding
lancecli jsonl examples/events.lance -n 1 -c id,embedding --full

{"id": 0, "name": "user_0", "embedding": [0.0, 1.0, "… (4 items)"]}
{"id": 1, "name": "user_1", "embedding": [4.0, 5.0, "… (4 items)"]}
{"id": 0, "embedding": [0.0, 1.0, 2.0, 3.0]}

Same scan flags as show: -c, --filter, --order-by, -n, --full.

schema

Arrow schema of the opened version:

lancecli schema examples/events.lance

id: int64
name: string
age: int64
city: string
embedding: fixed_size_list<item: float>[4]

--physical prints the Lance-native physical schema (field ids, encodings). --format json is available.

inspect

Verbose dump: URI, version, row counts, fragment count, index flag, then every column name and the full schema. Use when stat is not enough.

lancecli inspect examples/events.lance

############ dataset ############
uri: examples/events.lance
version: 1
latest_version: 1
num_rows: 50
num_deleted_rows: 0
num_fragments: 1
num_columns: 5
has_index: false
data_storage_version: 2.1

############ columns ############
id
name
age
city
embedding

############ schema ############
id: int64
name: string
age: int64
city: string
embedding: fixed_size_list<item: float>[4]

count

Unfiltered count is metadata-only. --filter goes through the scanner (and can use a scalar index).

lancecli count examples/events.lance
lancecli count examples/events.lance --filter "age > 40"

50
19

freq

Value counts for one column (-c / --column here is that column, not a projection list). Good for class balance / enum sanity. Default: top 20 values.

lancecli freq examples/events.lance -c city

city  ·  50 rows
┏━━━━━━━┳━━━━━━━┳━━━━━━━━━┓
┃ value ┃ count ┃ percent ┃
┡━━━━━━━╇━━━━━━━╇━━━━━━━━━┩
│ TLV   │    13 │   26.0% │
│ NYC   │    13 │   26.0% │
│ BER   │    12 │   24.0% │
│ LON   │    12 │   24.0% │
└───────┴───────┴─────────┘

--filter restricts the population first (freq -c city --filter "age > 40"). -n caps how many distinct values are printed.

versions

Lance time travel. Each write creates a new version; show --version N opens that snapshot.

lancecli versions examples/events.lance

┏━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ version ┃ timestamp                  ┃ metadata                  ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ 1       │ 2026-09-03 09:48:02.207822 │ {'total_rows': '50', …}   │
└─────────┴────────────────────────────┴───────────────────────────┘

Then:

lancecli show examples/events.lance --version 1 -n 3 -c id,name

--tag and --asof are the other time-travel knobs (only one of --version / --tag / --asof).

fragments

Physical layout: one row per fragment, with physical rows, deletions, live rows, and data-file names. Many tiny fragments or a high deleted % are a reason to compact; stat already hints at that.

lancecli fragments examples/events.lance

┏━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓
┃ id ┃ physical_rows ┃ num_deletions ┃ num_rows ┃ files                ┃
┡━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩
│ 0  │ 50            │ 0             │ 50       │ 1000001000001011011… │
└────┴───────────────┴───────────────┴──────────┴──────────────────────┘

indices

Lists vector, full-text, and scalar indexes. The demo file has none:

lancecli indices examples/events.lance

(no indices)

When indexes exist, stat summarizes them on one line and suggests using --filter / --order-by on those columns.

CLI reference

This is the full surface of lancecli. lancecli <command> --help prints the same flags. There is no separate Python import API — the package is this CLI.

Command × flags

● = accepted. Empty = not on that command.

csv and jsonl have no --format: they are the format. versions always lists the whole history (no --version checkout). take has no --filter — it addresses rows by position.

Metadata-only (no row scan unless noted): stat, inspect, schema, unfiltered count, versions, fragments, indices. count --filter and freq do scan.

URI

Last positional argument (flags may sit before or after it).

Must be the dataset directory (contains _versions/ or data/), not a single .lance fragment file inside data/.

Dataset (time travel)

Only one of these per invocation:

Default: latest version. versions lists history; it does not take these flags.

Scan

Output

--format values per command

json for row commands is one JSON array of objects (loads the printed table). jsonl is one object per line and streams. csv writes a header then records; embeddings are stringified.

Truncation (unless --full): lists preview 2 items; strings longer than 80 characters are clipped; blobs become <binary …>.

Unbounded csv/jsonl to a TTY with more than 10,000 matching rows prints a stderr warning; pass -n or redirect.

Storage flags (optional — env is enough)

Common KEYs (passed through to pylance / object_store):

Do not put access keys on the command line; use environment variables.

Environment variables

For a remote URI (s3://, gs://, az://, …), lancecli reads the process environment. CLI flags override env. Local paths ignore AWS connection env so show ./events.lance stays local.

S3 and S3-compatible (MinIO, Ceph, …) — set these and omit storage flags:

Real AWS S3 (no custom endpoint) usually needs only credentials or an instance role — no lancecli flags.

GCS (object_store / Lance, not lancecli-specific): GOOGLE_APPLICATION_CREDENTIALS, GOOGLE_SERVICE_ACCOUNT, GOOGLE_SERVICE_ACCOUNT_KEY.

Azure: AZURE_STORAGE_ACCOUNT_NAME, AZURE_STORAGE_ACCOUNT_KEY, AZURE_STORAGE_CONNECTION_STRING, AZURE_STORAGE_SAS_KEY, plus Entra ID vars when using AAD.

Exit codes

Cloud storage

Lance talks to object stores natively. Pass a cloud URI; do not download files first.

Preferred: environment only (no --endpoint-url, no --storage-option):

export AWS_ACCESS_KEY_ID=<AWS_ACCESS_KEY_ID>
export AWS_SECRET_ACCESS_KEY=<AWS_SECRET_ACCESS_KEY>
export AWS_ENDPOINT_URL=http://s3-compatible.example   # skip on real AWS
export AWS_DEFAULT_REGION=us-east-1                    # often required on-prem
# AWS_ALLOW_HTTP=true  # optional; inferred from http:// endpoints

lancecli stat s3://bucket/path/dataset.lance
lancecli show -n 20 -c id,name,score --order-by score:desc s3://bucket/path/dataset.lance

Flags still work and override env when you need a one-off:

lancecli stat s3://bucket/events.lance --endpoint-url http://localhost:9000
lancecli show gs://bucket/events.lance --storage-option service_account=./sa.json -n 5 -c id

On a wide remote table, always project columns (-c). take is cheaper than show when you need specific row indexes.