zarrwhals

Package
CI/CD
Docs & Tools

Dataframe-agnostic Zarr storage powered by Narwhals

Overview

Zarrwhals provides a simple API for storing DataFrames in Zarr format, using Narwhals for DataFrame interoperability. Inspired by AnnData's approach to storing obs/var DataFrames as Zarr Stores.

Quick Start

import pandas as pd
import zarrwhals as zw

# Create a DataFrame
df = pd.DataFrame({
    "id": [1, 2, 3],
    "name": ["Alice", "Bob", "Charlie"],
    "value": [10.5, 20.3, 30.1],
    "category": pd.Categorical(["A", "B", "A"])
})

# Write to Zarr
zw.to_zarr(df, "data.zarr")

# Read back
df_loaded = zw.from_zarr("data.zarr")

With Polars

import polars as pl
import zarrwhals as zw

# Write polars DataFrame
df_pl = pl.DataFrame({
    "a": [1, 2, 3],
    "b": ["x", "y", "z"]
})
zw.to_zarr(df_pl, "data.zarr")

# Read as pandas
df_pd = zw.from_zarr("data.zarr", backend="pandas")

# Or read back as polars
df_pl2 = zw.from_zarr("data.zarr", backend="polars")

Getting started

Please refer to the documentation, in particular, the API documentation.

Installation

You need to have Python 3.11 or newer installed on your system. If you don't have Python installed, we recommend installing uv.

There are several alternative options to install zarrwhals:

1. Install the latest development version:

pip install git+https://github.com/srivarra/zarrwhals.git@main

Release notes

See the changelog.

Contact

For questions and help requests, you can reach out in the scverse discourse. If you found a bug, please use the issue tracker.

Citation

t.b.a