Bundlebase
A versioned dataset that carries its own context, so an AI agent can query it correctly and every number can be traced back to its source.
Hand an agent a CSV and it writes SQL that looks right. A schema says a column is called amt and holds a number; it doesn't say the value is net of refunds, or what your company means by "revenue". A bundle carries that context with the data: what each column means, how your metrics are defined, where every row came from, and every transformation applied. Serve it over MCP and an agent reads all of it in one call.
The same bundle works for people — Python, the CLI, or any BI tool over Arrow Flight SQL.
Documentation | PyPI | Issues
Features
- Provenance: Every commit records who changed what, when, and how. Every attached file keeps its source and checksum, so any number can be traced back
- Semantic model: Column descriptions and named metrics travel in the bundle, so everyone (and every agent) computes a number the same way
- Agent-native: MCP server with
describe,analyze,data_quality, andmetrictools.bundlebase setup-agentwrites the config - Version history: Commit, undo, and reset. History is append-only and never rewritten
- Zero infrastructure: A bundle is a directory of YAML manifests and Parquet files, on local disk, S3, GCS, or Azure. No server, no catalog
- Works with your tools: Python (pandas, polars, pyarrow), a CLI, and BI tools over Arrow Flight SQL
- Many sources: Parquet, CSV, and JSONL from local files, S3, HTTP, SFTP, FTP, Kaggle, or PostgreSQL
- Streaming: Rust core on Apache Arrow and DataFusion. Datasets larger than RAM, constant memory
Installation
pip install bundlebase
Quick Start
Async API
import bundlebase
# Create a new bundle and chain operations
c = await (bundlebase.create()
.attach("data.parquet")
.filter("SELECT * FROM bundle WHERE age >= 18")
.drop_column("ssn")
.rename_column("fname", "first_name"))
# Convert to pandas
df = await c.to_pandas()
# Commit changes
await c.commit("Cleaned customer data")
Sync API
import bundlebase.sync as bb
# Same operations, no await needed
c = (bb.create()
.attach("data.parquet")
.filter("SELECT * FROM bundle WHERE age >= 18")
.drop_column("ssn")
.rename_column("fname", "first_name"))
df = c.to_pandas()
c.commit("Cleaned customer data")
Streaming Large Datasets
Process data larger than RAM efficiently:
import bundlebase
# Stream batches instead of loading everything
c = await bundlebase.create().attach("huge_dataset.parquet")
total_rows = 0
async for batch in bundlebase.stream_batches(c):
# Each batch is a slice of the data, not the entire dataset
total_rows += batch.num_rows
# Memory is freed after each iteration
print(f"Processed {total_rows} rows")
Core Operations
Data Loading
c = await bundlebase.create()
c = await c.attach("data.parquet") # Parquet files
c = await c.attach("data.csv") # CSV files
c = await c.attach("data.jsonl") # JSON Lines files
Data Transformation
c = await c.filter("SELECT * FROM bundle WHERE active = true") # Filter rows
c = await c.drop_column("temp_field") # Remove columns
c = await c.rename_column("old", "new") # Rename columns
rs = await c.query("SELECT * FROM bundle WHERE id > 10") # SQL queries
df = await rs.to_pandas()
Data Export
df = await c.to_pandas() # → pandas DataFrame
df = await c.to_polars() # → polars DataFrame
arr = await c.to_numpy() # → dict of NumPy arrays
data = await c.to_dict() # → Python dict
Indexing
c = await c.create_index("email", "btree") # Create index for fast lookups
c = await c.rebuild_index("email") # Rebuild existing index
Joining
c = await bundlebase.create()
c = await c.attach("customers.parquet")
c = await c.join(
"orders",
"bundle.customer_id = orders.id",
location="orders.parquet",
join_type="inner",
)
Development
Prerequisites
- Rust (latest stable)
- Python 3.9+
- Poetry
Setup
# Install Python dependencies
poetry install
# Build Rust extension
./scripts/maturin-dev.sh
# Run tests
cargo test --profile test # Rust tests
poetry run pytest python/tests/ # Python tests
Contributing
Contributions are welcome!
License
Distributed under the Apache 2.0 license.
Metadata
Release files for bundlebase 0.13.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| bundlebase-0.13.0.tar.gz | 956.4 kB | Without provenance | Details for bundlebase-0.13.0.tar.gz |
Built distributions (wheels)
Total release size: 395.5 MB
Release files / bundlebase-0.13.0.tar.gz
| Download URL | bundlebase-0.13.0.tar.gz |
|---|---|
| Size | 956.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8b0eef539b33bc6ee0603a5d72c61d7f53da25af6d5557de29a9f0f325ba691e
|
|
BLAKE2b-256 checksum How to use checksums |
caaf95bd725591570b5a994b8d1bfd858d7c090297e3a7db19e52c263b5191cd
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / bundlebase-0.13.0-cp314-cp314-win_amd64.whl
| Download URL | bundlebase-0.13.0-cp314-cp314-win_amd64.whl |
|---|---|
| Size | 70.6 MB |
| Tags | CPython 3.14 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
46f3a21ad77a0de1275607acc678e12555f23fb04bfecb53df7f39416e748d89
|
|
BLAKE2b-256 checksum How to use checksums |
1ac03c56e90b5c521ac03a5f2e1110f94c159f716e3f07147890e0110c41aadf
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / bundlebase-0.13.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | bundlebase-0.13.0-cp314-cp314-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 66.2 MB |
| Tags | CPython 3.14 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
209f39253d99937c0e710bf7ed625cd5724a157056e122313e07ee80cbb99ab9
|
|
BLAKE2b-256 checksum How to use checksums |
3c696800b9a035b28305166d81a0acba2707357cd056f2c3fced9a799ae8b331
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / bundlebase-0.13.0-cp314-cp314-macosx_11_0_arm64.whl
| Download URL | bundlebase-0.13.0-cp314-cp314-macosx_11_0_arm64.whl |
|---|---|
| Size | 60.4 MB |
| Tags | CPython 3.14 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
2012241edc37c7e7a32b62d9f12095a3ee72fc79da58f451288119e2f2930e07
|
|
BLAKE2b-256 checksum How to use checksums |
f218944c15d6b866672620c247b942adb0414350a48fb243f9315d8398ce5f27
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / bundlebase-0.13.0-cp313-cp313-win_amd64.whl
| Download URL | bundlebase-0.13.0-cp313-cp313-win_amd64.whl |
|---|---|
| Size | 70.6 MB |
| Tags | CPython 3.13 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
0bc74555825e925266df7bf0f6bb328a35ff0d22152a95d38b806707ccc246e4
|
|
BLAKE2b-256 checksum How to use checksums |
0f3f9bc7ebc1c74529884e84d3d00f912d481a6a9420e9ab4a15171a59c0434f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / bundlebase-0.13.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | bundlebase-0.13.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 66.2 MB |
| Tags | CPython 3.13 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
b2d256c3b288ead61f5a35f4d3fae831e668053b5bf16257c5bba73d53de84f1
|
|
BLAKE2b-256 checksum How to use checksums |
fb6ed987a5b5299ef70883f9a6eec7a6df3c0746bce9f08eb95f9e32c6d15ae6
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Release files / bundlebase-0.13.0-cp313-cp313-macosx_11_0_arm64.whl
| Download URL | bundlebase-0.13.0-cp313-cp313-macosx_11_0_arm64.whl |
|---|---|
| Size | 60.4 MB |
| Tags | CPython 3.13 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
ef33ddd359e2d7d8a94f065b1a2f2a1a4599cc0c1062822ff21aac203a9e8541
|
|
BLAKE2b-256 checksum How to use checksums |
eaf4508caf38796e01989001cb310bb19323196cd82f3812c4d1af35a5a977dc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.13.14
|
Security for bundlebase 0.13.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.