Tiny unofficial ChromaDB operations CLI to help you manage your ChromaDB instance.
Project description
ChromaDB Operations Tools
Tiny collection of utilities to help you managed ChromaDB indices.
WARNING: These tools rely on internal ChromaDB APIs and may break in the future.
☠️☠️☠️ BEFORE YOU BEGIN ☠️☠️☠️
Before you use these tools make sure your ChromaDB persistent dir, on which you intend to run these tools, is backed up.
Installation
pip install chromadb-ops
Usage
WAL Commit
This command ensures your WAL is committed to binary vector index (HNSW).
chops commit-wal /path/to/persist_dir
Note: You can skip certain collections by running
chops commit-wal /path/to/persist_dir --skip <collection_name>
WAL Cleanup
This command cleans up the committed portion of the WAL and VACUUMs the database.
chops cleanup-wal /path/to/persist_dir
WAL Export
This commands exports the WAL to a jsonl
file. The command can be useful in taking backups of the WAL.
chops export-wal /path/to/persist_dir --out /path/to/export.jsonl
Note: If --out or -o is not specified the command will print the output to stdout.
Full-Text Search Index Rebuild
This command rebuilds the full-text search index.
Note: Why is this needed? Users have reported broken FTS indices that result in a error of this kind:
no such table: embedding_fulltext_search
chops rebuild-fts /path/to/persist_dir
Using Docker
Note: You have to mount your persist directory into the container for the commands to work.
Building the image:
docker build -t chops .
WAL Commit
docker run -it --rm -v ./persist_dir:/chroma-data ghcr.io/amikos-tech/chromadb-ops/chops:latest commit-wal /chroma-data
WAL Cleanup
docker run -it --rm -v ./persist_dir:/chroma-data ghcr.io/amikos-tech/chromadb-ops/chops:latest clean-wal /chroma-data
WAL Export
docker run -it --rm -v ./persist_dir:/chroma-data -v ./backup:/backup ghcr.io/amikos-tech/chromadb-ops/chops:latest export-wal /chroma-data --out /backup/export.jsonl
Full-Text Search Index Rebuild
docker run -it --rm -v ./persist_dir:/chroma-data ghcr.io/amikos-tech/chromadb-ops/chops:latest rebuild-fts /chroma-data
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for chromadb_ops-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e6a12bd211ea8a8215add63e1736c26dfeaf232d77cacef2a01e0786f02eca69 |
|
MD5 | 49c474f14f255abd48748bb2f7852941 |
|
BLAKE2b-256 | 5a7b38baa369ee6e01a0a2eb042ca1b66290146ff1256fc5a5ee9b4782d99c7b |