Tiny unofficial ChromaDB operations CLI to help you manage your ChromaDB instance.
Project description
ChromaDB Operations Tools
Tiny collection of utilities to help you managed ChromaDB indices.
WARNING: These tools rely on internal ChromaDB APIs and may break in the future.
☠️☠️☠️ BEFORE YOU BEGIN ☠️☠️☠️
Before you use these tools make sure your ChromaDB persistent dir, on which you intend to run these tools, is backed up.
Installation
pip install chromadb-ops
Usage
WAL Commit
This command ensures your WAL is committed to binary vector index (HNSW).
chops commit-wal /path/to/persist_dir
Note: You can skip certain collections by running
chops commit-wal /path/to/persist_dir --skip <collection_name>
WAL Cleanup
This command cleans up the committed portion of the WAL and VACUUMs the database.
chops clean-wal /path/to/persist_dir
WAL Export
This commands exports the WAL to a jsonl
file. The command can be useful in taking backups of the WAL.
chops export-wal /path/to/persist_dir --out /path/to/export.jsonl
Note: If --out or -o is not specified the command will print the output to stdout.
Full-Text Search Index Rebuild
This command rebuilds the full-text search index.
Note: Why is this needed? Users have reported broken FTS indices that result in a error of this kind:
no such table: embedding_fulltext_search
chops rebuild-fts /path/to/persist_dir
Using Docker
Note: You have to mount your persist directory into the container for the commands to work.
Building the image:
docker build -t chops .
WAL Commit
docker run -it --rm -v ./persist_dir:/chroma-data ghcr.io/amikos-tech/chromadb-ops/chops:latest commit-wal /chroma-data
WAL Cleanup
docker run -it --rm -v ./persist_dir:/chroma-data ghcr.io/amikos-tech/chromadb-ops/chops:latest clean-wal /chroma-data
WAL Export
docker run -it --rm -v ./persist_dir:/chroma-data -v ./backup:/backup ghcr.io/amikos-tech/chromadb-ops/chops:latest export-wal /chroma-data --out /backup/export.jsonl
Full-Text Search Index Rebuild
docker run -it --rm -v ./persist_dir:/chroma-data ghcr.io/amikos-tech/chromadb-ops/chops:latest rebuild-fts /chroma-data
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for chromadb_ops-0.0.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b369afa8dbbba6a43e1f4861ce4b2d64923cead1eec506e025957b380ae350b0 |
|
MD5 | 27bf4c70af8d3074c234e68ebc7bcd45 |
|
BLAKE2b-256 | dce03ad755f7dad50cab6a6a5b96b643d017abc1de03bc0f143237e34449655d |