A distributed vector search microservice.
Project description
Needlestack is a distributed vector search microservice.
Free software: MIT license
Documentation: https://needlestack.readthedocs.io.
Features
gRPC server for kNN vector search
Shard vectors over multiple nodes
Replicate shard over multiple nodes
Retrieve vectors by ID
Caveats
These are some limitations of the current alpha builds
Vectors must be sharded and indexed before loading to Needlestack
To reload updated vectors, all MergerServicer and SearcherServicer nodes must be restarted
Vectors must be saved locally to disk for Needlestack to load them
Only kNN library supported currently is Faiss
Next Steps
These are valuable features to make Needlestack usable in production
Endpoint to index vectors in realtime
Update vectors from pre-built data sources without restarting the cluster
Allow vectors to be loaded from data sources like S3 or GCS
Add support for other kNN libraries
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for needlestack-0.0.7-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0f986d645a44fbcc34ee79a6dfc2fbbfa2957c61abb983a76a9bfdc1dd6ff9bb |
|
MD5 | 4c1172e22147c39084e67aed2bca9978 |
|
BLAKE2b-256 | 2c858cc2fe6ed2033a5265c176e1e5de4f1b94b55f10f11ea2f2b14abc1c66d9 |