naan
Table of Contents
What is Naan?
- Naan is a wrapper around FAISS indexes that provides metadata storage and retrieval for the vectors added to the index.
- Naan's job is to eliminate the tedious task of keeping around the original content before it's encoded and added to the index.
- Naan is NOT a vector database. All the vector-search operations are demanded to FAISS.
Installation
pip install naan
Index data
To see Naan in action, let's first get some data to embed:
from io import StringIO
import requests
import json
res = requests.get("https://raw.githubusercontent.com/masci/naan/main/example/sentences.json")
sentences = json.load(StringIO(res.text))
Naan tries not to get in the way you manage your FAISS index, so the first step is always setting up the FAISS side of things:
from sentence_transformers import SentenceTransformer
import faiss
model = SentenceTransformer("bert-base-nli-mean-tokens")
sentence_embeddings = model.encode(sentences)
dim = sentence_embeddings.shape[1]
index = faiss.IndexFlatL2(dim)
Now it's time to wrap the FAISS index with Naan and use it to index data:
from naan import NaanDB
# Create a Naan database from scratch
db = NaanDB("db.naan", index, force_recreate=True)
db.add(sentence_embeddings, sentences)
Naan will add the vector embeddings to the FAISS index, and will also store the original sentences. This way, a vector search will look like this:
# Reopen an existing Naan database
db = NaanDB("db.naan")
query_embeddings = model.encode(["The book is on the table"])
# Naan's search API is the same as FAISS, let's get the 3 closest vectors
results = db.search(query_embeddings, 3)
for result in results:
print(result)
# Document(vector_id=11451, content='A group of people sitting around a desk.', embeddings=None)
# Document(vector_id=2754, content='A close-up picture of a desk with a computer and papers on it.', embeddings=None)
# Document(vector_id=11853, content='A computer on a desk.', embeddings=None)
License
naan is distributed under the terms of the MIT license.
Metadata
Release files for naan 0.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| naan-0.0.4.tar.gz | 6.4 kB | Without provenance | Details for naan-0.0.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| naan-0.0.4-py3-none-any.whl | Python 3 | none | any | Details for naan-0.0.4-py3-none-any.whl |
Total release size: 12.9 kB
Release files / naan-0.0.4.tar.gz
| Download URL | naan-0.0.4.tar.gz |
|---|---|
| Size | 6.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8e8d4b32e1c96ab6dea15e86df891f6a56f721e17085e5020ac6bb123b92b984
|
|
BLAKE2b-256 checksum How to use checksums |
3153d2182f75c917b40fe8679f2ff9f847f5a2400bce534b4546ed7953737397
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
python-httpx/0.27.0
|
Release files / naan-0.0.4-py3-none-any.whl
| Download URL | naan-0.0.4-py3-none-any.whl |
|---|---|
| Size | 6.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
cedc458dfc35a5d3cf9c1e53751698858a829d9407a19e9493b9e09d6a66a5bf
|
|
BLAKE2b-256 checksum How to use checksums |
8618ba235729e63f180e6f483953630955ccbc5c676f7de0607f17d5aa029581
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
python-httpx/0.27.0
|
Security for naan 0.0.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.