GraphVec

GraphVec is a local, file-based vector search library for Python. It stores an index in a directory on disk and finds nearest neighbours with its own HNSW-style graph. It needs no database server, cloud service, or background process.

It is written in pure Python (3.11+) and NumPy. That makes it easy to read and embed, but it is slower than compiled libraries such as FAISS or hnswlib.

Installation

pip install graphvec-ann           # the import name and command are `graphvec`
pip install "graphvec-ann[parquet]"  # optional Parquet import

Quick start

import numpy as np
from graphvec import GraphIndex

rng = np.random.default_rng(42)
vectors = rng.random((1_000, 128), dtype=np.float32)
ids = [f"doc-{i}" for i in range(len(vectors))]

with GraphIndex.create("./vectors.gvec", dimensions=128, metric="cosine") as index:
    index.add(vectors, ids=ids)
    index.save()  # nothing is written to disk until save()

with GraphIndex.open("./vectors.gvec", readonly=True) as index:
    query = rng.random(128, dtype=np.float32)
    for result in index.search(query, k=5, ef_search=100):
        print(result.id, result.score)

Python API

Higher ef_search gives better recall but slower queries. All errors inherit from graphvec.GraphVecError.

Metrics (metric=):

• cosine: score is cosine similarity, higher is better. Vectors are normalised when added, so get() returns the normalised vector. Zero vectors score 0.
• dot: score is the dot product, higher is better.
• squared_euclidean: score is the squared L2 distance, lower is better.

Command line

graphvec create ./index.gvec --dimensions 128 --metric cosine
graphvec import ./index.gvec vectors.npy --ids ids.json   # or .jsonl / .parquet
graphvec search ./index.gvec --query-file query.npy --top-k 10
graphvec info ./index.gvec
graphvec validate ./index.gvec
graphvec benchmark ./index.gvec --queries queries.npy --top-k 10
graphvec compact ./index.gvec
graphvec unlock ./index.gvec                              # after a crashed writer
graphvec export ./index.gvec --output vectors.npy         # also writes vectors.npy.ids.json

JSONL records look like {"id": "doc-1", "vector": [0.1, ...]}; Parquet files need id and vector columns and the parquet extra. Most commands accept --json; see graphvec --help.

Limitations

• Read-only opens memory-map vectors and the graph. Writable indexes keep them in RAM up to memory_limit (default: a quarter of physical memory), then in memory-mapped working files, so building can exceed RAM; the ID map always stays in RAM.
• One writer at a time. A crash leaves a stale lock; clear it with graphvec unlock.
• Deleted IDs cannot be reused until compact().
• Insertion is slow (hundreds of vectors per second in our measurements).
• More detail and crash-safety guarantees: docs/limitations.md.

Documentation

Architecture and algorithms · File format · Benchmarks · Changelog · Security

Development

Add tests for behaviour changes, and never commit benchmark numbers you did not measure.

pip install -e ".[dev]"
pytest                                   # add -m slow for large tests
ruff check . && ruff format --check .
mypy src/graphvec

License

MIT. See LICENSE.