Skip to main content

mlxVS

cuVS for your Mac: GPU vector search on Apple silicon.

CI License

Index a million vectors in 11 seconds, then answer 76,000 queries per second at 97.6% recall, on a MacBook Pro's GPU: 2.7 times the queries per second of HNSW on all 12 CPU cores at the same recall, from an index built 8 times faster. On a million 1536-d OpenAI embeddings: built 6 times faster, twice the queries per second (numbers). mlxVS brings NVIDIA cuVS's operations and parameter names to the Mac, built on MLX: exact k-NN, k-means, NN-Descent, batched all-neighbors graphs, CAGRA graph pruning and search, and export to hnswlib. NumPy or MLX arrays in, any M-series Mac.

import mlxvs
index = mlxvs.CagraIndex.build(vectors, graph_degree=32)   # 1M x 128 in ~11 s
ids, dist = index.search(queries, k=10)                     # ~76k queries/s at 97.6% recall
index.save("vectors.safetensors")
Queries per second against recall@10 on a MacBook Pro M3 Pro. SIFT1M: mlxVS float32 reaches 76,400 queries/s at 0.976 recall (index built in 10.8 s); hnswlib on 12 CPU cores reaches 28,100 at 0.964 (built in 91.8 s). ada-002 1M: mlxVS 10,000 queries/s at 0.979 (built in 125 s); hnswlib 5,100 at 0.974 (built in 732 s). Numbers in the Benchmarks tables.

Use it for approximate nearest-neighbor search, for the k-NN graph behind UMAP, t-SNE, clustering and deduplication, for exact k-NN ground truth, or as the build step of a graph index (HNSW, Vamana, CAGRA).

What's in it

Function What it does cuVS counterpart
brute_force_knn(base, queries, k) Exact k-NN: matrix multiply and a fused top-k kernel, tiled over the base brute_force::search
kmeans_fit(x, clusters), kmeans_predict(x, centroids, m) k-means++ and Lloyd; the m nearest centroids per row cluster::kmeans
nn_descent(rows, k) Approximate k-NN graph of the rows, FP16 SIMD-group joins nn_descent::build
all_neighbors(rows, k) k-NN graph built in overlapping k-means clusters, one on the GPU at a time; reads a NumPy array or memory-mapped file larger than memory where it is all_neighbors::build
cagra_build(rows, graph_degree) Search graph: k-NN graph (all_neighbors) pruned by cagra_optimize cagra::build
cagra_optimize(knn_graph, graph_degree) CAGRA's rank-based pruning (fewest two-hop detours) plus reverse edges cagra::optimize
graph_search(base, graph, queries, k) Approximate k-NN of queries by beam search of a graph, one threadgroup per query (CAGRA's single-CTA search) cagra::search
CagraIndex.build(rows, graph_degree), .search(queries, k), .save(path), CagraIndex.load(path) A search index: the dataset (float32 or float16, normalized for cosine) and its CAGRA graph, stored once for search; one .safetensors file cagra::build, search, serialize
CagraIndex.save_hnswlib(path) The index in hnswlib's format: build on the GPU, serve with hnswlib on any CPU cagra::serialize_to_hnswlib
NearestNeighbors(n_neighbors).fit(X).kneighbors(Q) scikit-learn's interface over the above, NumPy in and out

Metrics: "l2" (squared), "cosine", "inner_product". Next: Vamana pruning and export to HNSW and JVector; see Roadmap.

1. Install

Needs an Apple-silicon Mac on macOS 14 or later (as MLX) and Python 3.10+.

pip install mlxvs

Wheels for Python 3.10 to 3.14 on Apple silicon, built against the mlx release they pin (MLX has no stable C++ ABI). The latest code from GitHub builds the extension against your mlx wheel; that needs the Xcode Command Line Tools (xcode-select --install) and takes about a minute:

pip install git+https://github.com/mikepapadim/mlxvs

2. Use

import numpy as np
import mlxvs

base = np.random.rand(1_000_000, 128).astype(np.float32)
queries = base[:10_000]

# exact k-NN: ids [nq, k], distances [nq, k], nearest first
ids, dist = mlxvs.brute_force_knn(base, queries, k=10, metric="l2")

# approximate k-NN graph: every row's 32 nearest other rows
graph, graph_dist = mlxvs.nn_descent(base, k=32, metric="cosine")

# the same graph in overlapping k-means clusters, as many as the GPU's memory asks for
graph, graph_dist = mlxvs.all_neighbors(base, k=32, metric="cosine")

# a search index: the k-NN graph pruned by CAGRA's rules, with the rows stored for search
# (float16: half the memory); itopk_size trades speed for recall
index = mlxvs.CagraIndex.build(base, graph_degree=32, metric="cosine", dtype="float16")
ids, sims = index.search(queries, k=10, itopk_size=64)
index.save("index.safetensors")
index = mlxvs.CagraIndex.load("index.safetensors")

# or serve it from the CPU, anywhere hnswlib runs (Linux servers too): labels are row ids
index.save_hnswlib("index.hnsw")   # then hnswlib.Index(space="cosine", dim=128).load_index("index.hnsw")

graph = np.asarray(graph)  # results are mlx arrays; np.asarray converts without a copy
  • Inputs: MLX arrays, NumPy arrays, or anything np.asarray accepts, in any real dtype (computed in float32).
  • Outputs: int32 ids, -1 where fewer neighbors exist; float32 distances, squared for "l2", similarities (larger is closer) for "cosine" and "inner_product".
  • help(mlxvs.nn_descent) and the others document every parameter.
  • Searching many times: use CagraIndex, or pass the same MLX array to graph_search each time. A NumPy array is copied to the GPU on every call, and graph_search with "cosine" recomputes the row norms on every call; CagraIndex stores the rows once, normalized.

Data larger than memory: pass a numpy.memmap. The rows stay on disk; all_neighbors copies one cluster at a time to the GPU.

rows = np.memmap("embeddings.f32", dtype=np.float32, mode="r").reshape(-1, 1024)
graph, graph_dist = mlxvs.all_neighbors(rows, k=32, metric="cosine", verbose=True)

Coming from scikit-learn, change the import:

nn = mlxvs.NearestNeighbors(n_neighbors=10, metric="euclidean").fit(base)
distances, indices = nn.kneighbors(queries)   # NumPy arrays, scikit-learn's units
distances, indices = nn.kneighbors()          # the base rows' own neighbors

Examples: UMAP from a GPU k-NN graph and exact ground truth for an ANN benchmark (.fvecs in, .ivecs out).

C++: the same functions in include/mlxvs/mlxvs.h, taking mx::array and a trailing mx::StreamOrDevice s = {} as MLX ops do. In CMake (MLX found through CMAKE_PREFIX_PATH, e.g. Homebrew's):

add_subdirectory(mlxvs)          # or FetchContent
target_link_libraries(app PRIVATE mlxvs)

3. Run the benchmarks

curl -O ftp://ftp.irisa.fr/local/texmex/corpus/sift.tar.gz && tar xzf sift.tar.gz
pip install faiss-cpu pynndescent   # optional CPU baselines
python bench/bench_graph.py sift/sift_base.fvecs --k 32 --clusters 1,8 --baselines faiss,pynndescent

It builds the k-NN graph with each method and scores 2,000 random rows against the exact neighbors.

Search: build a graph, then recall@10 against queries per second over a sweep of itopk_size, optionally against hnswlib on the CPU.

pip install hnswlib   # optional CPU baseline
python bench/bench_search.py sift/sift_base.fvecs sift/sift_query.fvecs --degree 32 --intermediate 64 \
    --baselines hnswlib

Benchmarks

MacBook Pro M3 Pro (18-core GPU, 12-core CPU, 36 GB), on power. 10,000 queries in one batch, recall@10 scored from the true distances of the returned ids. mlxVS: CagraIndex, graph degree 32 pruned from 64. hnswlib 0.8: M=16, ef_construction=200, all 12 cores. Reproduce with bench/bench_search.py.

SIFT1M (985,462 x 128, L2)

Index Build Setting Recall@10 Queries/s
mlxVS, float32 10.8 s itopk_size 32 0.932 134,900
itopk_size 64 0.976 76,400
itopk_size 128 0.993 46,200
itopk_size 256 0.998 26,700
mlxVS, float16 10.9 s itopk_size 64 0.976 84,400
itopk_size 128 0.993 56,700
hnswlib (CPU) 91.8 s ef 32 0.903 50,100
ef 64 0.964 28,100
ef 128 0.990 15,000
ef 256 0.998 9,700

ada-002 1M (982,790 x 1536 OpenAI embeddings, cosine)

Index Build Setting Recall@10 Queries/s
mlxVS, float32 125 s itopk_size 32 0.952 19,100
itopk_size 64 0.979 10,000
itopk_size 128 0.991 5,600
mlxVS, float16 121 s itopk_size 64 0.975 12,800
itopk_size 128 0.987 7,000
hnswlib (CPU) 732 s ef 32 0.942 8,200
ef 64 0.974 5,100
ef 128 0.990 3,000

One query at a time (SIFT1M, 1,000 queries each timed alone): the CPU wins this case. A single query is a short chain of dependent steps, which suits one fast CPU core; on the GPU it occupies one of 18 cores and pays MLX's launch overhead. A batch of 64 takes about as long as one query (1.1 ms).

Index Setting Recall@10 Median latency p95
mlxVS (GPU), batch of 1 itopk_size 64 0.973 0.95 ms 1.34 ms
itopk_size 128 0.992 1.69 ms 1.97 ms
hnswlib (CPU), 1 thread ef 64 0.961 0.21 ms 0.26 ms
ef 128 0.988 0.38 ms 0.48 ms

So: build on the GPU (6 to 9x faster), search on the GPU when queries come in batches or from many users at once, and for one query at a time at low load, save_hnswlib and serve from the CPU.

Coming from cuVS

mlxvs cuVS Note
nn_descent(rows, k, ...) nn_descent::index_params::graph_degree cuVS default 64
lists of max(64, k) entries intermediate_graph_degree cuVS default 128
max_iterations = 24 max_iterations = 20 mlxvs samples 16 new neighbors per round and stops early when the lists settle
termination_threshold = 1e-4 termination_threshold = 1e-3 tuned for graph quality in JVector; pass 1e-3 for cuVS's behaviour
all_neighbors(..., n_clusters, overlap_factor) all_neighbors::index_params::{n_clusters, overlap_factor} overlap 2 as cuVS; n_clusters defaults to 0 (chosen from the GPU's memory), cuVS's to 1
algo="brute_force" / "nn_descent" all_neighbors::index_params::graph_build_params cuVS also offers IVF-PQ
"cosine" returns the cosine similarity CosineExpanded returns 1 - cosine similarities are "larger is closer"; "l2" is squared, as L2Expanded
cagra_build(rows, graph_degree=32, intermediate_graph_degree=64) cagra::index_params::{graph_degree, intermediate_graph_degree} cuVS defaults 64 and 128; NN-Descent here caps the intermediate degree at 128
cagra_optimize(knn_graph, graph_degree) cagra::optimize rank-based pruning and reverse edges, as cuVS; the reverse half fills from the lowest-rank edges
graph_search(base, graph, queries, k, ...) cagra::search takes the dataset and graph directly, no index object yet
itopk_size = 64, search_width = 1 search_params::{itopk_size, search_width} same defaults; itopk_size at most 256
max_iterations = 0 (automatic) search_params::max_iterations stops earlier once every list entry is expanded
num_random_samplings = 1 search_params::num_random_samplings random start nodes; no seed nodes yet
int32 ids int64_t ids n < 2^31 rows

Roadmap

  1. Export to JVector's layout; Vamana alpha pruning.
  2. Lower single-query latency: several threadgroups per query for small batches (CAGRA's multi-CTA search). One query takes about 1 ms today (SIFT1M, recall 0.97); hnswlib on one CPU thread takes 0.2 ms.
  3. int8 datasets; filtered search.
  4. A C API for other languages (Java via Panama, as cuVS-java does).

How the kernels work: docs/DESIGN.md. Building, testing and code style: CONTRIBUTING.md.

License

Apache-2.0. The NN-Descent kernels are a port of JVector's jvector-gpu module (Apache-2.0, DataStax, Inc.); see NOTICE.

Metadata

Release files for mlxvs 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mlxvs 0.2.0
File Size Uploaded Provenance Actions
mlxvs-0.2.0.tar.gz 58.2 kB With provenance Details for mlxvs-0.2.0.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for mlxvs 0.2.0
File Actions
mlxvs-0.2.0-cp314-cp314-macosx_14_0_arm64.whl CPython 3.14 CPython 3.14 macOS 14.0+ ARM64 Details for mlxvs-0.2.0-cp314-cp314-macosx_14_0_arm64.whl
mlxvs-0.2.0-cp313-cp313-macosx_14_0_arm64.whl CPython 3.13 CPython 3.13 macOS 14.0+ ARM64 Details for mlxvs-0.2.0-cp313-cp313-macosx_14_0_arm64.whl
mlxvs-0.2.0-cp312-cp312-macosx_14_0_arm64.whl CPython 3.12 CPython 3.12 macOS 14.0+ ARM64 Details for mlxvs-0.2.0-cp312-cp312-macosx_14_0_arm64.whl
mlxvs-0.2.0-cp311-cp311-macosx_14_0_arm64.whl CPython 3.11 CPython 3.11 macOS 14.0+ ARM64 Details for mlxvs-0.2.0-cp311-cp311-macosx_14_0_arm64.whl
mlxvs-0.2.0-cp310-cp310-macosx_14_0_arm64.whl CPython 3.10 CPython 3.10 macOS 14.0+ ARM64 Details for mlxvs-0.2.0-cp310-cp310-macosx_14_0_arm64.whl

Total release size: 816.1 kB

Release files / mlxvs-0.2.0.tar.gz

Download URL mlxvs-0.2.0.tar.gz
Size 58.2 kB
Tags Source
SHA-256 checksum
How to use checksums
5754a3a99338c49160bf6ad5af3f552da8a78c3971f9c708f310d892e5e2e4c4
BLAKE2b-256 checksum
How to use checksums
eb61f45b4d9536ecde2c4da1384527e4262cba8c78441643224d23a0f3b6a5c6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Release files / mlxvs-0.2.0-cp314-cp314-macosx_14_0_arm64.whl

Download URL mlxvs-0.2.0-cp314-cp314-macosx_14_0_arm64.whl
Size 151.6 kB
Tags CPython 3.14 macOS 14.0+ ARM64
SHA-256 checksum
How to use checksums
efdba20cbf464f908d3d178f51bb4d6a6e308104f2722151db2ee0c02f347f62
BLAKE2b-256 checksum
How to use checksums
90ec1505dfe2d79855dce6af4921ed822ea862bc361a21758fcadcb76d0c1927
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Release files / mlxvs-0.2.0-cp313-cp313-macosx_14_0_arm64.whl

Download URL mlxvs-0.2.0-cp313-cp313-macosx_14_0_arm64.whl
Size 151.4 kB
Tags CPython 3.13 macOS 14.0+ ARM64
SHA-256 checksum
How to use checksums
6289646d99ec90f7720b19a0aaab5e35bb635b1ddd2734fa22fdfae05dab3b2c
BLAKE2b-256 checksum
How to use checksums
5b397a0b7f1c5020955553d1dbd1e0cb04c4889ae4c9332da1c2f167970e9966
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Release files / mlxvs-0.2.0-cp312-cp312-macosx_14_0_arm64.whl

Download URL mlxvs-0.2.0-cp312-cp312-macosx_14_0_arm64.whl
Size 151.4 kB
Tags CPython 3.12 macOS 14.0+ ARM64
SHA-256 checksum
How to use checksums
e8f24564cbbb394d1b2c603abb5b3065efec151948aae6816c5496e76dc36c02
BLAKE2b-256 checksum
How to use checksums
85aa89ad1ee1f68e574c21ea135ab640224ad56d188de34fb634ecebeacb213a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Release files / mlxvs-0.2.0-cp311-cp311-macosx_14_0_arm64.whl

Download URL mlxvs-0.2.0-cp311-cp311-macosx_14_0_arm64.whl
Size 152.0 kB
Tags CPython 3.11 macOS 14.0+ ARM64
SHA-256 checksum
How to use checksums
9416a24409d3ef4873365e17d400b052de94bdd20645dbed4cf82d0eeebe147f
BLAKE2b-256 checksum
How to use checksums
17417647b21d7463526f7a82fb91716812b6f6fb075dcf56ee83a8a7e2d3920c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Release files / mlxvs-0.2.0-cp310-cp310-macosx_14_0_arm64.whl

Download URL mlxvs-0.2.0-cp310-cp310-macosx_14_0_arm64.whl
Size 151.4 kB
Tags CPython 3.10 macOS 14.0+ ARM64
SHA-256 checksum
How to use checksums
31f367aa2fa945df8d84a25dbe928c1b59caa5b786b4c5269edd52f6da36a18a
BLAKE2b-256 checksum
How to use checksums
669d19ed66913a346206444e30f1531d99d5bd80aa3d12d701c3adf5415057c0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

Security for mlxvs 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 6 files in this release have been uploaded with provenance metadata.

Select from 6 files

Attestation for mlxvs-0.2.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 11, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page