Skip to main content

Pure python implementation of product quantization for nearest neighbor search

Project description

nanopq

Build Status Documentation Status PyPI version Downloads

Nano Product Quantization (nanopq): a vanilla implementation of Product Quantization (PQ) and Optimized Product Quantization (OPQ) written in pure python without any third party dependencies.

Installing

You can install the package via pip. This library works with Python 3.5+ on linux.

pip install nanopq

Documentation

Example

import nanopq
import numpy as np

N, Nt, D = 10000, 2000, 128
X = np.random.random((N, D)).astype(np.float32)  # 10,000 128-dim vectors to be indexed
Xt = np.random.random((Nt, D)).astype(np.float32)  # 2,000 128-dim vectors for training
query = np.random.random((D,)).astype(np.float32)  # a 128-dim query vector

# Instantiate with M=8 sub-spaces
pq = nanopq.PQ(M=8)

# Train codewords
pq.fit(Xt)

# Encode to PQ-codes
X_code = pq.encode(X)  # (10000, 8) with dtype=np.uint8

# Results: create a distance table online, and compute Asymmetric Distance to each PQ-code 
dists = pq.dtable(query).adist(X_code)  # (10000, ) 

Author

Contributors

Reference

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nanopq-0.2.2.tar.gz (13.5 kB view details)

Uploaded Source

Built Distribution

nanopq-0.2.2-py3-none-any.whl (11.3 kB view details)

Uploaded Python 3

File details

Details for the file nanopq-0.2.2.tar.gz.

File metadata

  • Download URL: nanopq-0.2.2.tar.gz
  • Upload date:
  • Size: 13.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for nanopq-0.2.2.tar.gz
Algorithm Hash digest
SHA256 27564b13f06b8b42821730bf8fb981c85e9a10fa08cc3b919b7f84bbaf21771f
MD5 256297515220906b3662ac7a508aa917
BLAKE2b-256 ac1754352917ab78a1589162b112a1223c28eecf8b14efa957573b26d63af009

See more details on using hashes here.

Provenance

The following attestation bundles were made for nanopq-0.2.2.tar.gz:

Publisher: publish.yml on matsui528/nanopq

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file nanopq-0.2.2-py3-none-any.whl.

File metadata

  • Download URL: nanopq-0.2.2-py3-none-any.whl
  • Upload date:
  • Size: 11.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for nanopq-0.2.2-py3-none-any.whl
Algorithm Hash digest
SHA256 e00eff2d23e37e077b1eb7ad30afc43c97d524f37c771074bf77de1003d1fcf9
MD5 c01ea34f324d0d227123ff9087a748a1
BLAKE2b-256 55624927b957cbd9ec6623f2c9c20d4abf87a1a5052be73fec38e1aa3b3b6120

See more details on using hashes here.

Provenance

The following attestation bundles were made for nanopq-0.2.2-py3-none-any.whl:

Publisher: publish.yml on matsui528/nanopq

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page