Skip to main content

A fast Python implementation of locality sensitive hashing with persistance support.

Project description

Version:
0.0.4dev

A fast Python implementation of locality sensitive hashing with persistance support.

Highlights

  • Fast hash calculation for large amount of high dimensional data through the use of numpy arrays.

  • Built-in support for persistency through Redis.

  • Multiple hash indexes support.

  • Built-in support for common distance/objective functions for ranking outputs.

Installation

LSHash depends on the following libraries:

  • numpy

  • redis (if persistency through Redis is needed)

  • bitarray (if hamming distance is used as distance function)

To install:

$ pip install lshash

Quickstart

To create 6-bit hashes for input data of 8 dimensions:

>>> from lshash import LSHash

>>> lsh = LSHash(6, 8)
>>> lsh.index([1,2,3,4,5,6,7,8])
>>> lsh.index([2,3,4,5,6,7,8,9])
>>> lsh.index([10,12,99,1,5,31,2,3])
>>> lsh.query([1,2,3,4,5,6,7,7])
[((1, 2, 3, 4, 5, 6, 7, 8), 1.0),
 ((2, 3, 4, 5, 6, 7, 8, 9), 11)]

Main Interface

  • To initialize a LSHash instance:

LSHash(hash_size, input_dim, num_of_hashtables=1, storage=None, matrices_filename=None, overwrite=False)

parameters:

hash_size:

The length of the resulting binary hash.

input_dim:

The dimension of the input vector.

num_hashtables = 1:

(optional) The number of hash tables used for multiple lookups.

storage = None:

(optional) Specify the name of the storage to be used for the index storage. Options include “redis”.

matrices_filename = None:

(optional) Specify the path to the .npz file random matrices are stored or to be stored if the file does not exist yet

overwrite = False:

(optional) Whether to overwrite the matrices file if it already exist

  • To index a data point of a given LSHash instance, e.g., lsh:

lsh.index(input_point, extra_data=None):

parameters:

input_point:

The input data point is an array or tuple of numbers of input_dim.

extra_data = None:

(optional) Extra data to be added along with the input_point.

  • To query a data point against a given LSHash instance, e.g., lsh:

lsh.query(query_point, num_results=None, distance_func="euclidean"):

parameters:

query_point:

The query data point is an array or tuple of numbers of input_dim.

num_results = None:

(optional) The number of query results to return in ranked order. By default all results will be returned.

distance_func = "euclidean":

(optional) Distance function to use to rank the candidates. By default euclidean distance function will be used.

v0.0.3, 2012/12/28 – Doc fixes. v0.0.2, 2012/12/28 – Doc fixes and lowercase package name. v0.0.1, 2012/12/20 – Initial release.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lshash-0.0.4dev.tar.gz (7.2 kB view details)

Uploaded Source

File details

Details for the file lshash-0.0.4dev.tar.gz.

File metadata

  • Download URL: lshash-0.0.4dev.tar.gz
  • Upload date:
  • Size: 7.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for lshash-0.0.4dev.tar.gz
Algorithm Hash digest
SHA256 556442ba36f939aec8df7f50ed9251a09a45d434346f94205b2144aa854c7153
MD5 c4a7606b4be59ec302b1a19ef8154052
BLAKE2b-256 2b9e75e5ce8eec67126e4919f2410ccccf84ebd8db90e2cd28d8ffcb05905419

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page