Skip to main content

pyarrowspace

Python bindings for arrowspace-rs.

arrowspace is a graph-based analytics library for vector spaces, supported by a graph representation and a key-value store. The main use-cases targeted are: AI search capabilities as advanced vector similarity, graph characterisation analysis and search, indexing of high-dimensional vectors. Design principles described in this article.

For labs and tests please see tests/

Installation

Core library

The core library ships as a compiled Rust extension. It only requires numpy, pyarrow, pandas, and scikit-learn:

pip install arrowspace

Optional extras

Additional capabilities are available as optional extras:

Extra Packages included Use case
embeddings datasets, sentence-transformers, transformers[torch]<5.0, tsdae Generating embeddings from text via HuggingFace models; required for test_1_quora_questions.py and all tests above
benchmarks beir, nltk Running BEIR/MS-MARCO benchmark suites and NLP preprocessing
viz matplotlib, seaborn, tqdm Plotting results and progress bars in test scripts
full all of the above Full development and research environment

Install with one or more extras:

# Required to run test_1_quora_questions.py and above
pip install arrowspace[embeddings]

# Benchmarking + visualisation
pip install arrowspace[benchmarks,viz]

# Everything (for running the full test suite)
pip install arrowspace[full]

Note: tests/test_0_*.py only require the core install. All tests numbered test_1 and above require at minimum arrowspace[embeddings].

Build from source

If you have Cargo installed, you can compile and install locally using maturin:

pip install maturin[patchelf]
# quick development build
maturin develop
# optimised release build (recommended for large datasets)
maturin develop --release

Tests

test_0_*.py scripts only require the core install:

python tests/test_0_0.py

test_1 and above require the embeddings extra (pip install arrowspace[embeddings]):

python tests/test_1_quora_questions.py

Higher-numbered tests (test_3 and above) additionally require benchmarks and viz:

pip install arrowspace[full]
python tests/test_3_beir.py

Some tests require downloading a dataset separately or fine-tuning embeddings on a given dataset.

Simplest Example

from arrowspace import ArrowSpaceBuilder
import numpy as np

items: np.array = np.array(
    [[0.1, 0.2, 0.3], [0.0, 0.5, 0.1], [0.9, 0.1, 0.0]],
    dtype = np.float64
)

graph_params: dict = {
    "eps": 1.0,
    "k": 6,
    "topk": 3,
    "p": 2.0,
    "sigma": 1.0,
}

# Create an ArrowSpace instance, returning the computed
# signal graph and lambdas
aspace, gl = ArrowSpaceBuilder().build(graph_params, items)

# Search comparable items
# defaults: k = nitems, alpha = 0.9, beta = 0.1
query: np.array = np.array(
    [0.05, 0.2, 0.25],
    dtype = np.float64
)

tau: float = 1.0
hits: list = aspace.search(query, gl, tau)

# Search returns a list of `(index, score`) tuples, where
# expected value from the code above show the first index
# having the top score, i.e., being nearest.

print(hits)
# [ (0, 0.989743318610787), (1, 0.7565344158360029), (2, 0.22151940739207396) ]

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

arrowspace-0.26.4.tar.gz (17.7 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

arrowspace-0.26.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.5 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64

arrowspace-0.26.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (5.4 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ ARM64

arrowspace-0.26.4-cp313-cp313-macosx_11_0_arm64.whl (4.9 MB view details)

Uploaded CPython 3.13macOS 11.0+ ARM64

arrowspace-0.26.4-cp313-cp313-macosx_10_12_x86_64.whl (5.1 MB view details)

Uploaded CPython 3.13macOS 10.12+ x86-64

arrowspace-0.26.4-cp312-cp312-win_amd64.whl (4.8 MB view details)

Uploaded CPython 3.12Windows x86-64

arrowspace-0.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.5 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64

arrowspace-0.26.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (5.4 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ ARM64

arrowspace-0.26.4-cp312-cp312-macosx_11_0_arm64.whl (4.8 MB view details)

Uploaded CPython 3.12macOS 11.0+ ARM64

arrowspace-0.26.4-cp312-cp312-macosx_10_12_x86_64.whl (5.1 MB view details)

Uploaded CPython 3.12macOS 10.12+ x86-64

File details

Details for the file arrowspace-0.26.4.tar.gz.

File metadata

  • Download URL: arrowspace-0.26.4.tar.gz
  • Upload date:
  • Size: 17.7 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.12.1 {"installer":{"name":"uv","version":"0.12.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for arrowspace-0.26.4.tar.gz
Algorithm Hash digest
SHA256 79e98a330a6cc97e5859c1d4ba4b538e56f4cdc68764cc650cb34859a5f91215
MD5 09bbd562bc313d1a3c1e4b3038563556
BLAKE2b-256 19dc065507514d48c6ef08bd074a0069fc57023681d2892feac89a4bebd57725

See more details on using hashes here.

File details

Details for the file arrowspace-0.26.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arrowspace-0.26.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 bec4442258ee6e889a07779bcb751d45d2778942095716716c6209e567322673
MD5 6e0a680499b7de90ae7d3dd52bcef775
BLAKE2b-256 b56f1323c86bdc749163b8705de8b044e1d516658d9f0546c0060b6a1dd16678

See more details on using hashes here.

Provenance

The following attestation bundles were made for arrowspace-0.26.4-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: python-publish.yml on tuned-org-uk/pyarrowspace

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file arrowspace-0.26.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for arrowspace-0.26.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 763dfb181059f834ef7cd5241fc52f222b39f55d4fa04cf20b9f7b50fad4ab5a
MD5 24f9fce300f4cf4e74db36f87b099e67
BLAKE2b-256 15f544d7a4229cfc7deccb644fe6199e32f0b467ebb99ed83285da0775cc9249

See more details on using hashes here.

Provenance

The following attestation bundles were made for arrowspace-0.26.4-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: python-publish.yml on tuned-org-uk/pyarrowspace

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file arrowspace-0.26.4-cp313-cp313-macosx_11_0_arm64.whl.

File metadata

  • Download URL: arrowspace-0.26.4-cp313-cp313-macosx_11_0_arm64.whl
  • Upload date:
  • Size: 4.9 MB
  • Tags: CPython 3.13, macOS 11.0+ ARM64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.12.1 {"installer":{"name":"uv","version":"0.12.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for arrowspace-0.26.4-cp313-cp313-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 40a067ab430641b859d315734442c53719ff3c374f75141eae1d9f186c3d38ee
MD5 d5232632ff0292a8506c88df4082d246
BLAKE2b-256 1e7dd5a94555fdcb3738cd11c6232fe404633456ffd85a0d5c943eac144b6f4d

See more details on using hashes here.

File details

Details for the file arrowspace-0.26.4-cp313-cp313-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for arrowspace-0.26.4-cp313-cp313-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 7820893123c64ed15d7068433ddf90a7bcbc1ecc92914a1432d08d7c22f84886
MD5 c70d9a77fc28968d79ce65219305f95d
BLAKE2b-256 b10c077f258e57f0c5c69a3cc20cdb708e8eec1d99f06f374f776e71520f70c1

See more details on using hashes here.

Provenance

The following attestation bundles were made for arrowspace-0.26.4-cp313-cp313-macosx_10_12_x86_64.whl:

Publisher: python-publish.yml on tuned-org-uk/pyarrowspace

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file arrowspace-0.26.4-cp312-cp312-win_amd64.whl.

File metadata

File hashes

Hashes for arrowspace-0.26.4-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 afd50f4776dc69795b249d6aaf27407255712564334766fc2c5122475828f2e2
MD5 f81a94d71cdefb4e5566e81ba2f9dc43
BLAKE2b-256 bba2ebefcaed4cba1c13dc98ba8584b453a8d9c5f5891e4db982157e22d764db

See more details on using hashes here.

Provenance

The following attestation bundles were made for arrowspace-0.26.4-cp312-cp312-win_amd64.whl:

Publisher: python-publish.yml on tuned-org-uk/pyarrowspace

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file arrowspace-0.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for arrowspace-0.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 418fdd5c21475ad55560e2d4ec91d495f468af9bce1774f4a5cd850a0ff334de
MD5 1f1ad2e206ccfb38b40c47fbe1fd3927
BLAKE2b-256 84e3d7abca36fc104c5f72aeb6ae8b45cc56fe49c98986f6155cd48de2520afb

See more details on using hashes here.

Provenance

The following attestation bundles were made for arrowspace-0.26.4-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: python-publish.yml on tuned-org-uk/pyarrowspace

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file arrowspace-0.26.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for arrowspace-0.26.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 758d030f1a5d9b9e3cb4a140e4b70bb7f1477d9a1cf348c7af107b83375f620c
MD5 8ccccac04d26df579b8fd74fca441279
BLAKE2b-256 eaaf9e7d65a69785ecd40604b22907b22aaacfbcc88bcfb83140b37d454b9959

See more details on using hashes here.

Provenance

The following attestation bundles were made for arrowspace-0.26.4-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: python-publish.yml on tuned-org-uk/pyarrowspace

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file arrowspace-0.26.4-cp312-cp312-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for arrowspace-0.26.4-cp312-cp312-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 5aa279af99c0d4f1c67091a1140b2f3bda1c9796c9ca7abdd73cf00c0dc23a32
MD5 9a96ebd967d42175da9fdb9fc199321d
BLAKE2b-256 bc08b1455183160ac43d5a5c3354512131bfe60042ff4b88017af6dbb53dceaf

See more details on using hashes here.

Provenance

The following attestation bundles were made for arrowspace-0.26.4-cp312-cp312-macosx_11_0_arm64.whl:

Publisher: python-publish.yml on tuned-org-uk/pyarrowspace

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file arrowspace-0.26.4-cp312-cp312-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for arrowspace-0.26.4-cp312-cp312-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 d77d70f69a3af80140ef6b2eece8f6b7188ca335c4b4a0c569f3f971f2c241a0
MD5 9adf8fd2b9a262198323f59c2464ecb0
BLAKE2b-256 bb4e547f833139413ce61b5341aab80f2c24d25c06548833d09a004e2aa0f4d2

See more details on using hashes here.

Provenance

The following attestation bundles were made for arrowspace-0.26.4-cp312-cp312-macosx_10_12_x86_64.whl:

Publisher: python-publish.yml on tuned-org-uk/pyarrowspace

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page