pyarrowspace
Python bindings for arrowspace-rs.
arrowspace is a graph-based analytics library for vector spaces, supported by a graph representation and a key-value store. The main use-cases targeted are: AI search capabilities as advanced vector similarity, graph characterisation analysis and search, indexing of high-dimensional vectors. Design principles described in this article.
For labs and tests please see tests/
Installation
Core library
The core library ships as a compiled Rust extension. It only requires numpy, pyarrow, pandas, and scikit-learn:
pip install arrowspace
Optional extras
Additional capabilities are available as optional extras:
| Extra | Packages included | Use case |
|---|---|---|
embeddings |
datasets, sentence-transformers, transformers[torch]<5.0, tsdae |
Generating embeddings from text via HuggingFace models; required for test_1_quora_questions.py and all tests above |
benchmarks |
beir, nltk |
Running BEIR/MS-MARCO benchmark suites and NLP preprocessing |
viz |
matplotlib, seaborn, tqdm |
Plotting results and progress bars in test scripts |
full |
all of the above | Full development and research environment |
Install with one or more extras:
# Required to run test_1_quora_questions.py and above
pip install arrowspace[embeddings]
# Benchmarking + visualisation
pip install arrowspace[benchmarks,viz]
# Everything (for running the full test suite)
pip install arrowspace[full]
Note:
tests/test_0_*.pyonly require the core install. All tests numberedtest_1and above require at minimumarrowspace[embeddings].
Build from source
If you have Cargo installed, you can compile and install locally using maturin:
pip install maturin[patchelf]
# quick development build
maturin develop
# optimised release build (recommended for large datasets)
maturin develop --release
Tests
test_0_*.py scripts only require the core install:
python tests/test_0_0.py
test_1 and above require the embeddings extra (pip install arrowspace[embeddings]):
python tests/test_1_quora_questions.py
Higher-numbered tests (test_3 and above) additionally require benchmarks and viz:
pip install arrowspace[full]
python tests/test_3_beir.py
Some tests require downloading a dataset separately or fine-tuning embeddings on a given dataset.
Simplest Example
from arrowspace import ArrowSpaceBuilder
import numpy as np
items: np.array = np.array(
[[0.1, 0.2, 0.3], [0.0, 0.5, 0.1], [0.9, 0.1, 0.0]],
dtype = np.float64
)
graph_params: dict = {
"eps": 1.0,
"k": 6,
"topk": 3,
"p": 2.0,
"sigma": 1.0,
}
# Create an ArrowSpace instance, returning the computed
# signal graph and lambdas
aspace, gl = ArrowSpaceBuilder().build(graph_params, items)
# Search comparable items
# defaults: k = nitems, alpha = 0.9, beta = 0.1
query: np.array = np.array(
[0.05, 0.2, 0.25],
dtype = np.float64
)
tau: float = 1.0
hits: list = aspace.search(query, gl, tau)
# Search returns a list of `(index, score`) tuples, where
# expected value from the code above show the first index
# having the top score, i.e., being nearest.
print(hits)
# [ (0, 0.989743318610787), (1, 0.7565344158360029), (2, 0.22151940739207396) ]
2 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file arrowspace-0.28.0.tar.gz.
File metadata
- Download URL: arrowspace-0.28.0.tar.gz
- Upload date:
- Size: 18.7 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.12.9 {"installer":{"name":"uv","version":"0.12.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
502db4b25e1ce25340cf97a0d9f4c8f8f379b7b46ab659a51434fd996e20234e
|
|
| MD5 |
5b8476a0adc65540347ffa8d3e66d8b8
|
|
| BLAKE2b-256 |
106634b87671e161b7a810af785bac7e9669b17af05668f8b229e583c9c6682c
|
File details
Details for the file arrowspace-0.28.0-cp314-cp314-macosx_11_0_arm64.whl.
File metadata
- Download URL: arrowspace-0.28.0-cp314-cp314-macosx_11_0_arm64.whl
- Upload date:
- Size: 3.3 MB
- Tags: CPython 3.14, macOS 11.0+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.12.9 {"installer":{"name":"uv","version":"0.12.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2964326d8e0f2c9dc29c9dd390acfb5c3aa6750f51c9939d22851ffa30d227af
|
|
| MD5 |
93cb4e9cd0894fbe4549864e132f8c4f
|
|
| BLAKE2b-256 |
088b1aaaecb73d31e67654898cb7f94d46ea516f6bd2bf98abb5b7038f76b6bb
|
File details
Details for the file arrowspace-0.28.0-cp313-cp313-macosx_11_0_arm64.whl.
File metadata
- Download URL: arrowspace-0.28.0-cp313-cp313-macosx_11_0_arm64.whl
- Upload date:
- Size: 3.3 MB
- Tags: CPython 3.13, macOS 11.0+ ARM64
- Uploaded using Trusted Publishing? No
- Uploaded via:
uv/0.12.9 {"installer":{"name":"uv","version":"0.12.9","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1590fdfcb6928f220856061f4516375a25866fda4cecb8cbd1375c610db06d63
|
|
| MD5 |
9cd8c2033f3facaa3dda7074f893b8e2
|
|
| BLAKE2b-256 |
fd68ba2f9c844860a3e1e5fe766b1782146489c7e5f8f6311fd7eaedb9c5c576
|
