BMP - Block-Max Pruning
Faster Learned Sparse Retrieval with Block-Max Pruning
Installation
From PyPI:
pip install bmp
From source:
pip install maturin
maturin build -r
pip install target/wheels/*.whl
Usage
Index from CIFF
from bmp import ciff2bmp
ciff2bmp(ciff_file="/path/to/ciff", output="/path/to/index", bsize=32, compress_range=False)
Index with Python
from bmp import Indexer
indexer = Indexer('/path/to/index', bsize=32, compress_range=False)
indexer.add_document('doc1', {'a': 1, 'b': 5, 'c': 8}) # docid, vector
indexer.add_document('doc2', {'a': 2, 'c': 1, 'd': 8, 'f': 2})
# ... add more documents
indexer.finish()
Search
from bmp import search, Searcher
# Batch operation
results = search(index="/path/to/index", queries="/path/to/queries", k=10, alpha=1.0, beta=1.0)
# Returns: str (TREC run file)
# Query-by-query operation
searcher = Searcher("/path/to/index") # loads index into memory once
searcher.search({'tok1': 5.3, 'tok2': 1.1}, k=10, alpha=1.0, beta=1.0)
# Returns: Tuple[List[str], List[float]] (doc IDs, scores) for this query
Citation
If you use this code, please cite:
@inproceedings{BMP,
author = {Antonio Mallia and Torsten Suel and Nicola Tonellotto},
title = {Faster Learned Sparse Retrieval with Block-Max Pruning},
booktitle = {The 47th International ACM SIGIR Conference on Research and Development in Information Retrieval ({SIGIR})},
publisher = {ACM},
year = {2024}
}
License
Apache-2.0
Release files for bmp 0.2.6
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| bmp-0.2.6-cp312-cp312-win_amd64.whl | CPython 3.12 | CPython 3.12 | Windows x86-64 | Details |
| bmp-0.2.6-cp312-cp312-manylinux_2_34_x86_64.whl | CPython 3.12 | CPython 3.12 | Linux glibc 2.34+ x86-64 | Details |
| bmp-0.2.6-cp312-cp312-macosx_10_12_x86_64.whl | CPython 3.12 | CPython 3.12 | macOS 10.12+ x86-64 | Details |
| bmp-0.2.6-cp311-cp311-win_amd64.whl | CPython 3.11 | CPython 3.11 | Windows x86-64 | Details |
| bmp-0.2.6-cp311-cp311-manylinux_2_34_x86_64.whl | CPython 3.11 | CPython 3.11 | Linux glibc 2.34+ x86-64 | Details |
| bmp-0.2.6-cp311-cp311-macosx_10_12_x86_64.whl | CPython 3.11 | CPython 3.11 | macOS 10.12+ x86-64 | Details |
| bmp-0.2.6-cp310-cp310-win_amd64.whl | CPython 3.10 | CPython 3.10 | Windows x86-64 | Details |
| bmp-0.2.6-cp310-cp310-manylinux_2_34_x86_64.whl | CPython 3.10 | CPython 3.10 | Linux glibc 2.34+ x86-64 | Details |
| bmp-0.2.6-cp310-cp310-macosx_10_12_x86_64.whl | CPython 3.10 | CPython 3.10 | macOS 10.12+ x86-64 | Details |
| bmp-0.2.6-cp39-cp39-win_amd64.whl | CPython 3.9 | CPython 3.9 | Windows x86-64 | Details |
| bmp-0.2.6-cp39-cp39-manylinux_2_34_x86_64.whl | CPython 3.9 | CPython 3.9 | Linux glibc 2.34+ x86-64 | Details |
| bmp-0.2.6-cp39-cp39-macosx_10_12_x86_64.whl | CPython 3.9 | CPython 3.9 | macOS 10.12+ x86-64 | Details |
| bmp-0.2.6-cp38-cp38-win_amd64.whl | CPython 3.8 | CPython 3.8 | Windows x86-64 | Details |
| bmp-0.2.6-cp38-cp38-manylinux_2_34_x86_64.whl | CPython 3.8 | CPython 3.8 | Linux glibc 2.34+ x86-64 | Details |
| bmp-0.2.6-cp38-cp38-macosx_10_12_x86_64.whl | CPython 3.8 | CPython 3.8 | macOS 10.12+ x86-64 | Details |
| bmp-0.2.6-cp37-cp37m-win_amd64.whl | CPython 3.7 | CPython 3.7 pymalloc | Windows x86-64 | Details |
Total release size: 11.9 MB
Release files / bmp-0.2.6-cp312-cp312-win_amd64.whl
| Download URL | bmp-0.2.6-cp312-cp312-win_amd64.whl |
|---|---|
| Size | 636.2 kB |
| Tags | CPython 3.12 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
05de2f3ef0f0ff6738aca3a99aaa2a7e8b892de069bb55a09e679a796d306de6
|
|
BLAKE2b-256 checksum How to use checksums |
3d1081b20c4b02f23fed3f79d929b70287a6862158c32e3e17cd39b534aabab4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp312-cp312-manylinux_2_34_x86_64.whl
| Download URL | bmp-0.2.6-cp312-cp312-manylinux_2_34_x86_64.whl |
|---|---|
| Size | 867.7 kB |
| Tags | CPython 3.12 Linux glibc 2.34+ x86-64 |
|
SHA-256 checksum How to use checksums |
ca823e1af6950abf9053375ae353398544af5aae1a99c881e3cd3327ad6d06a8
|
|
BLAKE2b-256 checksum How to use checksums |
43d8426d1bb5c44b6edeb9e90edf07797ab58831d9d9effa274a1dc43aaf2ae2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp312-cp312-macosx_10_12_x86_64.whl
| Download URL | bmp-0.2.6-cp312-cp312-macosx_10_12_x86_64.whl |
|---|---|
| Size | 745.0 kB |
| Tags | CPython 3.12 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
764fea5e8180558e44dcc14819a46a1715844b3cd6c735576dbcc24fd6ecfce4
|
|
BLAKE2b-256 checksum How to use checksums |
a4ca7ef3cb9b9ac8aa74da188566923d11a3d9de417a6d1814ee36c1be40ea8d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp311-cp311-win_amd64.whl
| Download URL | bmp-0.2.6-cp311-cp311-win_amd64.whl |
|---|---|
| Size | 633.7 kB |
| Tags | CPython 3.11 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
be03d1b8f286c11fd3fc77b889c07ac2a30e9c1b7537e4c682937878d51d61ce
|
|
BLAKE2b-256 checksum How to use checksums |
798963882633bfcdd7ca5aec6ff7619e2a60d37cdea26e7b1d721d224e92defa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp311-cp311-manylinux_2_34_x86_64.whl
| Download URL | bmp-0.2.6-cp311-cp311-manylinux_2_34_x86_64.whl |
|---|---|
| Size | 870.9 kB |
| Tags | CPython 3.11 Linux glibc 2.34+ x86-64 |
|
SHA-256 checksum How to use checksums |
807199fd625032b121587f70356a45f6d06921af984506b0e174ea7369788ec4
|
|
BLAKE2b-256 checksum How to use checksums |
bd48ef0f4a0136299f31fdc074d96ebf2852b24dfcf8fa071733042b8c20fd20
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp311-cp311-macosx_10_12_x86_64.whl
| Download URL | bmp-0.2.6-cp311-cp311-macosx_10_12_x86_64.whl |
|---|---|
| Size | 745.4 kB |
| Tags | CPython 3.11 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
b75d43d55a435cbf82be18749505e2ee605d63313dd2ae7b6da5dd480b53b15c
|
|
BLAKE2b-256 checksum How to use checksums |
cab335c87c22f7d3ad2a1525f89ee6cb909439183cd9d967f8d2ca918de2cf57
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp310-cp310-win_amd64.whl
| Download URL | bmp-0.2.6-cp310-cp310-win_amd64.whl |
|---|---|
| Size | 636.5 kB |
| Tags | CPython 3.10 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
29ec36b8b739a418a975747b92280e9c39489efc0732a00b55d0fb2e116a7453
|
|
BLAKE2b-256 checksum How to use checksums |
89c2e221328eeeaccf85940dce4b36560c855a95ee487af100eacb6f44b494df
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp310-cp310-manylinux_2_34_x86_64.whl
| Download URL | bmp-0.2.6-cp310-cp310-manylinux_2_34_x86_64.whl |
|---|---|
| Size | 858.5 kB |
| Tags | CPython 3.10 Linux glibc 2.34+ x86-64 |
|
SHA-256 checksum How to use checksums |
766dc2527c47e6aa0fba3e703090d0d415826b220aafb067e3ef71d11360ffa9
|
|
BLAKE2b-256 checksum How to use checksums |
32e79a82ea1e571b07213474a81c947e44b16f2f78063ae01b8f996e180c59af
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp310-cp310-macosx_10_12_x86_64.whl
| Download URL | bmp-0.2.6-cp310-cp310-macosx_10_12_x86_64.whl |
|---|---|
| Size | 745.5 kB |
| Tags | CPython 3.10 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
67e769392bce530ac1a5895d02e6ed9f6c8cc750c58b2b72d56df44b583f7cc9
|
|
BLAKE2b-256 checksum How to use checksums |
21b7e21d17439446f4caf9ddd5cfc3a3ac8825ee67e66591582c47b8e0039c14
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp39-cp39-win_amd64.whl
| Download URL | bmp-0.2.6-cp39-cp39-win_amd64.whl |
|---|---|
| Size | 635.9 kB |
| Tags | CPython 3.9 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
8639d1ce6ec6738b9cfdb0a63da5bfac49ef626549419f5272ba55daf711c48e
|
|
BLAKE2b-256 checksum How to use checksums |
e185f94a67b240009cb5bf49fae2dc064299d82b0762782a814a3e50ffbe8e86
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp39-cp39-manylinux_2_34_x86_64.whl
| Download URL | bmp-0.2.6-cp39-cp39-manylinux_2_34_x86_64.whl |
|---|---|
| Size | 858.8 kB |
| Tags | CPython 3.9 Linux glibc 2.34+ x86-64 |
|
SHA-256 checksum How to use checksums |
350414aa64370eb92dc356ced73ca0d076224bc5b4d5c8f29e58300ac5aaa753
|
|
BLAKE2b-256 checksum How to use checksums |
a3f5b1995f2ea7af9804cdca53843f0d918bb06ebc644e7208069e4d2f68c59a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp39-cp39-macosx_10_12_x86_64.whl
| Download URL | bmp-0.2.6-cp39-cp39-macosx_10_12_x86_64.whl |
|---|---|
| Size | 745.8 kB |
| Tags | CPython 3.9 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
1dfdc2835055e418034690941cccfabb0fbb2ec7b34babfea4db5604e0b4b42b
|
|
BLAKE2b-256 checksum How to use checksums |
435d0a2ae73e3b09e589a9b52c9d85e6a14a88105f2d2f1904f3c1a6692e2da1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp38-cp38-win_amd64.whl
| Download URL | bmp-0.2.6-cp38-cp38-win_amd64.whl |
|---|---|
| Size | 636.2 kB |
| Tags | CPython 3.8 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
e2e1eb02f94e996cfea97b2c9b28fefde0898a086ebc9776fe0f4e5d2451636b
|
|
BLAKE2b-256 checksum How to use checksums |
0b2ab9ce795ab71eb3f12f51fd4dff6a5cb1adfb616854eb250843ba50963f64
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp38-cp38-manylinux_2_34_x86_64.whl
| Download URL | bmp-0.2.6-cp38-cp38-manylinux_2_34_x86_64.whl |
|---|---|
| Size | 865.0 kB |
| Tags | CPython 3.8 Linux glibc 2.34+ x86-64 |
|
SHA-256 checksum How to use checksums |
1d5028b0f2f23a9a9e865222b8bee17013dda87054ca3c18ecff6157127d3136
|
|
BLAKE2b-256 checksum How to use checksums |
adc4643d8a3ec7bae0c31f48ca3d0a628d56b180da0809fad07bd46fc159f24b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp38-cp38-macosx_10_12_x86_64.whl
| Download URL | bmp-0.2.6-cp38-cp38-macosx_10_12_x86_64.whl |
|---|---|
| Size | 745.3 kB |
| Tags | CPython 3.8 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
8f9f56223e41dd8a62437c9c202df1cb9948968e2c67a80f494b21b58834f2ee
|
|
BLAKE2b-256 checksum How to use checksums |
ee858013305807c746d67294500a261d74eba73f973e917d309ff4c47c4592c0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
Release files / bmp-0.2.6-cp37-cp37m-win_amd64.whl
| Download URL | bmp-0.2.6-cp37-cp37m-win_amd64.whl |
|---|---|
| Size | 636.2 kB |
| Tags | CPython 3.7 CPython 3.7 pymalloc Windows x86-64 |
|
SHA-256 checksum How to use checksums |
a1b9bef98563f4b21a346e42c3586eeec04849222fc1b5acba8386127a6638e0
|
|
BLAKE2b-256 checksum How to use checksums |
593d498fa557895abe0935529f431a2a2915b8ff9327893e5d479432cd544e02
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
maturin/1.10.2
|
