probabilistic-rs

Probabilistic data structures in Rust with Python bindings and HTTP API.

Features

• Bloom Filter — fast membership testing with bulk ops and optional Fjall persistence
• Expiring Bloom Filter — auto-expires elements via sliding time windows
• HTTP API — REST server with Swagger UI, managing multiple named filters
• Python Bindings — native wheels via PyO3/maturin
• CLI + TUI — interactive terminal interface

Installation

Rust:

cargo add probabilistic-rs

Python:

pip install probabilistic-rs

Quick Start

Rust — Bloom Filter

use probabilistic_rs::bloom::{BloomFilter, BloomFilterConfigBuilder, BloomFilterOps};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = BloomFilterConfigBuilder::default()
        .capacity(10000)
        .false_positive_rate(0.01)
        .build()?;

    let filter = BloomFilter::create(config).await?;
    filter.insert(b"item1")?;
    assert!(filter.contains(b"item1")?);
    Ok(())
}

Rust — Expiring Bloom Filter

use probabilistic_rs::ebloom::{
    ExpiringBloomFilter, ExpiringFilterConfigBuilder, ExpiringBloomFilterOps,
};
use std::time::Duration;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = ExpiringFilterConfigBuilder::default()
        .capacity(1000)
        .false_positive_rate(0.01)
        .level_duration(Duration::from_secs(60))
        .max_levels(3)
        .build()?;

    let mut filter = ExpiringBloomFilter::new(config)?;
    filter.insert(b"test_item")?;
    assert!(filter.query(b"test_item")?);
    Ok(())
}

Python

from probabilistic_rs import BloomFilter, ExpiringBloomFilter

bf = BloomFilter(capacity=10000, false_positive_rate=0.01)
bf.insert(b"item1")
assert bf.contains(b"item1")

ebf = ExpiringBloomFilter(capacity=1000, false_positive_rate=0.01, ttl_seconds=60)
ebf.insert(b"temp_item")
assert ebf.query(b"temp_item")

HTTP API

# Start server (default: localhost:3000)
probabilistic-server

# Create a filter, insert, query
curl -X POST http://localhost:3000/api/v1/bloom/create \
  -H "Content-Type: application/json" \
  -d '{"name":"my-filter","capacity":10000,"false_positive_rate":0.01}'

curl -X POST http://localhost:3000/api/v1/bloom/insert \
  -H "Content-Type: application/json" \
  -d '{"name":"my-filter","item":"hello"}'

curl -X POST http://localhost:3000/api/v1/bloom/contains \
  -H "Content-Type: application/json" \
  -d '{"name":"my-filter","item":"hello"}'

Endpoints: create, delete, insert, contains, bulk_insert, bulk_contains, clear, stats, list — available for both /api/v1/bloom and /api/v1/ebloom. Swagger UI at /swagger-ui.

CLI

# Create filter
expblf create --db-path myfilter.fjall --capacity 10000 --fpr 0.01

# Operations
expblf load --db-path myfilter.fjall insert --element "key"
expblf load --db-path myfilter.fjall check --element "key"

# Interactive TUI
expblf tui --db-path myfilter.fjall

Benchmarks

Measured on Apple M-series via cargo bench (criterion, 100 samples). Times are total for N operations.

Bloom Filter (in-memory)

Expiring Bloom Filter (in-memory, 3 levels — 5 levels nearly identical)

Both filters sustain ~15–17M ops/s (60–65 ns/op) across all dataset sizes. The expiring filter adds ~5–10% overhead over plain bloom due to multi-level bookkeeping. Bulk operations match or slightly outperform single-item ops. Level rotation (TTL expiry) takes ~250 µs regardless of filter size.

Configuration

License

MIT