Skip to main content

Lustro API

Lustro is a 256-bit, deterministic, single-pass diffusion engine with three uses: a hash, a PRNG and an XOF. A stream can be forked into child streams, copied, and saved with its exact position. Forking does not change the parent. A batch advances many independent streams in one call and uses AVX2 and a thread pool when the CPU and the settings allow it. It is available as a Rust crate, a Python module (with NumPy) and a C/C++ library.

Hash PRNG XOF
Input a message a 32-byte seed and a 128-bit stream ID a message of any length
Output a 16 or 32-byte digest a byte stream a byte stream
One at a time hash128, hash256, LustroHasher LustroPrng LustroXof, LustroXofAbsorber
Many at once hash*_many_* LustroPrngBatch LustroXofBatch

Questions? Please see the FAQ.

The core mechanism and the API implementation are currently considered non-cryptographic.

Build · Hashing · Streams · Batches · State and Runtime · Language Notes · Reference

Get the Code

git clone https://github.com/Ligatum/Lustro_API.git
cd Lustro_API

Build

# Rust
cargo build --release

# Rust with the rand_core traits (also enables entropy)
cargo build --release --features rand

# Python 3.11-3.14
pip install maturin numpy
maturin develop --release

# C/C++ FFI bindings
cargo build --release --features ffi
Feature Default Adds
parallel yes thread pool for batches
entropy no from_entropy constructors (system entropy source)
rand no rand_core traits for LustroPrng and LustroXof; enables entropy
ffi no C/C++ bindings and lustro.h; enables entropy
python no Python module, built by maturin; enables entropy

The ffi build generates lustro.h in Cargo's output directory. To also write it into the source tree, set LUSTRO_HEADER_DIR, for example LUSTRO_HEADER_DIR=include cargo build --release --features ffi.

In the release profiles (panic = "unwind") the C FFI boundary catches internal panics and returns LUSTRO_ERROR_INTERNAL_PANIC; the Python module raises them as pyo3_runtime.PanicException (a BaseException). Direct Rust calls panic as usual, for example on a length mismatch. LUSTRO_ERROR_INTERNAL_PANIC is not transactional: after it the state of the context and of the output buffer is unspecified, so do not retry on the same context.

NumPy is needed only by the functions that take or return arrays (hash*_many*, fill_blocks, blocks, fill_u64); without it they raise ImportError. The install command above includes NumPy; a package built from the repo also has it as the numpy extra.

The Python module declares that it requires the GIL (gil_used = true). On free-threaded CPython builds, importing the module enables the GIL unless this behavior is explicitly overridden (PYTHON_GIL=0 or -X gil=0). Running Lustro with the GIL disabled is not supported.


Speed Test

The repo also contains Lustro_Speed_Tester.py. It runs equivalent sections on native Rust, Python and C layers.

# rust target (built on request only, no other build includes it)
cargo build --release --example speed

Tested Platforms

Tested platforms: Windows x86_64 (Python 3.11-3.14) and Linux x86_64 (Python 3.11-3.14). 32-bit: the Rust and C layers were tested on i686-pc-windows-msvc, the Python layer only compiles there. Not tested: macOS, aarch64 and other ARM targets, big-endian targets and free-threaded CPython.


C / C++ Conventions

The C / C++ examples in this README mostly omit status checks for brevity. In real code, check every returned LustroError (lustro_strerror returns a message).

  • Constructors return a LustroError and store the new context in their last argument (out_ctx). *out_ctx is NULL after a failure and receives the context only when the status is Ok; the status tells why. A NULL or misaligned out_ctx is rejected with LUSTRO_ERROR_INVALID_POINTER and nothing is written. The same status is returned, with nothing written, by lustro_hasher_clone, lustro_xof_absorber_clone, lustro_xof_absorber_finalize and the PRNG/XOF constructors that take a source context when out_ctx overlaps that context (details in ARCHITECTURE.md). Set the pointer to NULL yourself and check the status.
  • Freeing: the lustro_*_free functions accept NULL.
  • 128-bit values: a stream ID or a path element goes in as two uint64_t, hi first. next_u128 and position return 16 bytes, little-endian. lustro_u128_from_parts(hi, lo, out) and lustro_u128_to_parts(in, &hi, &lo) from lustro.h convert between the two forms.

Hashing

Hash

Rust

use lustro::hash256;

let digest = hash256(b"hello world");

Python

import lustro

digest = lustro.hash256(b"hello world")

C / C++

#include "lustro.h"

uint8_t digest[LUSTRO_HASH256_LEN];
lustro_hash256((const uint8_t *)"hello world", 11, digest);

hash128 is available in the same way and returns LUSTRO_HASH128_LEN (16) bytes.

Printing and comparing digests (Rust)

Hash128 and Hash256 print as hex of the digest bytes, in the order returned by as_bytes(): 32 and 64 digits. Width, fill and alignment work as for strings ({digest:>70}, {digest:*^70}), and a precision shortens the output ({digest:.8}).

use lustro::{hash256, Hash256};

let digest = hash256(b"hello world");

println!("{digest}");     // lowercase hex, 64 digits
println!("{digest:X}");   // uppercase
println!("{digest:#x}");  // with a 0x prefix

let parsed: Hash256 = format!("{digest:#x}").parse()?;
assert_eq!(parsed, digest);

Digests are Hash and Ord, so they work as keys in HashMap and BTreeMap. Ord compares the bytes lexicographically, not as a number. Conversion to and from [u8; N] is available through From / Into. In Python a digest is bytes, so use digest.hex().

parse() reads a digest back from hex: 64 digits for Hash256, 32 for Hash128, either case, with an optional 0x or 0X prefix. Spaces are not trimmed. ParseHashError says whether the length or a digit is wrong. Hash256::first_128() returns the first 16 bytes as a Hash128, the same value hash128 returns for that message. In Python use bytes.fromhex(s).

Incremental Hash

A message that is not in memory at once, such as a large file, can be hashed in pieces. The length of the whole message goes into the initial state, so it has to be known before the first byte. Pass it to the hasher; update then takes the bytes in pieces of any size, and the digest equals hash256 of the whole message.

Rust

use lustro::{hash256, hash256_reader, LustroHasher};
use std::fs::File;

let mut h = LustroHasher::new(11);
h.update(b"hello ")?;
h.update(b"world")?;
assert_eq!(h.finalize()?, hash256(b"hello world"));

// A file, read in 8 KiB pieces.
let file = File::open("data.bin")?;
let len = file.metadata()?.len();
let digest = hash256_reader(len, file)?;

Python

import os
import lustro

h = lustro.LustroHasher(11)
h.update(b"hello ")
h.update(b"world")
assert h.digest() == lustro.hash256(b"hello world")

h = lustro.LustroHasher(os.path.getsize("data.bin"))
with open("data.bin", "rb") as f:
    while chunk := f.read(1 << 16):
        h.update(chunk)
digest = h.digest()

C / C++

#include "lustro.h"

LustroHasher *h = NULL;
lustro_hasher_new(11, &h);
lustro_hasher_update(h, (const uint8_t *)"hello ", 6);
lustro_hasher_update(h, (const uint8_t *)"world", 5);

uint8_t digest[LUSTRO_HASH256_LEN];
lustro_hasher_finalize256(h, digest);
lustro_hasher_free(h);
  • Length: a piece that would go past the declared length is rejected whole (HashError::TooMuchData in Rust, ValueError in Python, LUSTRO_ERROR_INVALID_LENGTH in C/C++) and nothing of it is absorbed. Asking for the digest early fails the same way (HashError::TooLittleData), and more pieces can follow. remaining() (lustro_hasher_remaining) tells how many bytes are still expected.
  • Digest: reading it does not end the hasher; every call gives the same value. Its first 16 bytes are hash128 of the message.
  • Reader: hash256_reader(len, reader) reads exactly len bytes and leaves the rest in the reader (pass &mut reader to keep it). A reader that ends early gives ErrorKind::UnexpectedEof.
  • Copy: clone() in Rust, copy() in Python and lustro_hasher_clone in C/C++ continue independently, for example to hash several messages of the same length that share a prefix.

Batch Hash

Hash many messages in one call. *_many_fixed takes messages of one length stored back to back in a single buffer; *_many_var takes messages of any length.

Rust

use lustro::{
    hash256_many_fixed, hash256_many_fixed_into, hash256_many_var, hash256_many_var_into, Hash256,
};

// Messages of one length, stored back to back.
let data = [0u8; 3 * 16];
let fixed = hash256_many_fixed(&data, 16);

// The same, written into a slice you provide.
let mut out = vec![Hash256::default(); 3];
hash256_many_fixed_into(&data, 16, &mut out);

// Messages of any length.
let messages = [b"a".as_ref(), b"bc".as_ref(), b"".as_ref()];
let digests = hash256_many_var(&messages);

// The same, written into a slice you provide.
hash256_many_var_into(&messages, &mut out);

Python

import numpy as np
import lustro

# Equal-length messages: one message per row of a 2D uint8 array.
rows = np.zeros((3, 16), dtype=np.uint8)
digests = lustro.hash256_many_fixed(rows)  # uint8 array, shape (3, 32)

# Messages of any length: a sequence of bytes or byte buffers.
digests = lustro.hash256_many_var([b"a", b"bc", b""])  # shape (3, 32)

# Into an array you provide, see "GIL and Throughput (Python)".
out = np.empty((3, 32), dtype=np.uint8)
lustro.hash256_many_fixed_into(rows, out)
lustro.hash256_many_var_into([b"a", b"bc", b""], out)

# Messages packed in one buffer: message i is data[offsets[i]:offsets[i + 1]].
data = b"abc"
offsets = np.array([0, 1, 3, 3], dtype=np.int64)  # three messages: b"a", b"bc", b""
digests = lustro.hash256_many_offsets(data, offsets)  # shape (3, 32)
lustro.hash256_many_offsets_into(data, offsets, out)

C / C++

#include "lustro.h"

/* Equal-length messages, stored back to back. */
uint8_t rows[3][16] = {{0}};
uint8_t digests[3][LUSTRO_HASH256_LEN];
lustro_hash256_many_fixed((const uint8_t *)rows, 3, 16, (uint8_t *)digests);

/* Messages of any length. */
const uint8_t *messages[] = {
    (const uint8_t *)"a",
    (const uint8_t *)"bc",
    (const uint8_t *)""
};
uintptr_t lengths[] = {1, 2, 0};

lustro_hash256_many_var(messages, lengths, 3, (uint8_t *)digests);

The names mean the same in Rust, Python and C/C++ for *_many_fixed and *_many_var; *_many_offsets exists in Python only. hash128_* and lustro_hash128_* work the same way and return 16-byte digests.

AVX2 and the thread pool are used as far as the CPU and the runtime settings allow; the digests do not depend on either.


Streams

PRNG and XOF

LustroPrng generates a stream from a 32-byte seed and a stream ID. LustroXof generates a stream from a message of any length.

Rust

// PRNG
use lustro::{LustroPrng, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let mut rng = LustroPrng::new(&seed, StreamId(0));

let value = rng.next_u64();
// XOF
use lustro::LustroXof;

let mut xof = LustroXof::new(b"hello world");

let value = xof.next_u64();

StreamId wraps a u128, so StreamId(5) works. StreamId::from takes u8, u16, u32, u64 and u128 values; StreamId::from(5) does not compile, because an untyped literal is an i32.

Python

# PRNG
from lustro import LustroPrng

rng = LustroPrng(bytes(32), 0)  # stream_id defaults to 0
value = rng.next_u64()
# XOF
from lustro import LustroXof

xof = LustroXof(b"hello world")
value = xof.next_u64()

C / C++

/* PRNG */
#include <stdio.h>
#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};

LustroPrng *rng = NULL;
LustroError err = lustro_prng_new(seed, 0, 0, &rng);
if (err != LUSTRO_ERROR_OK) {
    fprintf(stderr, "%s\n", lustro_strerror(err));
    return 1;
}

uint64_t value;
lustro_prng_next_u64(rng, &value);

lustro_prng_free(rng);
/* XOF */
#include "lustro.h"

LustroXof *xof = NULL;
lustro_xof_new((const uint8_t *)"hello world", 11, &xof);

uint64_t value;
lustro_xof_next_u64(xof, &value);

lustro_xof_free(xof);

Incremental XOF

LustroXofAbsorber is the XOF counterpart of the hasher from Incremental Hash: give it the total length, feed the message with update, and finalize returns the LustroXof of the whole message. LustroXof::from_reader does the same from a reader, like hash256_reader.

Rust

use lustro::{LustroXof, LustroXofAbsorber};
use std::fs::File;

let mut a = LustroXofAbsorber::new(11);
a.update(b"hello ")?;
a.update(b"world")?;
let mut xof = a.finalize()?;
assert_eq!(xof.next_u64(), LustroXof::new(b"hello world").next_u64());

// A file, read in 8 KiB pieces.
let file = File::open("data.bin")?;
let len = file.metadata()?.len();
let mut xof = LustroXof::from_reader(len, file)?;

Python

import lustro

a = lustro.LustroXofAbsorber(11)
a.update(b"hello ")
a.update(b"world")
xof = a.finalize()
assert xof.fill(64) == lustro.LustroXof(b"hello world").fill(64)

C / C++

#include "lustro.h"

LustroXofAbsorber *a = NULL;
lustro_xof_absorber_new(11, &a);
lustro_xof_absorber_update(a, (const uint8_t *)"hello ", 6);
lustro_xof_absorber_update(a, (const uint8_t *)"world", 5);

LustroXof *xof = NULL;
lustro_xof_absorber_finalize(a, &xof);
lustro_xof_absorber_free(a);

uint64_t value;
lustro_xof_next_u64(xof, &value);
lustro_xof_free(xof);
  • Errors: as for the hasher. finalize before the declared length is reached fails (HashError::TooLittleData, ValueError, LUSTRO_ERROR_INVALID_LENGTH with *out_ctx set to NULL), and more pieces can follow.
  • Stream: finalize does not end the absorber; each call returns a new stream at position 0. In C it is an ordinary LustroXof context, freed with lustro_xof_free, also after the absorber is freed.

Random Seed

Rust (--features entropy, also enabled by rand, python and ffi)

use lustro::{LustroPrng, StreamId};

// Err(EntropyError) if the operating system cannot provide entropy.
// Stream 0 of a fresh seed; the seed is not kept.
let mut rng = LustroPrng::from_entropy()?;

// Stream 5 of another fresh seed; unrelated to `rng`.
let mut other = LustroPrng::from_entropy_with_id(StreamId(5))?;

To keep the seed, take it first: Seed256::from_entropy()?, then LustroPrng::new(&seed, StreamId(0)).

Python

from lustro import LustroPrng

rng = LustroPrng.from_entropy()  # OSError if the system source is unavailable
other = LustroPrng.from_entropy(stream_id=5)  # another fresh seed, unrelated to `rng`

The seed is not kept. To keep it, use LustroPrng(os.urandom(32), 0).

C / C++

lustro_seed_from_entropy fills a 32-byte seed from the system's entropy source; lustro_prng_from_entropy creates a PRNG directly (stream ID as hi, lo, the seed is not kept):

#include "lustro.h"

LustroPrng *rng = NULL;
LustroError err = lustro_prng_from_entropy(0, 0, &rng);
if (err != LUSTRO_ERROR_OK) {
    /* LUSTRO_ERROR_ENTROPY_UNAVAILABLE: the system entropy source is unavailable */
}

/* To keep the seed, take it first */
uint8_t seed[LUSTRO_SEED_LEN];
LustroPrng *kept = NULL;
err = lustro_seed_from_entropy(seed);
if (err == LUSTRO_ERROR_OK) {
    err = lustro_prng_new(seed, 0, 0, &kept);
}

lustro_prng_free(kept);
lustro_prng_free(rng);

Typed Reads

Read integers, a full block, or any number of bytes from a single stream.

Rust

use lustro::{LustroPrng, Seed256, StreamId};
use std::io::Read;

let seed = Seed256::from_bytes([0u8; 32]);
let mut rng = LustroPrng::new(&seed, StreamId(0));

let a = rng.next_u32();
let b = rng.next_u64();
let c = rng.next_u128();
let block = rng.next_block();   // [u8; 32]

let mut buf = [0u8; 100];
rng.fill_bytes(&mut buf);

let mut bytes = Vec::new();
rng.by_ref().take(100).read_to_end(&mut bytes).unwrap();   // std::io::Read

rng.advance_bytes(50);   // skip 50 bytes
rng.advance_blocks(2);   // skip 2 blocks

let position: u128 = rng.position();

Python

from lustro import LustroPrng

rng = LustroPrng(bytes(32), 0)

a = rng.next_u32()
b = rng.next_u64()
c = rng.next_u128()
block = rng.next_block()        # bytes, 32
data = rng.fill(100)            # bytes, 100

rng.advance_bytes(50)           # skip 50 bytes
rng.advance_blocks(2)           # skip 2 blocks

position = rng.position()       # int

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);

uint32_t a;
uint64_t b;
lustro_prng_next_u32(rng, &a);
lustro_prng_next_u64(rng, &b);

uint8_t c[16];                      /* u128, 16 bytes little-endian */
lustro_prng_next_u128(rng, c);

uint8_t block[LUSTRO_BLOCK_LEN];
lustro_prng_next_block(rng, block);

uint8_t data[100];
lustro_prng_fill(rng, data, sizeof(data));

lustro_prng_advance_bytes(rng, 50);   /* skip 50 bytes */
lustro_prng_advance_blocks(rng, 2);   /* skip 2 blocks */

uint8_t position[16];                 /* u128, 16 bytes little-endian */
lustro_prng_position(rng, position);

lustro_prng_free(rng);

LustroXof reads the same way: use a LustroXof in place of rng (Rust and Python), or the lustro_xof_* functions in C/C++ (lustro_xof_next_u32, lustro_xof_next_u64, lustro_xof_next_u128, lustro_xof_next_block, lustro_xof_fill).

All values are little-endian and taken from the same byte stream: in the examples above a is bytes 0–3, b bytes 4–11 and c bytes 12–27. next_block() always advances to a full new 32-byte block and discards the unread rest of the current one.

advance_bytes(n) skips n bytes and advance_blocks(n) skips n blocks, leaving the stream where fill of n bytes or n calls of next_block() would. Both cost O(n); there is no jump-ahead. position() returns the bytes taken so far, and the discarded rest of a block counts as taken. They exist for LustroPrng and LustroXof in Rust, Python and C/C++ (lustro_prng_* and lustro_xof_*). In C/C++ n is a uint64_t, and the position is written as 16 bytes, little-endian; lustro_u128_to_parts splits it into (hi, lo).

In Rust, LustroPrng and LustroXof also implement std::io::Read: read fills the whole buffer and returns its length, like fill_bytes. The stream has no end, so read_to_end and read_to_string return an InvalidInput error and read nothing, and io::copy and bytes() without a limit never stop. Bound the read, for example with take.

Clone

Copy a stream with its exact state. The copy is independent: both continue with the same bytes, and neither affects the other. The clone keeps the position inside the current block as well.

Rust

use lustro::{LustroPrng, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let mut rng = LustroPrng::new(&seed, StreamId(0));
rng.next_u64();

let mut twin = rng.clone();
assert_eq!(rng.next_u64(), twin.next_u64());

Python

from lustro import LustroPrng

rng = LustroPrng(bytes(32), 0)
rng.next_u64()

twin = rng.copy()  # copy.copy() and copy.deepcopy() work too
assert rng.next_u64() == twin.next_u64()

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);

LustroPrng *twin = NULL;
lustro_prng_clone(rng, &twin);

lustro_prng_free(twin);
lustro_prng_free(rng);

LustroXof is cloned the same way: xof.clone() (Rust), xof.copy() (Python), lustro_xof_clone(xof, &twin) (C/C++, freed with lustro_xof_free).

Fork

Derive a child stream from the current state and a 128-bit identifier. The parent is not changed. The child depends on the parent's current block, not on the position inside it: forking after 1 byte or after 31 bytes of the same block gives the same child.

Rust

use lustro::{LustroPrng, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let rng = LustroPrng::new(&seed, StreamId(0));

let mut child = rng.fork(StreamId(1));
let value = child.next_u64();

Python

from lustro import LustroPrng

rng = LustroPrng(bytes(32), 0)

child = rng.fork(1)
value = child.next_u64()

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);

LustroPrng *child = NULL;
lustro_prng_fork(rng, 0, 1, &child);  /* id as (hi, lo) */

uint64_t value;
lustro_prng_next_u64(child, &value);

lustro_prng_free(child);
lustro_prng_free(rng);

LustroXof forks the same way: xof.fork(StreamId(1)) (Rust), xof.fork(1) (Python), lustro_xof_fork(xof, 0, 1, &child) (C/C++).

The same parent state and identifier always give the same child. The child starts at its own first block and does not inherit the parent's position.

A 32-byte output block is the stream's state at that point, and fork() derives from that state. So once a PRNG has generated a block, fork(id) equals LustroPrng::new with that block as the seed and the same identifier.

Derive Path

Derive a stream directly from a seed (or message) and a path of identifiers, without exposing the intermediate streams.

Rust

use lustro::{LustroPrng, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let rng = LustroPrng::derive_path(&seed, &[StreamId(12), StreamId(7), StreamId(99)]);

Python

from lustro import LustroPrng

seed = bytes(32)
rng = LustroPrng.derive_path(seed, [12, 7, 99])

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};

uint64_t ids_hi[3] = {0, 0, 0};
uint64_t ids_lo[3] = {12, 7, 99};

LustroPrng *rng = NULL;
lustro_prng_derive_path(seed, ids_hi, ids_lo, 3, &rng);

lustro_prng_free(rng);

For an XOF the message takes the place of the seed: LustroXof::derive_path(b"hello world", &[StreamId(12), StreamId(7), StreamId(99)]) (Rust), LustroXof.derive_path(b"hello world", [12, 7, 99]) (Python) and lustro_xof_derive_path(message, message_len, ids_hi, ids_lo, 3, &xof) (C/C++).

For a PRNG, derive_path(seed, path) gives the same stream as new with the first identifier followed by fork for each remaining one, in a single call. For an XOF, every identifier in path is a fork, since the message already is the root.

path must not be empty: derive_path panics in Rust (try_derive_path returns an error), Python raises ValueError, and C/C++ returns LUSTRO_ERROR_INVALID_LENGTH. The same holds for the batch versions below.


Batches

Batch PRNG and XOF

Many independent streams in one object, for PRNG and XOF alike.

A PRNG or XOF batch produces output in steps: fill_blocks(out, steps) writes steps blocks for every stream. steps only sets how much one call produces. The same total split over several calls gives every stream the same blocks. suggested_steps() returns a steps chosen for speed at the current number of streams; it is only a hint, any steps gives correct output. The output holds len() * steps blocks, so with many streams check its size before allocating.

Rust

// PRNG
use lustro::{LustroPrngBatch, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let mut prng = LustroPrngBatch::new(&seed, &ids);

let steps = prng.suggested_steps(); // or a fixed number, e.g. 2
let mut out = vec![[0u8; 32]; prng.len() * steps];
prng.fill_blocks(&mut out, steps);

// Or let the library allocate the vector:
let more = prng.blocks(steps); // Vec<[u8; 32]>, prng.len() * steps blocks
// XOF
use lustro::LustroXofBatch;

let messages = [b"msg0".as_ref(), b"msg1".as_ref(), b"msg2".as_ref()];
let mut xof = LustroXofBatch::new(&messages);

let steps = xof.suggested_steps(); // or a fixed number, e.g. 2
let mut out = vec![[0u8; 32]; xof.len() * steps];
xof.fill_blocks(&mut out, steps);

let more = xof.blocks(steps);

Python

# PRNG
import numpy as np
from lustro import LustroPrngBatch

prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])

steps = prng.suggested_steps()  # or a fixed number, e.g. 2
out = np.empty((steps, len(prng), 4), dtype=np.uint64)  # (steps, n_streams, 4)
prng.fill_blocks(out, steps)

# Or let the library allocate the array:
more = prng.blocks(steps)
# XOF
import numpy as np
from lustro import LustroXofBatch

xof = LustroXofBatch([b"msg0", b"msg1", b"msg2"])

steps = xof.suggested_steps()  # or a fixed number, e.g. 2
out = np.empty((steps, len(xof), 4), dtype=np.uint64)  # (steps, n_streams, 4)
xof.fill_blocks(out, steps)

more = xof.blocks(steps)

fill_blocks(out, steps) writes one 32-byte block per stream per step, step-major: stream i at step s is block s * n + i (n = number of streams). In Python that is out[s, i], with out shaped (steps, n_streams, 4); each block is four uint64 values. blocks(steps) (Rust and Python) returns the same blocks in a newly allocated result: a Vec<[u8; 32]> in Rust, a new, writable array of that shape in Python. In Python steps can be left out or None, and blocks() then uses suggested_steps().

In Python, a batch keeps an internal buffer the size of its largest fill_blocks() call made with the default release_gil=True; release_buffer() frees it earlier. fill_blocks(..., release_gil=False) on little-endian targets and blocks() do not use that buffer.

C / C++

/* PRNG */
#include <stdlib.h>
#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};

uint64_t ids_hi[4] = {0};
uint64_t ids_lo[4] = {0, 1, 2, 3};

LustroPrngBatch *prng = NULL;
lustro_prng_batch_new(seed, ids_hi, ids_lo, 4, &prng);

size_t steps = lustro_prng_batch_suggested_steps(prng);  /* or a fixed number, e.g. 2 */
size_t out_len = steps * lustro_prng_batch_len(prng) * LUSTRO_BLOCK_LEN;
uint8_t *out = (uint8_t *)malloc(out_len);

lustro_prng_batch_fill_blocks(prng, out, out_len, steps);

free(out);
lustro_prng_batch_free(prng);
/* XOF */
#include <stdlib.h>
#include "lustro.h"

const uint8_t *messages[] = {
    (const uint8_t *)"msg0",
    (const uint8_t *)"msg1",
    (const uint8_t *)"msg2"
};
uintptr_t lengths[] = {4, 4, 4};

LustroXofBatch *xof = NULL;
lustro_xof_batch_new(messages, lengths, 3, &xof);

size_t steps = lustro_xof_batch_suggested_steps(xof);  /* or a fixed number, e.g. 2 */
size_t out_len = steps * lustro_xof_batch_len(xof) * LUSTRO_BLOCK_LEN;
uint8_t *out = (uint8_t *)malloc(out_len);

lustro_xof_batch_fill_blocks(xof, out, out_len, steps);

free(out);
lustro_xof_batch_free(xof);

lustro_*_batch_len returns 0 for an empty batch and also for NULL. lustro_*_batch_get_len(ctx, &len) tells the two apart: it returns LUSTRO_ERROR_INVALID_POINTER for NULL and writes nothing.

LustroPrngBatch can also be created for consecutive stream IDs: LustroPrngBatch::new_range(&seed, StreamId(first), count) (Rust), LustroPrngBatch.new_range(seed, first, count) (Python) and lustro_prng_batch_new_range(seed, first_hi, first_lo, count, &prng) (C/C++) use the IDs first, first + 1, ... XOF batches have no new_range.

In C/C++, lustro_batch_block_offset(step, batch_len, lane) from lustro.h gives the byte offset of one block in the output: (step * batch_len + lane) * LUSTRO_BLOCK_LEN. It expects step < steps and lane < batch_len and does not check its arguments.

/* lane 2 at step 0 of the PRNG output above */
const uint8_t *block = out + lustro_batch_block_offset(0, 4, 2);

Batch into Single Lane

Take one lane out of a batch as a single stream. The stream continues the lane: its next block is the block the batch would write for that lane next. The batch is not changed, and the two evolve independently afterwards.

Rust

use lustro::{LustroPrngBatch, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let mut prng = LustroPrngBatch::new(&seed, &ids);

prng.blocks(2);

// None if the index is out of range.
let mut lane = prng.lane(1).expect("lane in range");
let block = lane.next_block();

Python

from lustro import LustroPrngBatch

prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])
prng.blocks(2)

lane = prng.lane(1)  # IndexError if the index is out of range
same = prng[1]       # the same lane; prng[-1] is the last one
block = lane.next_block()

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrngBatch *prng = NULL;
lustro_prng_batch_new_range(seed, 0, 0, 4, &prng);

LustroPrng *lane = NULL;
LustroError err = lustro_prng_batch_lane(prng, 1, &lane);  /* LUSTRO_ERROR_INVALID_ARGUMENT if index >= len */

uint8_t block[LUSTRO_BLOCK_LEN];
lustro_prng_next_block(lane, block);

lustro_prng_free(lane);  /* a single stream, not a batch */
lustro_prng_batch_free(prng);

LustroXofBatch has lane too: xof.lane(1) (Rust, Python) and lustro_xof_batch_lane(xof, 1, &lane) (C/C++, the lane is a LustroXof, freed with lustro_xof_free).

The result is an ordinary single stream (LustroPrng or LustroXof), so it supports typed reads, fork and snapshots.

In Python, batch[i] does what batch.lane(i) does, and a negative i counts from the end (lane(i) itself rejects it). for lane in batch goes through all lanes in order, as copies of the lane states from the start of the loop. Slices are not supported.

Single Streams into Batch

Build a batch from single streams, one lane per stream, in order. Each lane continues its stream at the next block, so batch.lane(i) gives back a copy of stream i. The streams are not changed.

A lane holds whole blocks only, so every stream must be at a block boundary: freshly created, or read in whole 32-byte blocks. A stream in the middle of a block is rejected rather than moved on, so no bytes are dropped. Read or skip the rest of the block first: a stream whose position() is not a multiple of 32 is inside a block, and 32 - position() % 32 more bytes bring it to the next boundary.

Rust

use lustro::{LustroPrng, LustroPrngBatch, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let a = LustroPrng::new(&seed, StreamId(0));
let mut b = LustroPrng::new(&seed, StreamId(1));
b.next_u64();   // b is now inside a block

// Err(BatchError::PartialBlock { index: 1 }); from_streams would panic.
assert!(LustroPrngBatch::try_from_streams(&[a.clone(), b.clone()]).is_err());

b.advance_bytes(24);   // the rest of the block
let batch = LustroPrngBatch::from_streams(&[a, b]);

Python

from lustro import LustroPrng, LustroPrngBatch

a = LustroPrng(bytes(32), 0)
b = LustroPrng(bytes(32), 1)
b.next_u64()          # b is now inside a block; from_streams would raise ValueError

b.advance_bytes(24)   # the rest of the block
batch = LustroPrngBatch.from_streams([a, b])

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *a = NULL;
LustroPrng *b = NULL;
lustro_prng_new(seed, 0, 0, &a);
lustro_prng_new(seed, 0, 1, &b);

const LustroPrng *streams[] = {a, b};
LustroPrngBatch *batch = NULL;
LustroError err = lustro_prng_batch_from_streams(streams, 2, &batch);  /* LUSTRO_ERROR_INVALID_ARGUMENT if a stream is inside a block */

lustro_prng_batch_free(batch);
lustro_prng_free(b);
lustro_prng_free(a);

LustroXof streams go into a LustroXofBatch the same way: LustroXofBatch::from_streams(&[a, b]) (Rust), LustroXofBatch.from_streams([a, b]) (Python) and lustro_xof_batch_from_streams(streams, 2, &batch) with an array of const LustroXof * (C/C++).

The streams can come from different seeds or messages, and the same stream can appear more than once. A stream inside a block gives BatchError::PartialBlock { index } from try_from_streams in Rust (index is the first such stream; from_streams panics), ValueError in Python and LUSTRO_ERROR_INVALID_ARGUMENT in C/C++. In C/C++ the streams stay owned by the caller and are freed separately. In Python the source can also be a batch, which gives a copy of it.

Fork into a Batch

Derive many children of one stream in a single call. fork_batch(ids) returns a batch whose lane i is fork(ids[i]), and fork_batch_range(first, count) uses the IDs first, first + 1, ... fork_path(path) returns fork(path[0]).fork(path[1])... without the intermediate streams. All three start from the current state, like fork, and do not change the parent.

Rust

use lustro::{LustroPrng, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let rng = LustroPrng::new(&seed, StreamId(0));

let children = rng.fork_batch(&[StreamId(10), StreamId(20)]); // LustroPrngBatch, 2 lanes
let numbered = rng.fork_batch_range(StreamId(100), 8);        // ids 100 to 107
let leaf = rng.fork_path(&[StreamId(12), StreamId(7)]);       // rng.fork(12).fork(7)

Python

from lustro import LustroPrng

rng = LustroPrng(bytes(32), 0)

children = rng.fork_batch([10, 20])        # LustroPrngBatch, 2 lanes
numbered = rng.fork_batch_range(100, 8)    # ids 100 to 107
leaf = rng.fork_path([12, 7])              # rng.fork(12).fork(7)

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);

uint64_t ids_hi[2] = {0, 0};
uint64_t ids_lo[2] = {10, 20};

LustroPrngBatch *children = NULL;
lustro_prng_fork_batch(rng, ids_hi, ids_lo, 2, &children);

LustroPrngBatch *numbered = NULL;
lustro_prng_fork_batch_range(rng, 0, 100, 8, &numbered);  /* first id as (hi, lo), then count */

uint64_t path_hi[2] = {0, 0};
uint64_t path_lo[2] = {12, 7};

LustroPrng *leaf = NULL;
lustro_prng_fork_path(rng, path_hi, path_lo, 2, &leaf);

lustro_prng_free(leaf);
lustro_prng_batch_free(numbered);
lustro_prng_batch_free(children);
lustro_prng_free(rng);

LustroXof has the same methods (lustro_xof_fork_batch, lustro_xof_fork_batch_range and lustro_xof_fork_path in C/C++). The first two return a LustroXofBatch, fork_path a LustroXof.

For a stream that has not produced output yet, fork_path gives what derive_path gives: LustroPrng::new(&seed, StreamId(5)).fork_path(path) is derive_path(&seed, [5] + path). It is for the cases derive_path does not cover: a stream from from_entropy(), a lane of a batch, a restored snapshot, or a stream that has already produced output. path must not be empty, with the same errors as derive_path.

Batch Clone

Copy a whole batch with the exact state of every lane. The copy is independent.

Rust

use lustro::{LustroPrngBatch, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let mut prng = LustroPrngBatch::new(&seed, &ids);
prng.blocks(2);

let twin = prng.clone();

Python

from lustro import LustroPrngBatch

prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])
prng.blocks(2)

twin = prng.copy()

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrngBatch *prng = NULL;
lustro_prng_batch_new_range(seed, 0, 0, 4, &prng);

LustroPrngBatch *twin = NULL;
LustroError err = lustro_prng_batch_clone(prng, &twin);

lustro_prng_batch_free(twin);
lustro_prng_batch_free(prng);

LustroXofBatch is cloned the same way: xof.clone() (Rust), xof.copy() (Python), lustro_xof_batch_clone(xof, &twin) (C/C++).

Batch Fork

Create multiple child streams. The parent is not changed.

Rust

use lustro::{LustroPrngBatch, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];

let prng = LustroPrngBatch::new(&seed, &ids);
let one_each = prng.fork(&[StreamId(10), StreamId(11), StreamId(12), StreamId(13)]);
let children = prng.fork_range(StreamId(100));
let many_children = prng.fork_many(&[StreamId(100), StreamId(200)]);

Python

from lustro import LustroPrngBatch

prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])

one_each = prng.fork([10, 11, 12, 13])
children = prng.fork_range(100)
many_children = prng.fork_many([100, 200])

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrngBatch *prng = NULL;
lustro_prng_batch_new_range(seed, 0, 0, 4, &prng);

uint64_t ids_hi[4] = {0, 0, 0, 0};
uint64_t ids_lo[4] = {10, 11, 12, 13};

LustroPrngBatch *one_each = NULL;
LustroError err = lustro_prng_batch_fork(prng, ids_hi, ids_lo, 4, &one_each);

LustroPrngBatch *children = NULL;
lustro_prng_batch_fork_range(prng, 0, 100, &children);

uint64_t k_hi[2] = {0, 0};
uint64_t k_lo[2] = {100, 200};

LustroPrngBatch *many_children = NULL;
lustro_prng_batch_fork_many(prng, k_hi, k_lo, 2, &many_children);

lustro_prng_batch_free(many_children);
lustro_prng_batch_free(children);
lustro_prng_batch_free(one_each);
lustro_prng_batch_free(prng);

LustroXofBatch has the same methods: fork, fork_range and fork_many (lustro_xof_batch_fork, lustro_xof_batch_fork_range and lustro_xof_batch_fork_many in C/C++).

Method ids Result Child at index
fork(ids) exactly batch.len() IDs batch.len() lanes i: child of lane i with ids[i]
fork_range(first) none batch.len() lanes i: child of lane i with ID first + i
fork_many(ids) any number k batch.len() * k lanes i * k + j: child j of lane i with ids[j]

If ids of fork does not have exactly batch.len() elements, fork panics and try_fork returns BatchError::LengthMismatch in Rust; Python raises ValueError and C/C++ returns LUSTRO_ERROR_INVALID_LENGTH. Each child starts at step 0.

Batch Derive Path

Derive a batch directly from a shared path, one lane per root (PRNG) or per message (XOF).

Rust

use lustro::{LustroPrngBatch, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let roots = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let path = [StreamId(12), StreamId(7)];

let batch = LustroPrngBatch::derive_path(&seed, &roots, &path);

Python

from lustro import LustroPrngBatch

seed = bytes(32)
batch = LustroPrngBatch.derive_path(seed, [0, 1, 2, 3], [12, 7])

C / C++

#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};

uint64_t roots_hi[4] = {0, 0, 0, 0};
uint64_t roots_lo[4] = {0, 1, 2, 3};

uint64_t path_hi[2] = {0, 0};
uint64_t path_lo[2] = {12, 7};

LustroPrngBatch *batch = NULL;
lustro_prng_batch_derive_path(
    seed, roots_hi, roots_lo, 4, path_hi, path_lo, 2, &batch
);

lustro_prng_batch_free(batch);

An XOF batch takes the messages instead of the seed and the roots: LustroXofBatch::derive_path(&messages, &path) (Rust), LustroXofBatch.derive_path(messages, path) (Python) and lustro_xof_batch_derive_path(messages, lengths, 3, path_hi, path_lo, 2, &batch) (C/C++).

For a PRNG batch, derive_path(seed, roots, path) derives a canonical batch: lane i equals LustroPrng::derive_path(seed, [roots[i]] + path), independent of any other stream's state. For an XOF batch, lane i equals LustroXof::derive_path(messages[i], path). In Python the messages can be any sequence of bytes or byte buffers. In Rust, try_derive_path returns DerivePathError::EmptyPath for an empty path instead of panicking.

Allocation Failures

Batch functions reserve memory for the lanes, the output and the buffer of a batch snapshot. If a reservation fails:

Plain function
Rust panics (clone() aborts the process); the try_ variant of the same name returns an error instead
Python MemoryError
C/C++ LUSTRO_ERROR_OUT_OF_MEMORY, except from lustro_*_batch_export_snapshot, which writes into your buffer and allocates nothing

The Rust try_ variants of LustroPrngBatch and LustroXofBatch (try_new_range for LustroPrngBatch only):

Variants Error
try_new, try_new_range, try_from_streams, try_fork, try_fork_range, try_clone, try_export_snapshot BatchError::Reserve
try_fork_batch, try_fork_batch_range of LustroPrng and LustroXof BatchError::Reserve
try_to_le_bytes of LustroPrngBatchSnapshot and LustroXofBatchSnapshot BatchError::Reserve
try_hash256_many_var, try_hash128_many_var BatchError::Reserve
try_hash256_many_fixed, try_hash128_many_fixed BatchError::Reserve; BatchError::InvalidMessageLen for a message_len of 0 or a data.len() that is not a multiple of it (the plain versions panic then)
try_blocks, try_fork_many BatchError::Reserve; BatchError::SizeOverflow when len() * steps or len() * ids.len() overflows usize (the plain versions panic then too)
try_derive_path DerivePathError::Reserve
try_from_le_bytes of the batch snapshots (also through TryFrom<&[u8]>) SnapshotError::Reserve

State and Runtime

Snapshot

Save and restore the exact stream position.

Rust

use lustro::types::LustroPrngSnapshot;
use lustro::{LustroPrng, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let mut rng = LustroPrng::new(&seed, StreamId(0));

let snapshot = rng.export_snapshot();
let bytes = snapshot.to_le_bytes();

let snapshot =
    LustroPrngSnapshot::from_le_bytes(&bytes).expect("valid snapshot");

let mut restored = LustroPrng::import_snapshot(snapshot);

Python

from lustro import LustroPrng

rng = LustroPrng(bytes(32), 0)

snapshot = rng.export_snapshot()
restored = LustroPrng.import_snapshot(snapshot)

C / C++

#include <stdio.h>
#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);

uint8_t snapshot[LUSTRO_SNAPSHOT_LEN];
lustro_prng_export_snapshot(rng, snapshot, LUSTRO_SNAPSHOT_LEN);

LustroPrng *restored = NULL;
LustroError err = lustro_prng_import_snapshot(snapshot, LUSTRO_SNAPSHOT_LEN, &restored);
if (err != LUSTRO_ERROR_OK) {
    fprintf(stderr, "%s\n", lustro_strerror(err));
}

lustro_prng_free(restored);
lustro_prng_free(rng);

LustroXof works the same way: export_snapshot(), LustroXofSnapshot::from_le_bytes (lustro::types) and LustroXof::import_snapshot in Rust; export_snapshot() and LustroXof.import_snapshot in Python; lustro_xof_export_snapshot and lustro_xof_import_snapshot with LUSTRO_SNAPSHOT_LEN in C/C++.

A snapshot holds the full stream state, so keep it as private as the seed (for an XOF, the message). Importing malformed bytes fails: SnapshotError from from_le_bytes (Rust), lustro.SnapshotError, a ValueError (Python), or LUSTRO_ERROR_UNSUPPORTED_VERSION, _INVALID_KIND, _INVALID_RESERVED, _INVALID_CURSOR or _INVALID_LENGTH (C/C++). A PRNG snapshot cannot be imported as an XOF or the other way round, and a single-stream snapshot cannot be imported as a batch.

The reserved bytes of a snapshot are zero: export writes zeros, and import rejects anything else (InvalidReserved). Apart from the header, the cursor and these bytes, a snapshot has no integrity check: a changed s0, s1 or step imports without error and continues another stream. If stored snapshots can be damaged, add your own checksum.

To find out which type exported some bytes, call SnapshotKind::detect(bytes) (Rust), lustro.snapshot_kind(bytes) (Python: "prng", "xof", "prng_batch" or "xof_batch") or lustro_snapshot_kind(bytes, len, &kind) (C/C++: LUSTRO_SNAPSHOT_KIND_*). It reads only the version and kind bytes, so a result does not mean the import will succeed.

Batch Snapshot

Save and restore every lane of a batch at once. The snapshot has a dynamic size: a header plus one entry per lane.

Rust

use lustro::types::LustroPrngBatchSnapshot;
use lustro::{LustroPrngBatch, Seed256, StreamId};

let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let mut prng = LustroPrngBatch::new(&seed, &ids);
prng.blocks(2);

let snapshot = prng.export_snapshot();
let bytes = snapshot.to_le_bytes();   // Vec<u8>

let snapshot =
    LustroPrngBatchSnapshot::from_le_bytes(&bytes).expect("valid snapshot");

let mut restored = LustroPrngBatch::import_snapshot(snapshot);

Python

from lustro import LustroPrngBatch

prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])
prng.blocks(2)

snapshot = prng.export_snapshot()   # bytes
restored = LustroPrngBatch.import_snapshot(snapshot)

C / C++

#include <stdio.h>
#include <stdlib.h>
#include "lustro.h"

uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrngBatch *prng = NULL;
lustro_prng_batch_new_range(seed, 0, 0, 4, &prng);

size_t size = lustro_prng_batch_snapshot_size(prng);
uint8_t *snapshot = (uint8_t *)malloc(size);
lustro_prng_batch_export_snapshot(prng, snapshot, size);

LustroPrngBatch *restored = NULL;
LustroError err = lustro_prng_batch_import_snapshot(snapshot, size, &restored);
if (err != LUSTRO_ERROR_OK) {
    fprintf(stderr, "%s\n", lustro_strerror(err));
}

lustro_prng_batch_free(restored);
free(snapshot);
lustro_prng_batch_free(prng);

LustroXofBatch works the same way: LustroXofBatchSnapshot (lustro::types) in Rust, LustroXofBatch.import_snapshot in Python, lustro_xof_batch_snapshot_size, lustro_xof_batch_export_snapshot and lustro_xof_batch_import_snapshot in C/C++.

The snapshot size is LUSTRO_BATCH_SNAPSHOT_HEADER_LEN + n * LUSTRO_BATCH_SNAPSHOT_LANE_LEN for n streams. In C/C++ call *_batch_snapshot_size(ctx) before export; out_len must equal it.

Runtime Settings

Thread count and SIMD backend follow environment variables by default. configure sets them from code, best before first use (see below). They apply to hash batches as well as to PRNG and XOF batches.

Variable configure Effect Values
LUSTRO_NUM_THREADS threads Pool size, capped at the CPU count; 1 disables the pool A positive number. Anything else (abc, 0, -1) is ignored and the pool gets the full CPU count
LUSTRO_DISABLE_MT threads=1 Never use the thread pool Flag
LUSTRO_DISABLE_SIMD allow_simd=False Scalar backend Flag

A flag is set by 1, true, on or yes, and not set by 0, false, off, no or an empty value (case is ignored). Any other value sets it.

Rust

use lustro::{configure, runtime_info, RuntimeOptions};
use std::num::NonZeroUsize;

let options = RuntimeOptions::new()
    .threads(NonZeroUsize::new(4).unwrap())
    .allow_simd(true);

configure(options)?;

// Read the settings.
let info = runtime_info();

Python

import lustro

lustro.configure(threads=4, allow_simd=True)

info = lustro.runtime_info()
print(info["backend"], info["threads"], info["fixed"])

C / C++

#include <stdio.h>
#include "lustro.h"

LustroError err = lustro_configure(4, LUSTRO_SIMD_AUTO);
if (err != LUSTRO_ERROR_OK) {
    fprintf(stderr, "%s\n", lustro_strerror(err));
}

Arguments you omit (None in Python, 0 and LUSTRO_SIMD_KEEP in C/C++) still follow the environment; the ones you pass override it. In C/C++, simd is LUSTRO_SIMD_KEEP, LUSTRO_SIMD_AUTO (allow_simd=True) or LUSTRO_SIMD_SCALAR (allow_simd=False).

The settings are fixed once, by whichever of these comes first:

  • configure;
  • a hash batch call;
  • batch fill_blocks or blocks;
  • creating an XOF batch from at least one message (LustroXofBatch::new, LustroXofBatch(...), lustro_xof_batch_new).

These do not fix them: runtime_info(), suggested_steps() and, in C/C++, lustro_runtime_threads() and lustro_runtime_backend(). Two edge cases:

  • A batch call that fails once it has started the hash (for example hash*_many_var_into with release_gil=False on a big list that has a bad item) has fixed them.
  • In C/C++ the *_many_* functions with n == 0 and *_batch_fill_blocks with n == 0 or steps == 0 return before the dispatcher and fix nothing. In Rust and Python a hash batch call with zero messages fixes them.

The settings belong to the whole process. An application calls configure early, before any batch; a library that uses Lustro should not call it and can read the settings with runtime_info().

A later configure succeeds if it asks for the same effect as the fixed settings, and fails otherwise: ConfigError::AlreadyConfigured (Rust), lustro.ConfigError, a subclass of RuntimeError (Python), LUSTRO_ERROR_ALREADY_CONFIGURED (C/C++). Same effect means the same SIMD choice and, with the parallel feature, the same pool size. For the pool size, threads above the CPU count counts as the CPU count, and threads=1 equals LUSTRO_DISABLE_MT. The SIMD choice is compared as given: on a CPU without AVX2, allow_simd=True and False still differ.

When calls race, one wins and the rest are compared with it. Bad arguments change nothing: ValueError for threads < 1 (Python), LUSTRO_ERROR_INVALID_ARGUMENT for an unknown simd (C/C++).

runtime_info() (Rust, Python, and lustro_runtime_info() in C/C++) returns the report without starting the pool. Before the settings are fixed it reads the environment as it is now (fixed is false). The report may be out of date by the time it returns.

Field Meaning
backend "scalar" or "avx2", as used for the largest batches
threads configured pool size, not running workers; 0 = batches run on the calling thread (also without parallel)
fixed the settings are fixed; false before the first configure or batch call
parallel_built built with the parallel feature
simd_available the CPU has AVX2; backend shows whether it is used
pool_usable the pool exists and this process can use it. False before the first parallel batch, after a failed pool build, and in a child forked after its parent started the pool
logical_cpus logical CPUs at the call; the cap for threads

In C/C++, lustro_runtime_info(&info, sizeof info) fills a LustroRuntimeInfo and returns LUSTRO_ERROR_OK, LUSTRO_ERROR_INVALID_POINTER for a NULL info, or LUSTRO_ERROR_INVALID_LENGTH if the size is too small. The struct has the fields above; the flags are 0 or 1 and backend is a static string (do not free it).

LustroRuntimeInfo info;
if (lustro_runtime_info(&info, sizeof info) == LUSTRO_ERROR_OK) {
    printf("%s, %zu threads, fixed=%d\n", info.backend, (size_t)info.threads, info.fixed);
}

Language Notes

Python Notes

Single streams (LustroPrng, LustroXof) can also return or fill larger amounts of data:

from lustro import LustroPrng

rng = LustroPrng(bytes(32), 0)

data = rng.fill(1000)         # bytes
buf = bytearray(1000)
rng.fill_into(buf)            # bytearray, writable memoryview or NumPy uint8 array
words = rng.fill_u64(1000)    # NumPy uint64 array of shape (1000,)

data = rng.read(1000)         # same as fill(1000)
n = rng.readinto(buf)         # n == 1000, any writable C-contiguous buffer

fill_u64(n) returns a new, writable array with the values that n calls of next_u64() would give. read(size) is the same as fill(size); there is no read() without a size, because the stream has no end.

readinto(buf) accepts any writable, C-contiguous buffer whatever its item type (for example array('I')), writes the stream as raw bytes and returns the number of bytes written; fill_into(buf) takes byte buffers only. A buffer that holds Python objects (NumPy dtype=object) raises TypeError. Other Python threads cannot change buf while the stream writes into it; native code that writes without the GIL is not covered.

Seeds, messages, hash input and snapshots are given as bytes. bytearray, memoryview, array('B') and NumPy uint8 arrays are accepted too (C-contiguous with byte items, TypeError otherwise) and are copied. hash*_many_var and LustroXofBatch take any sequence of such objects, but not a single string or buffer.

hash*_many_offsets and hash*_many_offsets_into hash messages packed in one buffer: message i is data[offsets[i]:offsets[i + 1]]. offsets is a 1D NumPy array of dtype int64, uint64, int32 or uint32 with one entry more than there are messages, non-negative, non-decreasing, the last at most len(data) (ValueError for bad entries, TypeError for another dtype or dimension).

Stream IDs and path elements are Python ints from 0 to 2**128 - 1 (OverflowError otherwise). A list of them can be any sequence, for example a tuple, a range or a NumPy integer array.

LustroPrng, LustroXof, LustroPrngBatch and LustroXofBatch can be pickled, for example to pass them to a worker process (LustroHasher and LustroXofAbsorber cannot). The restored object is a copy that continues from the same position, not a new stream: workers that need independent streams should use fork() or different stream IDs. Do not load pickles from untrusted sources; for long-term storage use export_snapshot().

An instance must not be used by several threads at the same time (an overlapping call raises RuntimeError). Use copy(), fork() or separate instances instead. The installed version is available as lustro.__version__, and the package ships type stubs (lustro.pyi, py.typed).

GIL and Throughput (Python)

Whether a call releases the GIL depends on what it does:

Calls GIL
hash256, hash128, fill, read, fill_u64, fill_into, readinto, advance_bytes (for advance_blocks, 128 blocks), LustroHasher.update, LustroXofAbsorber.update, LustroXof(message), LustroXof.derive_path released from 4096 bytes, held below that. The copy into buf in fill_into and readinto holds it
hash*_many_*, fill_blocks released; release_gil=False keeps it (for fill_blocks on little-endian targets only). hash256_many_fixed and hash128_many_fixed have no such argument, their _into variants do
blocks, LustroXofBatch(...), LustroXofBatch.derive_path released while they work
the other batch calls that build or copy lanes: constructors, new_range, fork*, from_streams, copy, iter(batch), snapshots, LustroPrngBatch.derive_path held; their time grows with the lane count

A pending Ctrl-C during advance_bytes or advance_blocks is raised between chunks of 2^20 blocks and leaves the stream unchanged.

By default (release_gil=True) other Python threads run during the call. Input that is not bytes is copied before the GIL is released (hash*_many_fixed* copy their rows), *_into calls compute the digests in a scratch buffer and copy them into out once the GIL is back, and calls that return a new array write the digests straight into it. On large batches the copies cost time.

For many messages of different lengths prefer hash*_many_offsets and hash*_many_offsets_into over hash*_many_var: no Python object is read or counted per message, in both modes.

With release_gil=False the GIL stays held and other Python threads wait. Where the input allows, data is read in place and digests are written straight into out. The thread pool and AVX2 are used in both modes.

out = np.empty((n, 32), dtype=np.uint8)
lustro.hash256_many_fixed_into(rows, out, release_gil=False)
lustro.hash256_many_var_into(messages, out, release_gil=False)

steps = prng.suggested_steps()
blocks = np.empty((steps, len(prng), 4), dtype=np.uint64)
prng.fill_blocks(blocks, steps, release_gil=False)

Not every input is read in place: for hash*_many_var*, an exact list or tuple of exact bytes is the fastest input, especially for many small messages. out must not share memory with the input (TypeError or ValueError, depending on how it is shared). If a big exact list or tuple (4096 items or more) given to hash*_many_var_into fails part-way, out may be partly written.

Each thread keeps up to four buffers between hash batch calls (copied rows, digests of *_into, copied offsets, the table of message slices), up to 32 MiB each and 128 MiB in all until the thread exits. The buffers are not cleared, so bytes of the last calls stay in them.

rand_core Integration (Rust)

With --features rand, LustroPrng and LustroXof implement the rand_core 0.10 traits TryRng and Rng (infallible, Error = Infallible). LustroPrng also implements SeedableRng. LustroXof does not: it is built from a message of any length.

[dependencies]
lustro_api = { version = "<version>", features = ["rand"] }
rand_core = "0.10"
The crate is published as `lustro_api`; its library is imported as `lustro`.
use lustro::{LustroPrng, LustroXof};
use rand_core::{Rng, SeedableRng};

fn draw(rng: &mut impl Rng) -> u64 {
    rng.next_u64()
}

// PRNG from a 32-byte seed (stream 0).
let mut prng = LustroPrng::from_seed([7u8; 32]);
let a = draw(&mut prng);

// Random seed (needs the system entropy source).
let mut random = LustroPrng::from_entropy()?;
let mut buf = [0u8; 100];
random.fill_bytes(&mut buf);

// XOF over a message of any length.
let mut xof = LustroXof::new(b"message");
let b = draw(&mut xof);
  • Same stream: Rng returns the same bytes, from the same position, as the inherent next_u32, next_u64 and fill_bytes.
  • Seeding: from_seed starts at stream 0. rand_core 0.10 has no from_entropy; use LustroPrng::from_entropy()?.
  • fork: on a concrete LustroPrng, prng.fork(id) is the inherent LustroPrng::fork(StreamId). The trait method SeedableRng::fork() runs in code generic over SeedableRng or when called as SeedableRng::fork(&mut prng); it seeds a new generator from the next 32 bytes of prng.
  • Clone and derive_path: both exist on LustroPrng and LustroXof and work without the rand feature. A clone repeats the same bytes, so do not use it to get a second independent stream; use fork or derive_path for that. derive_path is not a trait method, so generic code over Rng cannot call it; clone needs a Clone bound.
  • Quality: CryptoRng is not implemented.

Reference

Size Constants

Meaning Bytes C / C++ (lustro.h) Rust (lustro::types) Python (lustro)
Seed 32 LUSTRO_SEED_LEN LUSTRO_SEED_LEN SEED_LEN
Output block 32 LUSTRO_BLOCK_LEN LUSTRO_BLOCK_LEN BLOCK_LEN
hash128 digest 16 LUSTRO_HASH128_LEN LUSTRO_HASH128_LEN HASH128_LEN
hash256 digest 32 LUSTRO_HASH256_LEN LUSTRO_HASH256_LEN HASH256_LEN
Single snapshot 56 LUSTRO_SNAPSHOT_LEN LUSTRO_SNAPSHOT_LEN SNAPSHOT_LEN
Batch snapshot header 16 LUSTRO_BATCH_SNAPSHOT_HEADER_LEN LUSTRO_BATCH_SNAPSHOT_HEADER_LEN BATCH_SNAPSHOT_HEADER_LEN
Batch snapshot, per stream 48 LUSTRO_BATCH_SNAPSHOT_LANE_LEN LUSTRO_BATCH_SNAPSHOT_LANE_LEN BATCH_SNAPSHOT_LANE_LEN

For full API details please see ARCHITECTURE.md.

Metadata

Release files for lustro-api 1.0.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for lustro-api 1.0.1
File Size Uploaded Provenance Actions
lustro_api-1.0.1.tar.gz 213.5 kB With provenance Details for lustro_api-1.0.1.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for lustro-api 1.0.1
File Actions
lustro_api-1.0.1-cp311-abi3-win_amd64.whl CPython 3.11 abi3 Windows x86-64 Details for lustro_api-1.0.1-cp311-abi3-win_amd64.whl
lustro_api-1.0.1-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl CPython 3.11 abi3 Linux glibc 2.17+ x86-64 Details for lustro_api-1.0.1-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
lustro_api-1.0.1-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl CPython 3.11 abi3 Linux glibc 2.17+ ARM64 Details for lustro_api-1.0.1-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
lustro_api-1.0.1-cp311-abi3-macosx_11_0_arm64.whl CPython 3.11 abi3 macOS 11.0+ ARM64 Details for lustro_api-1.0.1-cp311-abi3-macosx_11_0_arm64.whl
lustro_api-1.0.1-cp311-abi3-macosx_10_12_x86_64.whl CPython 3.11 abi3 macOS 10.12+ x86-64 Details for lustro_api-1.0.1-cp311-abi3-macosx_10_12_x86_64.whl

Total release size: 2.4 MB

Release files / lustro_api-1.0.1.tar.gz

Download URL lustro_api-1.0.1.tar.gz
Size 213.5 kB
Tags Source
SHA-256 checksum
How to use checksums
3b9e450673a60d15abb217dbc70d6d566c423c320009a7ab2529d9b787b73f09
BLAKE2b-256 checksum
How to use checksums
931e1acb698be5e52fbf9b4a9f0f17e07c33ea919d800aa04a963d95cf61655b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.

Transparency log

Release files / lustro_api-1.0.1-cp311-abi3-win_amd64.whl

Download URL lustro_api-1.0.1-cp311-abi3-win_amd64.whl
Size 425.6 kB
Tags CPython 3.11 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
904da6ce5a72a273bb914e39b670e39740dfcf28c062bdb0bd6b85ad2ad0a3c3
BLAKE2b-256 checksum
How to use checksums
0f96eff525b22a6ff983dcd65af65a316e2d9a550b4efbd6d6d5b5930b5e1f1e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.

Transparency log

Release files / lustro_api-1.0.1-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl

Download URL lustro_api-1.0.1-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Size 486.0 kB
Tags CPython 3.11 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
d00efe267efca3a45b1d9ea6131684201d2057f11ef3df4e465695923c1aff91
BLAKE2b-256 checksum
How to use checksums
b01054682aafe6ef8d707a7f3221aef192d63f190819f578c8b9250f3a0ea9da
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.

Transparency log

Release files / lustro_api-1.0.1-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl

Download URL lustro_api-1.0.1-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
Size 433.4 kB
Tags CPython 3.11 Linux glibc 2.17+ ARM64 abi3
SHA-256 checksum
How to use checksums
3f584aae0abc39de1fada78e19d22f0a7a823fad4a39a1bb91b2ac3a3ba9276e
BLAKE2b-256 checksum
How to use checksums
5a92eaca6e2c3bc35fd8d3a157e4295bc50531663a1a9c78710c5f4ead7d885b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.

Transparency log

Release files / lustro_api-1.0.1-cp311-abi3-macosx_11_0_arm64.whl

Download URL lustro_api-1.0.1-cp311-abi3-macosx_11_0_arm64.whl
Size 413.6 kB
Tags CPython 3.11 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
76a20e2e76c54f207282c014e2731a894eb5558311455d31836345f1cd928e2e
BLAKE2b-256 checksum
How to use checksums
741216998445c3fca7f8e0f49a0e98a7417f543439a69c4bb6f9109428f0b65a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.

Transparency log

Release files / lustro_api-1.0.1-cp311-abi3-macosx_10_12_x86_64.whl

Download URL lustro_api-1.0.1-cp311-abi3-macosx_10_12_x86_64.whl
Size 460.1 kB
Tags CPython 3.11 abi3 macOS 10.12+ x86-64
SHA-256 checksum
How to use checksums
e293fd1f787c6f2c4ef749c895c6197908c28b792ba69c86e22748bc9a1c93a1
BLAKE2b-256 checksum
How to use checksums
8876b9c934d3e43602fff7de108cac7c933545442a12ef7c40b3bf4d1961735c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.

Transparency log

Security for lustro-api 1.0.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

All 6 files in this release have been uploaded with provenance metadata.

Select from 6 files

Attestation for lustro_api-1.0.1.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.0.1 This release

6 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page