Lustro API
Lustro is a 256-bit, deterministic, single-pass diffusion engine with three uses: a hash, a PRNG and an XOF. A stream can be forked into child streams, copied, and saved with its exact position. Forking does not change the parent. A batch advances many independent streams in one call and uses AVX2 and a thread pool when the CPU and the settings allow it. It is available as a Rust crate, a Python module (with NumPy) and a C/C++ library.
| Hash | PRNG | XOF | |
|---|---|---|---|
| Input | a message | a 32-byte seed and a 128-bit stream ID | a message of any length |
| Output | a 16 or 32-byte digest | a byte stream | a byte stream |
| One at a time | hash128, hash256, LustroHasher |
LustroPrng |
LustroXof, LustroXofAbsorber |
| Many at once | hash*_many_* |
LustroPrngBatch |
LustroXofBatch |
Questions? Please see the FAQ.
The core mechanism and the API implementation are currently considered non-cryptographic.
Build · Hashing · Streams · Batches · State and Runtime · Language Notes · Reference
Get the Code
git clone https://github.com/Ligatum/Lustro_API.git
cd Lustro_API
Build
# Rust
cargo build --release
# Rust with the rand_core traits (also enables entropy)
cargo build --release --features rand
# Python 3.11-3.14
pip install maturin numpy
maturin develop --release
# C/C++ FFI bindings
cargo build --release --features ffi
| Feature | Default | Adds |
|---|---|---|
parallel |
yes | thread pool for batches |
entropy |
no | from_entropy constructors (system entropy source) |
rand |
no | rand_core traits for LustroPrng and LustroXof; enables entropy |
ffi |
no | C/C++ bindings and lustro.h; enables entropy |
python |
no | Python module, built by maturin; enables entropy |
The ffi build generates lustro.h in Cargo's output directory. To also write it into the source tree, set LUSTRO_HEADER_DIR,
for example LUSTRO_HEADER_DIR=include cargo build --release --features ffi.
In the release profiles (panic = "unwind") the C FFI boundary catches internal panics and returns
LUSTRO_ERROR_INTERNAL_PANIC; the Python module raises them as pyo3_runtime.PanicException (a BaseException).
Direct Rust calls panic as usual, for example on a length mismatch. LUSTRO_ERROR_INTERNAL_PANIC is not transactional: after it
the state of the context and of the output buffer is unspecified, so do not retry on the same context.
NumPy is needed only by the functions that take or return arrays (hash*_many*, fill_blocks, blocks, fill_u64);
without it they raise ImportError. The install command above includes NumPy; a package built from the repo also has
it as the numpy extra.
The Python module declares that it requires the GIL (gil_used = true).
On free-threaded CPython builds, importing the module enables the GIL
unless this behavior is explicitly overridden (PYTHON_GIL=0 or -X gil=0).
Running Lustro with the GIL disabled is not supported.
Speed Test
The repo also contains Lustro_Speed_Tester.py. It runs equivalent sections on native Rust, Python and C layers.
# rust target (built on request only, no other build includes it)
cargo build --release --example speed
Tested Platforms
Tested platforms: Windows x86_64 (Python 3.11-3.14) and Linux x86_64 (Python 3.11-3.14). 32-bit: the Rust and C layers were tested on i686-pc-windows-msvc, the Python layer only compiles there. Not tested: macOS, aarch64 and other ARM targets, big-endian targets and free-threaded CPython.
C / C++ Conventions
The C / C++ examples in this README mostly omit status checks for brevity. In real code, check every returned LustroError
(lustro_strerror returns a message).
- Constructors return a
LustroErrorand store the new context in their last argument (out_ctx).*out_ctxisNULLafter a failure and receives the context only when the status isOk; the status tells why. ANULLor misalignedout_ctxis rejected withLUSTRO_ERROR_INVALID_POINTERand nothing is written. The same status is returned, with nothing written, bylustro_hasher_clone,lustro_xof_absorber_clone,lustro_xof_absorber_finalizeand the PRNG/XOF constructors that take a source context whenout_ctxoverlaps that context (details inARCHITECTURE.md). Set the pointer toNULLyourself and check the status. - Freeing: the
lustro_*_freefunctions acceptNULL. - 128-bit values: a stream ID or a path element goes in as two
uint64_t,hifirst.next_u128andpositionreturn 16 bytes, little-endian.lustro_u128_from_parts(hi, lo, out)andlustro_u128_to_parts(in, &hi, &lo)fromlustro.hconvert between the two forms.
Hashing
Hash
Rust
use lustro::hash256;
let digest = hash256(b"hello world");
Python
import lustro
digest = lustro.hash256(b"hello world")
C / C++
#include "lustro.h"
uint8_t digest[LUSTRO_HASH256_LEN];
lustro_hash256((const uint8_t *)"hello world", 11, digest);
hash128 is available in the same way and returns LUSTRO_HASH128_LEN (16) bytes.
Printing and comparing digests (Rust)
Hash128 and Hash256 print as hex of the digest bytes, in the order returned by as_bytes(): 32 and 64 digits.
Width, fill and alignment work as for strings ({digest:>70}, {digest:*^70}), and a precision shortens the
output ({digest:.8}).
use lustro::{hash256, Hash256};
let digest = hash256(b"hello world");
println!("{digest}"); // lowercase hex, 64 digits
println!("{digest:X}"); // uppercase
println!("{digest:#x}"); // with a 0x prefix
let parsed: Hash256 = format!("{digest:#x}").parse()?;
assert_eq!(parsed, digest);
Digests are Hash and Ord, so they work as keys in HashMap and BTreeMap. Ord compares the bytes
lexicographically, not as a number. Conversion to and from [u8; N] is available through From / Into.
In Python a digest is bytes, so use digest.hex().
parse() reads a digest back from hex: 64 digits for Hash256, 32 for Hash128, either case, with an optional
0x or 0X prefix. Spaces are not trimmed. ParseHashError says whether the length or a digit is wrong.
Hash256::first_128() returns the first 16 bytes as a Hash128, the same value hash128 returns for that message.
In Python use bytes.fromhex(s).
Incremental Hash
A message that is not in memory at once, such as a large file, can be hashed in pieces. The length of the whole
message goes into the initial state, so it has to be known before the first byte. Pass it to the hasher; update
then takes the bytes in pieces of any size, and the digest equals hash256 of the whole message.
Rust
use lustro::{hash256, hash256_reader, LustroHasher};
use std::fs::File;
let mut h = LustroHasher::new(11);
h.update(b"hello ")?;
h.update(b"world")?;
assert_eq!(h.finalize()?, hash256(b"hello world"));
// A file, read in 8 KiB pieces.
let file = File::open("data.bin")?;
let len = file.metadata()?.len();
let digest = hash256_reader(len, file)?;
Python
import os
import lustro
h = lustro.LustroHasher(11)
h.update(b"hello ")
h.update(b"world")
assert h.digest() == lustro.hash256(b"hello world")
h = lustro.LustroHasher(os.path.getsize("data.bin"))
with open("data.bin", "rb") as f:
while chunk := f.read(1 << 16):
h.update(chunk)
digest = h.digest()
C / C++
#include "lustro.h"
LustroHasher *h = NULL;
lustro_hasher_new(11, &h);
lustro_hasher_update(h, (const uint8_t *)"hello ", 6);
lustro_hasher_update(h, (const uint8_t *)"world", 5);
uint8_t digest[LUSTRO_HASH256_LEN];
lustro_hasher_finalize256(h, digest);
lustro_hasher_free(h);
- Length: a piece that would go past the declared length is rejected whole (
HashError::TooMuchDatain Rust,ValueErrorin Python,LUSTRO_ERROR_INVALID_LENGTHin C/C++) and nothing of it is absorbed. Asking for the digest early fails the same way (HashError::TooLittleData), and more pieces can follow.remaining()(lustro_hasher_remaining) tells how many bytes are still expected. - Digest: reading it does not end the hasher; every call gives the same value. Its first 16 bytes are
hash128of the message. - Reader:
hash256_reader(len, reader)reads exactlylenbytes and leaves the rest in the reader (pass&mut readerto keep it). A reader that ends early givesErrorKind::UnexpectedEof. - Copy:
clone()in Rust,copy()in Python andlustro_hasher_clonein C/C++ continue independently, for example to hash several messages of the same length that share a prefix.
Batch Hash
Hash many messages in one call. *_many_fixed takes messages of one length stored back to back in a single buffer;
*_many_var takes messages of any length.
Rust
use lustro::{
hash256_many_fixed, hash256_many_fixed_into, hash256_many_var, hash256_many_var_into, Hash256,
};
// Messages of one length, stored back to back.
let data = [0u8; 3 * 16];
let fixed = hash256_many_fixed(&data, 16);
// The same, written into a slice you provide.
let mut out = vec![Hash256::default(); 3];
hash256_many_fixed_into(&data, 16, &mut out);
// Messages of any length.
let messages = [b"a".as_ref(), b"bc".as_ref(), b"".as_ref()];
let digests = hash256_many_var(&messages);
// The same, written into a slice you provide.
hash256_many_var_into(&messages, &mut out);
Python
import numpy as np
import lustro
# Equal-length messages: one message per row of a 2D uint8 array.
rows = np.zeros((3, 16), dtype=np.uint8)
digests = lustro.hash256_many_fixed(rows) # uint8 array, shape (3, 32)
# Messages of any length: a sequence of bytes or byte buffers.
digests = lustro.hash256_many_var([b"a", b"bc", b""]) # shape (3, 32)
# Into an array you provide, see "GIL and Throughput (Python)".
out = np.empty((3, 32), dtype=np.uint8)
lustro.hash256_many_fixed_into(rows, out)
lustro.hash256_many_var_into([b"a", b"bc", b""], out)
# Messages packed in one buffer: message i is data[offsets[i]:offsets[i + 1]].
data = b"abc"
offsets = np.array([0, 1, 3, 3], dtype=np.int64) # three messages: b"a", b"bc", b""
digests = lustro.hash256_many_offsets(data, offsets) # shape (3, 32)
lustro.hash256_many_offsets_into(data, offsets, out)
C / C++
#include "lustro.h"
/* Equal-length messages, stored back to back. */
uint8_t rows[3][16] = {{0}};
uint8_t digests[3][LUSTRO_HASH256_LEN];
lustro_hash256_many_fixed((const uint8_t *)rows, 3, 16, (uint8_t *)digests);
/* Messages of any length. */
const uint8_t *messages[] = {
(const uint8_t *)"a",
(const uint8_t *)"bc",
(const uint8_t *)""
};
uintptr_t lengths[] = {1, 2, 0};
lustro_hash256_many_var(messages, lengths, 3, (uint8_t *)digests);
The names mean the same in Rust, Python and C/C++ for *_many_fixed and *_many_var; *_many_offsets exists in Python only.
hash128_* and lustro_hash128_* work the same way and return 16-byte digests.
AVX2 and the thread pool are used as far as the CPU and the runtime settings allow; the digests do not depend on either.
Streams
PRNG and XOF
LustroPrng generates a stream from a 32-byte seed and a stream ID. LustroXof generates a stream from a message of any length.
Rust
// PRNG
use lustro::{LustroPrng, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let mut rng = LustroPrng::new(&seed, StreamId(0));
let value = rng.next_u64();
// XOF
use lustro::LustroXof;
let mut xof = LustroXof::new(b"hello world");
let value = xof.next_u64();
StreamId wraps a u128, so StreamId(5) works. StreamId::from takes u8, u16, u32, u64 and u128 values;
StreamId::from(5) does not compile, because an untyped literal is an i32.
Python
# PRNG
from lustro import LustroPrng
rng = LustroPrng(bytes(32), 0) # stream_id defaults to 0
value = rng.next_u64()
# XOF
from lustro import LustroXof
xof = LustroXof(b"hello world")
value = xof.next_u64()
C / C++
/* PRNG */
#include <stdio.h>
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
LustroError err = lustro_prng_new(seed, 0, 0, &rng);
if (err != LUSTRO_ERROR_OK) {
fprintf(stderr, "%s\n", lustro_strerror(err));
return 1;
}
uint64_t value;
lustro_prng_next_u64(rng, &value);
lustro_prng_free(rng);
/* XOF */
#include "lustro.h"
LustroXof *xof = NULL;
lustro_xof_new((const uint8_t *)"hello world", 11, &xof);
uint64_t value;
lustro_xof_next_u64(xof, &value);
lustro_xof_free(xof);
Incremental XOF
LustroXofAbsorber is the XOF counterpart of the hasher from Incremental Hash: give it the total length,
feed the message with update, and finalize returns the LustroXof of the whole message. LustroXof::from_reader
does the same from a reader, like hash256_reader.
Rust
use lustro::{LustroXof, LustroXofAbsorber};
use std::fs::File;
let mut a = LustroXofAbsorber::new(11);
a.update(b"hello ")?;
a.update(b"world")?;
let mut xof = a.finalize()?;
assert_eq!(xof.next_u64(), LustroXof::new(b"hello world").next_u64());
// A file, read in 8 KiB pieces.
let file = File::open("data.bin")?;
let len = file.metadata()?.len();
let mut xof = LustroXof::from_reader(len, file)?;
Python
import lustro
a = lustro.LustroXofAbsorber(11)
a.update(b"hello ")
a.update(b"world")
xof = a.finalize()
assert xof.fill(64) == lustro.LustroXof(b"hello world").fill(64)
C / C++
#include "lustro.h"
LustroXofAbsorber *a = NULL;
lustro_xof_absorber_new(11, &a);
lustro_xof_absorber_update(a, (const uint8_t *)"hello ", 6);
lustro_xof_absorber_update(a, (const uint8_t *)"world", 5);
LustroXof *xof = NULL;
lustro_xof_absorber_finalize(a, &xof);
lustro_xof_absorber_free(a);
uint64_t value;
lustro_xof_next_u64(xof, &value);
lustro_xof_free(xof);
- Errors: as for the hasher.
finalizebefore the declared length is reached fails (HashError::TooLittleData,ValueError,LUSTRO_ERROR_INVALID_LENGTHwith*out_ctxset toNULL), and more pieces can follow. - Stream:
finalizedoes not end the absorber; each call returns a new stream at position 0. In C it is an ordinaryLustroXofcontext, freed withlustro_xof_free, also after the absorber is freed.
Random Seed
Rust (--features entropy, also enabled by rand, python and ffi)
use lustro::{LustroPrng, StreamId};
// Err(EntropyError) if the operating system cannot provide entropy.
// Stream 0 of a fresh seed; the seed is not kept.
let mut rng = LustroPrng::from_entropy()?;
// Stream 5 of another fresh seed; unrelated to `rng`.
let mut other = LustroPrng::from_entropy_with_id(StreamId(5))?;
To keep the seed, take it first: Seed256::from_entropy()?, then LustroPrng::new(&seed, StreamId(0)).
Python
from lustro import LustroPrng
rng = LustroPrng.from_entropy() # OSError if the system source is unavailable
other = LustroPrng.from_entropy(stream_id=5) # another fresh seed, unrelated to `rng`
The seed is not kept. To keep it, use LustroPrng(os.urandom(32), 0).
C / C++
lustro_seed_from_entropy fills a 32-byte seed from the system's entropy source;
lustro_prng_from_entropy creates a PRNG directly (stream ID as hi, lo, the seed is not kept):
#include "lustro.h"
LustroPrng *rng = NULL;
LustroError err = lustro_prng_from_entropy(0, 0, &rng);
if (err != LUSTRO_ERROR_OK) {
/* LUSTRO_ERROR_ENTROPY_UNAVAILABLE: the system entropy source is unavailable */
}
/* To keep the seed, take it first */
uint8_t seed[LUSTRO_SEED_LEN];
LustroPrng *kept = NULL;
err = lustro_seed_from_entropy(seed);
if (err == LUSTRO_ERROR_OK) {
err = lustro_prng_new(seed, 0, 0, &kept);
}
lustro_prng_free(kept);
lustro_prng_free(rng);
Typed Reads
Read integers, a full block, or any number of bytes from a single stream.
Rust
use lustro::{LustroPrng, Seed256, StreamId};
use std::io::Read;
let seed = Seed256::from_bytes([0u8; 32]);
let mut rng = LustroPrng::new(&seed, StreamId(0));
let a = rng.next_u32();
let b = rng.next_u64();
let c = rng.next_u128();
let block = rng.next_block(); // [u8; 32]
let mut buf = [0u8; 100];
rng.fill_bytes(&mut buf);
let mut bytes = Vec::new();
rng.by_ref().take(100).read_to_end(&mut bytes).unwrap(); // std::io::Read
rng.advance_bytes(50); // skip 50 bytes
rng.advance_blocks(2); // skip 2 blocks
let position: u128 = rng.position();
Python
from lustro import LustroPrng
rng = LustroPrng(bytes(32), 0)
a = rng.next_u32()
b = rng.next_u64()
c = rng.next_u128()
block = rng.next_block() # bytes, 32
data = rng.fill(100) # bytes, 100
rng.advance_bytes(50) # skip 50 bytes
rng.advance_blocks(2) # skip 2 blocks
position = rng.position() # int
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);
uint32_t a;
uint64_t b;
lustro_prng_next_u32(rng, &a);
lustro_prng_next_u64(rng, &b);
uint8_t c[16]; /* u128, 16 bytes little-endian */
lustro_prng_next_u128(rng, c);
uint8_t block[LUSTRO_BLOCK_LEN];
lustro_prng_next_block(rng, block);
uint8_t data[100];
lustro_prng_fill(rng, data, sizeof(data));
lustro_prng_advance_bytes(rng, 50); /* skip 50 bytes */
lustro_prng_advance_blocks(rng, 2); /* skip 2 blocks */
uint8_t position[16]; /* u128, 16 bytes little-endian */
lustro_prng_position(rng, position);
lustro_prng_free(rng);
LustroXof reads the same way: use a LustroXof in place of rng (Rust and Python), or the lustro_xof_* functions
in C/C++ (lustro_xof_next_u32, lustro_xof_next_u64, lustro_xof_next_u128, lustro_xof_next_block, lustro_xof_fill).
All values are little-endian and taken from the same byte stream: in the examples above a is bytes 0–3, b bytes 4–11
and c bytes 12–27. next_block() always advances to a full new 32-byte block and discards the unread rest of the current one.
advance_bytes(n) skips n bytes and advance_blocks(n) skips n blocks, leaving the stream where fill of n bytes
or n calls of next_block() would. Both cost O(n); there is no jump-ahead. position() returns the bytes taken so far,
and the discarded rest of a block counts as taken. They exist for LustroPrng and LustroXof in Rust, Python and C/C++
(lustro_prng_* and lustro_xof_*). In C/C++ n is a uint64_t, and the position is written as 16 bytes, little-endian;
lustro_u128_to_parts splits it into (hi, lo).
In Rust, LustroPrng and LustroXof also implement std::io::Read: read fills the whole buffer and returns its length,
like fill_bytes. The stream has no end, so read_to_end and read_to_string return an InvalidInput error and read nothing,
and io::copy and bytes() without a limit never stop. Bound the read, for example with take.
Clone
Copy a stream with its exact state. The copy is independent: both continue with the same bytes, and neither affects the other. The clone keeps the position inside the current block as well.
Rust
use lustro::{LustroPrng, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let mut rng = LustroPrng::new(&seed, StreamId(0));
rng.next_u64();
let mut twin = rng.clone();
assert_eq!(rng.next_u64(), twin.next_u64());
Python
from lustro import LustroPrng
rng = LustroPrng(bytes(32), 0)
rng.next_u64()
twin = rng.copy() # copy.copy() and copy.deepcopy() work too
assert rng.next_u64() == twin.next_u64()
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);
LustroPrng *twin = NULL;
lustro_prng_clone(rng, &twin);
lustro_prng_free(twin);
lustro_prng_free(rng);
LustroXof is cloned the same way: xof.clone() (Rust), xof.copy() (Python), lustro_xof_clone(xof, &twin) (C/C++, freed with lustro_xof_free).
Fork
Derive a child stream from the current state and a 128-bit identifier. The parent is not changed. The child depends on the parent's current block, not on the position inside it: forking after 1 byte or after 31 bytes of the same block gives the same child.
Rust
use lustro::{LustroPrng, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let rng = LustroPrng::new(&seed, StreamId(0));
let mut child = rng.fork(StreamId(1));
let value = child.next_u64();
Python
from lustro import LustroPrng
rng = LustroPrng(bytes(32), 0)
child = rng.fork(1)
value = child.next_u64()
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);
LustroPrng *child = NULL;
lustro_prng_fork(rng, 0, 1, &child); /* id as (hi, lo) */
uint64_t value;
lustro_prng_next_u64(child, &value);
lustro_prng_free(child);
lustro_prng_free(rng);
LustroXof forks the same way: xof.fork(StreamId(1)) (Rust), xof.fork(1) (Python), lustro_xof_fork(xof, 0, 1, &child) (C/C++).
The same parent state and identifier always give the same child. The child starts at its own first block and does not inherit the parent's position.
A 32-byte output block is the stream's state at that point, and fork() derives from that state. So once a PRNG
has generated a block, fork(id) equals LustroPrng::new with that block as the seed and the same identifier.
Derive Path
Derive a stream directly from a seed (or message) and a path of identifiers, without exposing the intermediate streams.
Rust
use lustro::{LustroPrng, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let rng = LustroPrng::derive_path(&seed, &[StreamId(12), StreamId(7), StreamId(99)]);
Python
from lustro import LustroPrng
seed = bytes(32)
rng = LustroPrng.derive_path(seed, [12, 7, 99])
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
uint64_t ids_hi[3] = {0, 0, 0};
uint64_t ids_lo[3] = {12, 7, 99};
LustroPrng *rng = NULL;
lustro_prng_derive_path(seed, ids_hi, ids_lo, 3, &rng);
lustro_prng_free(rng);
For an XOF the message takes the place of the seed: LustroXof::derive_path(b"hello world", &[StreamId(12), StreamId(7), StreamId(99)]) (Rust),
LustroXof.derive_path(b"hello world", [12, 7, 99]) (Python) and
lustro_xof_derive_path(message, message_len, ids_hi, ids_lo, 3, &xof) (C/C++).
For a PRNG, derive_path(seed, path) gives the same stream as new with the first
identifier followed by fork for each remaining one, in a single call. For an XOF,
every identifier in path is a fork, since the message already is the root.
path must not be empty: derive_path panics in Rust (try_derive_path returns an error), Python raises ValueError,
and C/C++ returns LUSTRO_ERROR_INVALID_LENGTH. The same holds for the batch versions below.
Batches
Batch PRNG and XOF
Many independent streams in one object, for PRNG and XOF alike.
A PRNG or XOF batch produces output in steps: fill_blocks(out, steps) writes steps blocks for every stream. steps
only sets how much one call produces. The same total split over several calls gives every stream the same blocks.
suggested_steps() returns a steps chosen for speed at the current number of streams; it is only a hint, any steps
gives correct output. The output holds len() * steps blocks, so with many streams check its size before allocating.
Rust
// PRNG
use lustro::{LustroPrngBatch, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let mut prng = LustroPrngBatch::new(&seed, &ids);
let steps = prng.suggested_steps(); // or a fixed number, e.g. 2
let mut out = vec![[0u8; 32]; prng.len() * steps];
prng.fill_blocks(&mut out, steps);
// Or let the library allocate the vector:
let more = prng.blocks(steps); // Vec<[u8; 32]>, prng.len() * steps blocks
// XOF
use lustro::LustroXofBatch;
let messages = [b"msg0".as_ref(), b"msg1".as_ref(), b"msg2".as_ref()];
let mut xof = LustroXofBatch::new(&messages);
let steps = xof.suggested_steps(); // or a fixed number, e.g. 2
let mut out = vec![[0u8; 32]; xof.len() * steps];
xof.fill_blocks(&mut out, steps);
let more = xof.blocks(steps);
Python
# PRNG
import numpy as np
from lustro import LustroPrngBatch
prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])
steps = prng.suggested_steps() # or a fixed number, e.g. 2
out = np.empty((steps, len(prng), 4), dtype=np.uint64) # (steps, n_streams, 4)
prng.fill_blocks(out, steps)
# Or let the library allocate the array:
more = prng.blocks(steps)
# XOF
import numpy as np
from lustro import LustroXofBatch
xof = LustroXofBatch([b"msg0", b"msg1", b"msg2"])
steps = xof.suggested_steps() # or a fixed number, e.g. 2
out = np.empty((steps, len(xof), 4), dtype=np.uint64) # (steps, n_streams, 4)
xof.fill_blocks(out, steps)
more = xof.blocks(steps)
fill_blocks(out, steps) writes one 32-byte block per stream per step, step-major: stream i at step s is block
s * n + i (n = number of streams). In Python that is out[s, i], with out shaped (steps, n_streams, 4);
each block is four uint64 values. blocks(steps) (Rust and Python) returns the same blocks in a newly allocated result:
a Vec<[u8; 32]> in Rust, a new, writable array of that shape in Python. In Python steps can be left out or None, and blocks()
then uses suggested_steps().
In Python, a batch keeps an internal buffer the size of its largest fill_blocks() call made with the default
release_gil=True; release_buffer() frees it earlier. fill_blocks(..., release_gil=False) on little-endian targets
and blocks() do not use that buffer.
C / C++
/* PRNG */
#include <stdlib.h>
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
uint64_t ids_hi[4] = {0};
uint64_t ids_lo[4] = {0, 1, 2, 3};
LustroPrngBatch *prng = NULL;
lustro_prng_batch_new(seed, ids_hi, ids_lo, 4, &prng);
size_t steps = lustro_prng_batch_suggested_steps(prng); /* or a fixed number, e.g. 2 */
size_t out_len = steps * lustro_prng_batch_len(prng) * LUSTRO_BLOCK_LEN;
uint8_t *out = (uint8_t *)malloc(out_len);
lustro_prng_batch_fill_blocks(prng, out, out_len, steps);
free(out);
lustro_prng_batch_free(prng);
/* XOF */
#include <stdlib.h>
#include "lustro.h"
const uint8_t *messages[] = {
(const uint8_t *)"msg0",
(const uint8_t *)"msg1",
(const uint8_t *)"msg2"
};
uintptr_t lengths[] = {4, 4, 4};
LustroXofBatch *xof = NULL;
lustro_xof_batch_new(messages, lengths, 3, &xof);
size_t steps = lustro_xof_batch_suggested_steps(xof); /* or a fixed number, e.g. 2 */
size_t out_len = steps * lustro_xof_batch_len(xof) * LUSTRO_BLOCK_LEN;
uint8_t *out = (uint8_t *)malloc(out_len);
lustro_xof_batch_fill_blocks(xof, out, out_len, steps);
free(out);
lustro_xof_batch_free(xof);
lustro_*_batch_len returns 0 for an empty batch and also for NULL. lustro_*_batch_get_len(ctx, &len) tells the
two apart: it returns LUSTRO_ERROR_INVALID_POINTER for NULL and writes nothing.
LustroPrngBatch can also be created for consecutive stream IDs: LustroPrngBatch::new_range(&seed, StreamId(first), count)
(Rust), LustroPrngBatch.new_range(seed, first, count) (Python) and lustro_prng_batch_new_range(seed, first_hi, first_lo, count, &prng)
(C/C++) use the IDs first, first + 1, ... XOF batches have no new_range.
In C/C++, lustro_batch_block_offset(step, batch_len, lane) from lustro.h gives the byte offset of one block in the output:
(step * batch_len + lane) * LUSTRO_BLOCK_LEN. It expects step < steps and lane < batch_len and does not check its arguments.
/* lane 2 at step 0 of the PRNG output above */
const uint8_t *block = out + lustro_batch_block_offset(0, 4, 2);
Batch into Single Lane
Take one lane out of a batch as a single stream. The stream continues the lane: its next block is the block the batch would write for that lane next. The batch is not changed, and the two evolve independently afterwards.
Rust
use lustro::{LustroPrngBatch, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let mut prng = LustroPrngBatch::new(&seed, &ids);
prng.blocks(2);
// None if the index is out of range.
let mut lane = prng.lane(1).expect("lane in range");
let block = lane.next_block();
Python
from lustro import LustroPrngBatch
prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])
prng.blocks(2)
lane = prng.lane(1) # IndexError if the index is out of range
same = prng[1] # the same lane; prng[-1] is the last one
block = lane.next_block()
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrngBatch *prng = NULL;
lustro_prng_batch_new_range(seed, 0, 0, 4, &prng);
LustroPrng *lane = NULL;
LustroError err = lustro_prng_batch_lane(prng, 1, &lane); /* LUSTRO_ERROR_INVALID_ARGUMENT if index >= len */
uint8_t block[LUSTRO_BLOCK_LEN];
lustro_prng_next_block(lane, block);
lustro_prng_free(lane); /* a single stream, not a batch */
lustro_prng_batch_free(prng);
LustroXofBatch has lane too: xof.lane(1) (Rust, Python) and lustro_xof_batch_lane(xof, 1, &lane) (C/C++, the lane is a LustroXof, freed with lustro_xof_free).
The result is an ordinary single stream (LustroPrng or LustroXof), so it supports typed reads, fork and snapshots.
In Python, batch[i] does what batch.lane(i) does, and a negative i counts from the end (lane(i) itself rejects it).
for lane in batch goes through all lanes in order, as copies of the lane states from the start of the loop. Slices are not supported.
Single Streams into Batch
Build a batch from single streams, one lane per stream, in order. Each lane continues its stream at the next block, so
batch.lane(i) gives back a copy of stream i. The streams are not changed.
A lane holds whole blocks only, so every stream must be at a block boundary: freshly created, or read in whole 32-byte blocks.
A stream in the middle of a block is rejected rather than moved on, so no bytes are dropped. Read or skip the rest of the
block first: a stream whose position() is not a multiple of 32 is inside a block, and 32 - position() % 32 more bytes
bring it to the next boundary.
Rust
use lustro::{LustroPrng, LustroPrngBatch, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let a = LustroPrng::new(&seed, StreamId(0));
let mut b = LustroPrng::new(&seed, StreamId(1));
b.next_u64(); // b is now inside a block
// Err(BatchError::PartialBlock { index: 1 }); from_streams would panic.
assert!(LustroPrngBatch::try_from_streams(&[a.clone(), b.clone()]).is_err());
b.advance_bytes(24); // the rest of the block
let batch = LustroPrngBatch::from_streams(&[a, b]);
Python
from lustro import LustroPrng, LustroPrngBatch
a = LustroPrng(bytes(32), 0)
b = LustroPrng(bytes(32), 1)
b.next_u64() # b is now inside a block; from_streams would raise ValueError
b.advance_bytes(24) # the rest of the block
batch = LustroPrngBatch.from_streams([a, b])
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *a = NULL;
LustroPrng *b = NULL;
lustro_prng_new(seed, 0, 0, &a);
lustro_prng_new(seed, 0, 1, &b);
const LustroPrng *streams[] = {a, b};
LustroPrngBatch *batch = NULL;
LustroError err = lustro_prng_batch_from_streams(streams, 2, &batch); /* LUSTRO_ERROR_INVALID_ARGUMENT if a stream is inside a block */
lustro_prng_batch_free(batch);
lustro_prng_free(b);
lustro_prng_free(a);
LustroXof streams go into a LustroXofBatch the same way: LustroXofBatch::from_streams(&[a, b]) (Rust), LustroXofBatch.from_streams([a, b]) (Python)
and lustro_xof_batch_from_streams(streams, 2, &batch) with an array of const LustroXof * (C/C++).
The streams can come from different seeds or messages, and the same stream can appear more than once. A stream inside
a block gives BatchError::PartialBlock { index } from try_from_streams in Rust (index is the first such stream;
from_streams panics), ValueError in Python and LUSTRO_ERROR_INVALID_ARGUMENT in C/C++. In C/C++ the streams stay
owned by the caller and are freed separately. In Python the source can also be a batch,
which gives a copy of it.
Fork into a Batch
Derive many children of one stream in a single call. fork_batch(ids) returns a batch whose lane i is fork(ids[i]),
and fork_batch_range(first, count) uses the IDs first, first + 1, ... fork_path(path) returns fork(path[0]).fork(path[1])...
without the intermediate streams. All three start from the current state, like fork, and do not change the parent.
Rust
use lustro::{LustroPrng, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let rng = LustroPrng::new(&seed, StreamId(0));
let children = rng.fork_batch(&[StreamId(10), StreamId(20)]); // LustroPrngBatch, 2 lanes
let numbered = rng.fork_batch_range(StreamId(100), 8); // ids 100 to 107
let leaf = rng.fork_path(&[StreamId(12), StreamId(7)]); // rng.fork(12).fork(7)
Python
from lustro import LustroPrng
rng = LustroPrng(bytes(32), 0)
children = rng.fork_batch([10, 20]) # LustroPrngBatch, 2 lanes
numbered = rng.fork_batch_range(100, 8) # ids 100 to 107
leaf = rng.fork_path([12, 7]) # rng.fork(12).fork(7)
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);
uint64_t ids_hi[2] = {0, 0};
uint64_t ids_lo[2] = {10, 20};
LustroPrngBatch *children = NULL;
lustro_prng_fork_batch(rng, ids_hi, ids_lo, 2, &children);
LustroPrngBatch *numbered = NULL;
lustro_prng_fork_batch_range(rng, 0, 100, 8, &numbered); /* first id as (hi, lo), then count */
uint64_t path_hi[2] = {0, 0};
uint64_t path_lo[2] = {12, 7};
LustroPrng *leaf = NULL;
lustro_prng_fork_path(rng, path_hi, path_lo, 2, &leaf);
lustro_prng_free(leaf);
lustro_prng_batch_free(numbered);
lustro_prng_batch_free(children);
lustro_prng_free(rng);
LustroXof has the same methods (lustro_xof_fork_batch, lustro_xof_fork_batch_range and lustro_xof_fork_path
in C/C++). The first two return a LustroXofBatch, fork_path a LustroXof.
For a stream that has not produced output yet, fork_path gives what derive_path gives: LustroPrng::new(&seed, StreamId(5)).fork_path(path) is derive_path(&seed, [5] + path). It is for the cases derive_path does not cover:
a stream from from_entropy(), a lane of a batch, a restored snapshot, or a stream that has already produced output.
path must not be empty, with the same errors as derive_path.
Batch Clone
Copy a whole batch with the exact state of every lane. The copy is independent.
Rust
use lustro::{LustroPrngBatch, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let mut prng = LustroPrngBatch::new(&seed, &ids);
prng.blocks(2);
let twin = prng.clone();
Python
from lustro import LustroPrngBatch
prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])
prng.blocks(2)
twin = prng.copy()
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrngBatch *prng = NULL;
lustro_prng_batch_new_range(seed, 0, 0, 4, &prng);
LustroPrngBatch *twin = NULL;
LustroError err = lustro_prng_batch_clone(prng, &twin);
lustro_prng_batch_free(twin);
lustro_prng_batch_free(prng);
LustroXofBatch is cloned the same way: xof.clone() (Rust), xof.copy() (Python), lustro_xof_batch_clone(xof, &twin) (C/C++).
Batch Fork
Create multiple child streams. The parent is not changed.
Rust
use lustro::{LustroPrngBatch, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let prng = LustroPrngBatch::new(&seed, &ids);
let one_each = prng.fork(&[StreamId(10), StreamId(11), StreamId(12), StreamId(13)]);
let children = prng.fork_range(StreamId(100));
let many_children = prng.fork_many(&[StreamId(100), StreamId(200)]);
Python
from lustro import LustroPrngBatch
prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])
one_each = prng.fork([10, 11, 12, 13])
children = prng.fork_range(100)
many_children = prng.fork_many([100, 200])
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrngBatch *prng = NULL;
lustro_prng_batch_new_range(seed, 0, 0, 4, &prng);
uint64_t ids_hi[4] = {0, 0, 0, 0};
uint64_t ids_lo[4] = {10, 11, 12, 13};
LustroPrngBatch *one_each = NULL;
LustroError err = lustro_prng_batch_fork(prng, ids_hi, ids_lo, 4, &one_each);
LustroPrngBatch *children = NULL;
lustro_prng_batch_fork_range(prng, 0, 100, &children);
uint64_t k_hi[2] = {0, 0};
uint64_t k_lo[2] = {100, 200};
LustroPrngBatch *many_children = NULL;
lustro_prng_batch_fork_many(prng, k_hi, k_lo, 2, &many_children);
lustro_prng_batch_free(many_children);
lustro_prng_batch_free(children);
lustro_prng_batch_free(one_each);
lustro_prng_batch_free(prng);
LustroXofBatch has the same methods: fork, fork_range and fork_many (lustro_xof_batch_fork, lustro_xof_batch_fork_range and
lustro_xof_batch_fork_many in C/C++).
| Method | ids |
Result | Child at index |
|---|---|---|---|
fork(ids) |
exactly batch.len() IDs |
batch.len() lanes |
i: child of lane i with ids[i] |
fork_range(first) |
none | batch.len() lanes |
i: child of lane i with ID first + i |
fork_many(ids) |
any number k |
batch.len() * k lanes |
i * k + j: child j of lane i with ids[j] |
If ids of fork does not have exactly batch.len() elements, fork panics and try_fork returns
BatchError::LengthMismatch in Rust; Python raises ValueError and C/C++ returns LUSTRO_ERROR_INVALID_LENGTH.
Each child starts at step 0.
Batch Derive Path
Derive a batch directly from a shared path, one lane per root (PRNG) or per message (XOF).
Rust
use lustro::{LustroPrngBatch, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let roots = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let path = [StreamId(12), StreamId(7)];
let batch = LustroPrngBatch::derive_path(&seed, &roots, &path);
Python
from lustro import LustroPrngBatch
seed = bytes(32)
batch = LustroPrngBatch.derive_path(seed, [0, 1, 2, 3], [12, 7])
C / C++
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
uint64_t roots_hi[4] = {0, 0, 0, 0};
uint64_t roots_lo[4] = {0, 1, 2, 3};
uint64_t path_hi[2] = {0, 0};
uint64_t path_lo[2] = {12, 7};
LustroPrngBatch *batch = NULL;
lustro_prng_batch_derive_path(
seed, roots_hi, roots_lo, 4, path_hi, path_lo, 2, &batch
);
lustro_prng_batch_free(batch);
An XOF batch takes the messages instead of the seed and the roots: LustroXofBatch::derive_path(&messages, &path) (Rust),
LustroXofBatch.derive_path(messages, path) (Python) and
lustro_xof_batch_derive_path(messages, lengths, 3, path_hi, path_lo, 2, &batch) (C/C++).
For a PRNG batch, derive_path(seed, roots, path) derives a canonical batch: lane i equals
LustroPrng::derive_path(seed, [roots[i]] + path), independent of any other
stream's state. For an XOF batch, lane i equals LustroXof::derive_path(messages[i], path).
In Python the messages can be any sequence of bytes or byte buffers. In Rust, try_derive_path returns
DerivePathError::EmptyPath for an empty path instead of panicking.
Allocation Failures
Batch functions reserve memory for the lanes, the output and the buffer of a batch snapshot. If a reservation fails:
| Plain function | |
|---|---|
| Rust | panics (clone() aborts the process); the try_ variant of the same name returns an error instead |
| Python | MemoryError |
| C/C++ | LUSTRO_ERROR_OUT_OF_MEMORY, except from lustro_*_batch_export_snapshot, which writes into your buffer and allocates nothing |
The Rust try_ variants of LustroPrngBatch and LustroXofBatch (try_new_range for LustroPrngBatch only):
| Variants | Error |
|---|---|
try_new, try_new_range, try_from_streams, try_fork, try_fork_range, try_clone, try_export_snapshot |
BatchError::Reserve |
try_fork_batch, try_fork_batch_range of LustroPrng and LustroXof |
BatchError::Reserve |
try_to_le_bytes of LustroPrngBatchSnapshot and LustroXofBatchSnapshot |
BatchError::Reserve |
try_hash256_many_var, try_hash128_many_var |
BatchError::Reserve |
try_hash256_many_fixed, try_hash128_many_fixed |
BatchError::Reserve; BatchError::InvalidMessageLen for a message_len of 0 or a data.len() that is not a multiple of it (the plain versions panic then) |
try_blocks, try_fork_many |
BatchError::Reserve; BatchError::SizeOverflow when len() * steps or len() * ids.len() overflows usize (the plain versions panic then too) |
try_derive_path |
DerivePathError::Reserve |
try_from_le_bytes of the batch snapshots (also through TryFrom<&[u8]>) |
SnapshotError::Reserve |
State and Runtime
Snapshot
Save and restore the exact stream position.
Rust
use lustro::types::LustroPrngSnapshot;
use lustro::{LustroPrng, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let mut rng = LustroPrng::new(&seed, StreamId(0));
let snapshot = rng.export_snapshot();
let bytes = snapshot.to_le_bytes();
let snapshot =
LustroPrngSnapshot::from_le_bytes(&bytes).expect("valid snapshot");
let mut restored = LustroPrng::import_snapshot(snapshot);
Python
from lustro import LustroPrng
rng = LustroPrng(bytes(32), 0)
snapshot = rng.export_snapshot()
restored = LustroPrng.import_snapshot(snapshot)
C / C++
#include <stdio.h>
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrng *rng = NULL;
lustro_prng_new(seed, 0, 0, &rng);
uint8_t snapshot[LUSTRO_SNAPSHOT_LEN];
lustro_prng_export_snapshot(rng, snapshot, LUSTRO_SNAPSHOT_LEN);
LustroPrng *restored = NULL;
LustroError err = lustro_prng_import_snapshot(snapshot, LUSTRO_SNAPSHOT_LEN, &restored);
if (err != LUSTRO_ERROR_OK) {
fprintf(stderr, "%s\n", lustro_strerror(err));
}
lustro_prng_free(restored);
lustro_prng_free(rng);
LustroXof works the same way: export_snapshot(), LustroXofSnapshot::from_le_bytes (lustro::types) and LustroXof::import_snapshot in Rust;
export_snapshot() and LustroXof.import_snapshot in Python; lustro_xof_export_snapshot and lustro_xof_import_snapshot
with LUSTRO_SNAPSHOT_LEN in C/C++.
A snapshot holds the full stream state, so keep it as private as the seed (for an XOF, the message). Importing
malformed bytes fails:
SnapshotError from from_le_bytes (Rust), lustro.SnapshotError, a ValueError (Python), or
LUSTRO_ERROR_UNSUPPORTED_VERSION, _INVALID_KIND, _INVALID_RESERVED, _INVALID_CURSOR or _INVALID_LENGTH
(C/C++). A PRNG snapshot cannot be imported as an XOF or the other way round, and a single-stream snapshot
cannot be imported as a batch.
The reserved bytes of a snapshot are zero: export writes zeros, and import rejects anything else (InvalidReserved).
Apart from the header, the cursor and these bytes, a snapshot has no integrity check: a changed s0, s1 or step
imports without error and continues another stream. If stored snapshots can be damaged, add your own checksum.
To find out which type exported some bytes, call SnapshotKind::detect(bytes) (Rust), lustro.snapshot_kind(bytes)
(Python: "prng", "xof", "prng_batch" or "xof_batch") or lustro_snapshot_kind(bytes, len, &kind)
(C/C++: LUSTRO_SNAPSHOT_KIND_*). It reads only the version and kind bytes, so a result does not mean the import will succeed.
Batch Snapshot
Save and restore every lane of a batch at once. The snapshot has a dynamic size: a header plus one entry per lane.
Rust
use lustro::types::LustroPrngBatchSnapshot;
use lustro::{LustroPrngBatch, Seed256, StreamId};
let seed = Seed256::from_bytes([0u8; 32]);
let ids = [StreamId(0), StreamId(1), StreamId(2), StreamId(3)];
let mut prng = LustroPrngBatch::new(&seed, &ids);
prng.blocks(2);
let snapshot = prng.export_snapshot();
let bytes = snapshot.to_le_bytes(); // Vec<u8>
let snapshot =
LustroPrngBatchSnapshot::from_le_bytes(&bytes).expect("valid snapshot");
let mut restored = LustroPrngBatch::import_snapshot(snapshot);
Python
from lustro import LustroPrngBatch
prng = LustroPrngBatch(bytes(32), [0, 1, 2, 3])
prng.blocks(2)
snapshot = prng.export_snapshot() # bytes
restored = LustroPrngBatch.import_snapshot(snapshot)
C / C++
#include <stdio.h>
#include <stdlib.h>
#include "lustro.h"
uint8_t seed[LUSTRO_SEED_LEN] = {0};
LustroPrngBatch *prng = NULL;
lustro_prng_batch_new_range(seed, 0, 0, 4, &prng);
size_t size = lustro_prng_batch_snapshot_size(prng);
uint8_t *snapshot = (uint8_t *)malloc(size);
lustro_prng_batch_export_snapshot(prng, snapshot, size);
LustroPrngBatch *restored = NULL;
LustroError err = lustro_prng_batch_import_snapshot(snapshot, size, &restored);
if (err != LUSTRO_ERROR_OK) {
fprintf(stderr, "%s\n", lustro_strerror(err));
}
lustro_prng_batch_free(restored);
free(snapshot);
lustro_prng_batch_free(prng);
LustroXofBatch works the same way: LustroXofBatchSnapshot (lustro::types) in Rust, LustroXofBatch.import_snapshot in Python,
lustro_xof_batch_snapshot_size, lustro_xof_batch_export_snapshot and lustro_xof_batch_import_snapshot in C/C++.
The snapshot size is LUSTRO_BATCH_SNAPSHOT_HEADER_LEN + n * LUSTRO_BATCH_SNAPSHOT_LANE_LEN for n streams.
In C/C++ call *_batch_snapshot_size(ctx) before export; out_len must equal it.
Runtime Settings
Thread count and SIMD backend follow environment variables by default. configure sets them from code, best before
first use (see below). They apply to hash batches as well as to PRNG and XOF batches.
| Variable | configure |
Effect | Values |
|---|---|---|---|
LUSTRO_NUM_THREADS |
threads |
Pool size, capped at the CPU count; 1 disables the pool |
A positive number. Anything else (abc, 0, -1) is ignored and the pool gets the full CPU count |
LUSTRO_DISABLE_MT |
threads=1 |
Never use the thread pool | Flag |
LUSTRO_DISABLE_SIMD |
allow_simd=False |
Scalar backend | Flag |
A flag is set by 1, true, on or yes, and not set by 0, false, off, no or an empty value (case is ignored).
Any other value sets it.
Rust
use lustro::{configure, runtime_info, RuntimeOptions};
use std::num::NonZeroUsize;
let options = RuntimeOptions::new()
.threads(NonZeroUsize::new(4).unwrap())
.allow_simd(true);
configure(options)?;
// Read the settings.
let info = runtime_info();
Python
import lustro
lustro.configure(threads=4, allow_simd=True)
info = lustro.runtime_info()
print(info["backend"], info["threads"], info["fixed"])
C / C++
#include <stdio.h>
#include "lustro.h"
LustroError err = lustro_configure(4, LUSTRO_SIMD_AUTO);
if (err != LUSTRO_ERROR_OK) {
fprintf(stderr, "%s\n", lustro_strerror(err));
}
Arguments you omit (None in Python, 0 and LUSTRO_SIMD_KEEP in C/C++) still follow the environment; the ones you
pass override it. In C/C++, simd is LUSTRO_SIMD_KEEP, LUSTRO_SIMD_AUTO (allow_simd=True) or LUSTRO_SIMD_SCALAR
(allow_simd=False).
The settings are fixed once, by whichever of these comes first:
configure;- a hash batch call;
- batch
fill_blocksorblocks; - creating an XOF batch from at least one message (
LustroXofBatch::new,LustroXofBatch(...),lustro_xof_batch_new).
These do not fix them: runtime_info(), suggested_steps() and, in C/C++, lustro_runtime_threads() and
lustro_runtime_backend(). Two edge cases:
- A batch call that fails once it has started the hash (for example
hash*_many_var_intowithrelease_gil=Falseon a big list that has a bad item) has fixed them. - In C/C++ the
*_many_*functions withn == 0and*_batch_fill_blockswithn == 0orsteps == 0return before the dispatcher and fix nothing. In Rust and Python a hash batch call with zero messages fixes them.
The settings belong to the whole process. An application calls configure early, before any batch; a library that uses
Lustro should not call it and can read the settings with runtime_info().
A later configure succeeds if it asks for the same effect as the fixed settings, and fails otherwise:
ConfigError::AlreadyConfigured (Rust), lustro.ConfigError, a subclass of RuntimeError (Python),
LUSTRO_ERROR_ALREADY_CONFIGURED (C/C++). Same effect means the same SIMD choice and, with the parallel feature,
the same pool size. For the pool size, threads above the CPU count counts as the CPU count, and threads=1 equals
LUSTRO_DISABLE_MT. The SIMD choice is compared as given: on a CPU without AVX2, allow_simd=True and False still differ.
When calls race, one wins and the rest are compared with it. Bad arguments change nothing: ValueError for
threads < 1 (Python), LUSTRO_ERROR_INVALID_ARGUMENT for an unknown simd (C/C++).
runtime_info() (Rust, Python, and lustro_runtime_info() in C/C++) returns the report without starting the pool. Before
the settings are fixed it reads the environment as it is now (fixed is false). The report may be out of date by the time it returns.
| Field | Meaning |
|---|---|
backend |
"scalar" or "avx2", as used for the largest batches |
threads |
configured pool size, not running workers; 0 = batches run on the calling thread (also without parallel) |
fixed |
the settings are fixed; false before the first configure or batch call |
parallel_built |
built with the parallel feature |
simd_available |
the CPU has AVX2; backend shows whether it is used |
pool_usable |
the pool exists and this process can use it. False before the first parallel batch, after a failed pool build, and in a child forked after its parent started the pool |
logical_cpus |
logical CPUs at the call; the cap for threads |
In C/C++, lustro_runtime_info(&info, sizeof info) fills a LustroRuntimeInfo and returns LUSTRO_ERROR_OK, LUSTRO_ERROR_INVALID_POINTER for a NULL info, or LUSTRO_ERROR_INVALID_LENGTH if the size is too small. The struct has the fields above; the flags are 0 or 1 and backend is a static string (do not free it).
LustroRuntimeInfo info;
if (lustro_runtime_info(&info, sizeof info) == LUSTRO_ERROR_OK) {
printf("%s, %zu threads, fixed=%d\n", info.backend, (size_t)info.threads, info.fixed);
}
Language Notes
Python Notes
Single streams (LustroPrng, LustroXof) can also return or fill larger amounts of data:
from lustro import LustroPrng
rng = LustroPrng(bytes(32), 0)
data = rng.fill(1000) # bytes
buf = bytearray(1000)
rng.fill_into(buf) # bytearray, writable memoryview or NumPy uint8 array
words = rng.fill_u64(1000) # NumPy uint64 array of shape (1000,)
data = rng.read(1000) # same as fill(1000)
n = rng.readinto(buf) # n == 1000, any writable C-contiguous buffer
fill_u64(n) returns a new, writable array with the values that n calls of next_u64() would give.
read(size) is the same as fill(size); there is no read() without a size, because the stream has no end.
readinto(buf) accepts any writable, C-contiguous buffer whatever its item type (for example array('I')), writes the
stream as raw bytes and returns the number of bytes written; fill_into(buf) takes byte buffers only. A buffer that holds
Python objects (NumPy dtype=object) raises TypeError. Other Python threads cannot change buf while the stream writes
into it; native code that writes without the GIL is not covered.
Seeds, messages, hash input and snapshots are given as bytes. bytearray, memoryview, array('B') and NumPy uint8
arrays are accepted too (C-contiguous with byte items, TypeError otherwise) and are copied. hash*_many_var and
LustroXofBatch take any sequence of such objects, but not a single string or buffer.
hash*_many_offsets and hash*_many_offsets_into hash messages packed in one buffer: message i is
data[offsets[i]:offsets[i + 1]]. offsets is a 1D NumPy array of dtype int64, uint64, int32 or uint32 with one
entry more than there are messages, non-negative, non-decreasing, the last at most len(data) (ValueError for bad
entries, TypeError for another dtype or dimension).
Stream IDs and path elements are Python ints from 0 to 2**128 - 1 (OverflowError otherwise). A list of them can be any
sequence, for example a tuple, a range or a NumPy integer array.
LustroPrng, LustroXof, LustroPrngBatch and LustroXofBatch can be pickled, for example to pass them to a worker
process (LustroHasher and LustroXofAbsorber cannot). The restored object is a copy that continues
from the same position, not a new stream: workers that need independent streams should use fork() or different stream IDs.
Do not load pickles from untrusted sources; for long-term storage use export_snapshot().
An instance must not be used by several threads at the same time (an overlapping call raises RuntimeError). Use copy(),
fork() or separate instances instead. The installed version is available as lustro.__version__, and the package ships
type stubs (lustro.pyi, py.typed).
GIL and Throughput (Python)
Whether a call releases the GIL depends on what it does:
| Calls | GIL |
|---|---|
hash256, hash128, fill, read, fill_u64, fill_into, readinto, advance_bytes (for advance_blocks, 128 blocks), LustroHasher.update, LustroXofAbsorber.update, LustroXof(message), LustroXof.derive_path |
released from 4096 bytes, held below that. The copy into buf in fill_into and readinto holds it |
hash*_many_*, fill_blocks |
released; release_gil=False keeps it (for fill_blocks on little-endian targets only). hash256_many_fixed and hash128_many_fixed have no such argument, their _into variants do |
blocks, LustroXofBatch(...), LustroXofBatch.derive_path |
released while they work |
the other batch calls that build or copy lanes: constructors, new_range, fork*, from_streams, copy, iter(batch), snapshots, LustroPrngBatch.derive_path |
held; their time grows with the lane count |
A pending Ctrl-C during advance_bytes or advance_blocks is raised between chunks of 2^20 blocks and leaves the stream unchanged.
By default (release_gil=True) other Python threads run during the call. Input that is not bytes is copied before the GIL
is released (hash*_many_fixed* copy their rows), *_into calls compute the digests in a scratch buffer and copy them into
out once the GIL is back, and calls that return a new array write the digests straight into it. On large batches the copies
cost time.
For many messages of different lengths prefer hash*_many_offsets and hash*_many_offsets_into over hash*_many_var:
no Python object is read or counted per message, in both modes.
With release_gil=False the GIL stays held and other Python threads wait. Where the input allows, data is read in place and
digests are written straight into out. The thread pool and AVX2 are used in both modes.
out = np.empty((n, 32), dtype=np.uint8)
lustro.hash256_many_fixed_into(rows, out, release_gil=False)
lustro.hash256_many_var_into(messages, out, release_gil=False)
steps = prng.suggested_steps()
blocks = np.empty((steps, len(prng), 4), dtype=np.uint64)
prng.fill_blocks(blocks, steps, release_gil=False)
Not every input is read in place: for hash*_many_var*, an exact list or tuple of exact bytes is the fastest input,
especially for many small messages. out must not share memory with the input (TypeError or ValueError, depending on
how it is shared). If a big exact list or tuple (4096 items or more) given to hash*_many_var_into fails part-way,
out may be partly written.
Each thread keeps up to four buffers between hash batch calls (copied rows, digests of *_into, copied offsets, the table
of message slices), up to 32 MiB each and 128 MiB in all until the thread exits. The buffers are not cleared, so bytes of the
last calls stay in them.
rand_core Integration (Rust)
With --features rand, LustroPrng and LustroXof implement the rand_core 0.10
traits TryRng and Rng (infallible, Error = Infallible). LustroPrng also
implements SeedableRng. LustroXof does not: it is built from a message of any
length.
[dependencies]
lustro_api = { version = "<version>", features = ["rand"] }
rand_core = "0.10"
The crate is published as `lustro_api`; its library is imported as `lustro`.
use lustro::{LustroPrng, LustroXof};
use rand_core::{Rng, SeedableRng};
fn draw(rng: &mut impl Rng) -> u64 {
rng.next_u64()
}
// PRNG from a 32-byte seed (stream 0).
let mut prng = LustroPrng::from_seed([7u8; 32]);
let a = draw(&mut prng);
// Random seed (needs the system entropy source).
let mut random = LustroPrng::from_entropy()?;
let mut buf = [0u8; 100];
random.fill_bytes(&mut buf);
// XOF over a message of any length.
let mut xof = LustroXof::new(b"message");
let b = draw(&mut xof);
- Same stream:
Rngreturns the same bytes, from the same position, as the inherentnext_u32,next_u64andfill_bytes. - Seeding:
from_seedstarts at stream 0.rand_core0.10 has nofrom_entropy; useLustroPrng::from_entropy()?. fork: on a concreteLustroPrng,prng.fork(id)is the inherentLustroPrng::fork(StreamId). The trait methodSeedableRng::fork()runs in code generic overSeedableRngor when called asSeedableRng::fork(&mut prng); it seeds a new generator from the next 32 bytes ofprng.- Clone and
derive_path: both exist onLustroPrngandLustroXofand work without therandfeature. A clone repeats the same bytes, so do not use it to get a second independent stream; useforkorderive_pathfor that.derive_pathis not a trait method, so generic code overRngcannot call it;cloneneeds aClonebound. - Quality:
CryptoRngis not implemented.
Reference
Size Constants
| Meaning | Bytes | C / C++ (lustro.h) |
Rust (lustro::types) |
Python (lustro) |
|---|---|---|---|---|
| Seed | 32 | LUSTRO_SEED_LEN |
LUSTRO_SEED_LEN |
SEED_LEN |
| Output block | 32 | LUSTRO_BLOCK_LEN |
LUSTRO_BLOCK_LEN |
BLOCK_LEN |
hash128 digest |
16 | LUSTRO_HASH128_LEN |
LUSTRO_HASH128_LEN |
HASH128_LEN |
hash256 digest |
32 | LUSTRO_HASH256_LEN |
LUSTRO_HASH256_LEN |
HASH256_LEN |
| Single snapshot | 56 | LUSTRO_SNAPSHOT_LEN |
LUSTRO_SNAPSHOT_LEN |
SNAPSHOT_LEN |
| Batch snapshot header | 16 | LUSTRO_BATCH_SNAPSHOT_HEADER_LEN |
LUSTRO_BATCH_SNAPSHOT_HEADER_LEN |
BATCH_SNAPSHOT_HEADER_LEN |
| Batch snapshot, per stream | 48 | LUSTRO_BATCH_SNAPSHOT_LANE_LEN |
LUSTRO_BATCH_SNAPSHOT_LANE_LEN |
BATCH_SNAPSHOT_LANE_LEN |
For full API details please see ARCHITECTURE.md.
Metadata
Release files for lustro-api 1.0.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| lustro_api-1.0.1.tar.gz | 213.5 kB | With provenance | Details for lustro_api-1.0.1.tar.gz |
Built distributions (wheels)
| File | Actions Reset | |||
|---|---|---|---|---|
| lustro_api-1.0.1-cp311-abi3-win_amd64.whl | CPython 3.11 | abi3 | Windows x86-64 | Details for lustro_api-1.0.1-cp311-abi3-win_amd64.whl |
| lustro_api-1.0.1-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl | CPython 3.11 | abi3 | Linux glibc 2.17+ x86-64 | Details for lustro_api-1.0.1-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
| lustro_api-1.0.1-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl | CPython 3.11 | abi3 | Linux glibc 2.17+ ARM64 | Details for lustro_api-1.0.1-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl |
| lustro_api-1.0.1-cp311-abi3-macosx_11_0_arm64.whl | CPython 3.11 | abi3 | macOS 11.0+ ARM64 | Details for lustro_api-1.0.1-cp311-abi3-macosx_11_0_arm64.whl |
| lustro_api-1.0.1-cp311-abi3-macosx_10_12_x86_64.whl | CPython 3.11 | abi3 | macOS 10.12+ x86-64 | Details for lustro_api-1.0.1-cp311-abi3-macosx_10_12_x86_64.whl |
Total release size: 2.4 MB
Release files / lustro_api-1.0.1.tar.gz
| Download URL | lustro_api-1.0.1.tar.gz |
|---|---|
| Size | 213.5 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3b9e450673a60d15abb217dbc70d6d566c423c320009a7ab2529d9b787b73f09
|
|
BLAKE2b-256 checksum How to use checksums |
931e1acb698be5e52fbf9b4a9f0f17e07c33ea919d800aa04a963d95cf61655b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.
Transparency logRelease files / lustro_api-1.0.1-cp311-abi3-win_amd64.whl
| Download URL | lustro_api-1.0.1-cp311-abi3-win_amd64.whl |
|---|---|
| Size | 425.6 kB |
| Tags | CPython 3.11 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
904da6ce5a72a273bb914e39b670e39740dfcf28c062bdb0bd6b85ad2ad0a3c3
|
|
BLAKE2b-256 checksum How to use checksums |
0f96eff525b22a6ff983dcd65af65a316e2d9a550b4efbd6d6d5b5930b5e1f1e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.
Transparency logRelease files / lustro_api-1.0.1-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
| Download URL | lustro_api-1.0.1-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
|---|---|
| Size | 486.0 kB |
| Tags | CPython 3.11 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
d00efe267efca3a45b1d9ea6131684201d2057f11ef3df4e465695923c1aff91
|
|
BLAKE2b-256 checksum How to use checksums |
b01054682aafe6ef8d707a7f3221aef192d63f190819f578c8b9250f3a0ea9da
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.
Transparency logRelease files / lustro_api-1.0.1-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl
| Download URL | lustro_api-1.0.1-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl |
|---|---|
| Size | 433.4 kB |
| Tags | CPython 3.11 Linux glibc 2.17+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
3f584aae0abc39de1fada78e19d22f0a7a823fad4a39a1bb91b2ac3a3ba9276e
|
|
BLAKE2b-256 checksum How to use checksums |
5a92eaca6e2c3bc35fd8d3a157e4295bc50531663a1a9c78710c5f4ead7d885b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.
Transparency logRelease files / lustro_api-1.0.1-cp311-abi3-macosx_11_0_arm64.whl
| Download URL | lustro_api-1.0.1-cp311-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 413.6 kB |
| Tags | CPython 3.11 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
76a20e2e76c54f207282c014e2731a894eb5558311455d31836345f1cd928e2e
|
|
BLAKE2b-256 checksum How to use checksums |
741216998445c3fca7f8e0f49a0e98a7417f543439a69c4bb6f9109428f0b65a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.
Transparency logRelease files / lustro_api-1.0.1-cp311-abi3-macosx_10_12_x86_64.whl
| Download URL | lustro_api-1.0.1-cp311-abi3-macosx_10_12_x86_64.whl |
|---|---|
| Size | 460.1 kB |
| Tags | CPython 3.11 abi3 macOS 10.12+ x86-64 |
|
SHA-256 checksum How to use checksums |
e293fd1f787c6f2c4ef749c895c6197908c28b792ba69c86e22748bc9a1c93a1
|
|
BLAKE2b-256 checksum How to use checksums |
8876b9c934d3e43602fff7de108cac7c933545442a12ef7c40b3bf4d1961735c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.
Transparency logSecurity for lustro-api 1.0.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 6 files in this release have been uploaded with provenance metadata.
Attestation for lustro_api-1.0.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 14, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.