nominal-streaming Python Bindings
nominal-streaming is a thin python wrapper around the existing nominal-streaming rust crate.
Usage semantics remain largely the same, but with some slight alterations to allow for a more pythonic interface.
Requires Python 3.11 or newer.
The library aims to balance three concerns:
- Data should exist in-memory only for a limited, configurable amount of time before it's sent to Core.
- Writes should fall back to disk if there are network failures.
- Backpressure should be applied to incoming requests when network throughput is saturated.
This library streams data to Nominal Core, to a file, or to Nominal Core with a file as backup (recommended to protect against network failures). It also provides configuration to manage the tradeoff between above listed concerns.
Usage example: streaming from memory to Nominal Core with file fallback
import pathlib
import time
from nominal.core import NominalClient
from nominal_streaming import NominalDatasetStream, PyNominalStreamOpts
if __name__ == "__main__":
num_points = 100_000
stream = (
NominalDatasetStream(
auth_header="<api key>",
opts=PyNominalStreamOpts(),
)
.enable_logging("info") # can set debug, warn, etc.
.with_core_consumer("<dataset rid>")
.with_file_fallback(pathlib.Path("local_fallback.avro"))
)
with stream:
# Stream 100_000 live readings (made up values)
for idx in range(num_points):
time_ns = int(time.time() * 1e9)
value = (idx % 50) + 0.5
stream.enqueue("channel_name", time_ns, value, tags={"tag_key": "tag_value"})
# Stream 100_000 points in one batch
start_time = int(time.time() * 1e9)
timestamp_offsets = int(1e9 / 1600)
timestamps = [start_time + timestamp_offsets * idx for idx in range(num_points)]
values = [(idx % 50) + 0.5 for idx in range(num_points)]
stream.enqueue_batch(
"channel_name",
timestamps,
values,
tags={"tag_key": "tag_value"}
)
Runtime metrics
Enable PyNominalStreamOpts(track_metrics=True) (or pass track_metrics=True to
NominalDatasetStream.create) to emit dictionary enqueue staleness and Core
request latency metrics. Metrics are disabled by default.
NumPy batches
NumPy is optional: install nominal-streaming[numpy], or use the NumPy installation
already provided by your application. Pass arrays directly to enqueue_batch:
import numpy as np
timestamps = np.array([1_700_000_000_000_000_000, 1_700_000_000_000_000_001], dtype="uint64")
values = np.array([1.25, 2.5], dtype="float64")
stream.enqueue_batch("temperature", timestamps, values)
Native-endian, aligned, one-dimensional NumPy arrays are copied into
Rust-owned memory without creating a Python object for every element. The fast path
supports int64/uint64 timestamps and float32/float64/int64/uint64 values,
including strided, reversed, and read-only views. Integer values retain the API's
existing conversion to doubles. Inputs can be changed or freed after the call returns.
Other dtypes, unaligned or non-native-endian arrays, and ndarray subclasses use the existing element-wise conversion. Masked values and datetime/timedelta values still require explicit conversion. Lists and tuples continue to work without NumPy.
Building and testing wheels
All builds use the Python 3.11 stable ABI (cp311-abi3) with native NumPy buffer
support. Build a wheel with the standard workspace command:
uv build --package nominal-streaming
NumPy is never needed to import the library or use lists. CI shares one wheel
across parallel Python 3.11 through 3.15 jobs. Each job exercises lists without
NumPy, then runs the batch regression tests and type checks with the locked test
dependencies. To run tests locally with the test dependency group:
uv run --group test python -m unittest discover -s py-nominal-streaming/tests -v
Measuring enqueue performance
With a wheel and the test dependencies installed:
python py-nominal-streaming/benchmarks/enqueue_batch.py
This compares direct arrays, pre-existing lists, and .tolist() plus the enqueue
call. It reports all samples and medians and reads the local Avro output to verify
point counts and value checksums. It measures the complete public enqueue call with
buffer capacity available; startup and draining are excluded. It does not measure
network throughput. Downstream serialization, compression, and upload backpressure
can limit the overall streaming improvement.
Log streams
NominalLogStream sends timestamped string messages and per-record string arguments to
an existing dataset's log channel. It uses bounded native buffering: enqueue blocks
when the byte budget is full, while releasing the Python GIL. Individual writes are
automatically batched into requests by Rust. Uploads use the same HTTP transport and
retry policy as time series streaming. stats().requests counts logical uploads, and
retries inside one upload do not add to it.
import os
from pathlib import Path
from nominal_streaming import NominalLogStream, PyNominalLogStreamOpts
opts = PyNominalLogStreamOpts(max_buffered_bytes=64 * 1024 * 1024)
with (NominalLogStream(os.environ["NOMINAL_TOKEN"], opts)
.with_core_consumer(os.environ["NOMINAL_DATASET_RID"])
.with_file_fallback(Path("log-backup"))) as stream:
stream.enqueue("engine", "2026-09-14T12:00:00.123456789Z", "started",
args={"engine": "left"})
stream.enqueue("engine", 1_800_000_000_000_000_001, "running",
args={"engine": "left", "phase": "test"})
stream.enqueue("engine", 1_800_000_000_000_000_002, "stopped",
args={"engine": "left", "phase": "done"})
print(stream.flush().acknowledged_records)
For local-only recording, use with NominalLogStream().to_file(Path("logs")) as stream:.
The destination is a directory, not a telemetry Avro file. Each channel gets one append-only
JSONL journal per stream, plus a manifest with its channel and nanosecond timestamp
metadata. Journal field names reserved for timestamp and message cannot also be argument
keys. File preservation increments
backed_up_records, not acknowledged_records.
Integer timestamps are signed Unix nanoseconds. Aware datetime values use exact
integer arithmetic. ISO 8601 strings require a timezone and support up to nine fractional
digits; ambiguous dates and naive datetimes are rejected. tags is an alias for args;
passing both is an error. All messages and argument keys/values must be strings.
close() refuses new writes, drains all accepted records, and reports delivery failures.
close(wait=False) starts graceful background draining; a later close() waits and
reports its result. stats() separates acceptance, acknowledgement, backup, and failure.
Use explicit close or a context manager to observe failures before process exit. No global
signal handlers are installed, and streams may be used from worker threads.
If a batch can be neither delivered nor written to the journal, the stream drops it,
counts it in failed_records, and keeps accepting writes. The next flush() raises once
to report the loss, and close() raises if no flush reported it yet. Configure a fallback
directory to avoid dropping logs during network failures.
opts = (
PyNominalLogStreamOpts()
.with_max_request_bytes(8 * 1024 * 1024)
.with_num_upload_workers(4)
.with_num_runtime_workers(2)
)
stream = NominalLogStream().with_options(opts).enable_logging("info")
Configure the destination before opening the stream. Runtime workers drive asynchronous HTTP I/O; upload workers perform compression and dispatch. Logs default to two runtime workers and four upload workers. Runtime workers must be positive but need not match the upload count. Configuration is frozen once opened; use a new stream to change it.
Metadata
Release files for nominal-streaming 0.11.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distributions (wheels)
Total release size: 31.4 MB
Release files / nominal_streaming-0.11.0-cp311-abi3-win_amd64.whl
| Download URL | nominal_streaming-0.11.0-cp311-abi3-win_amd64.whl |
|---|---|
| Size | 4.3 MB |
| Tags | CPython 3.11 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
d3e1ba0802f7b796bb159b600efafe0918834740645d2b8a2a0800d0c653cdfa
|
|
BLAKE2b-256 checksum How to use checksums |
8b1eece6a459ae85b28f582b2e27f341db5174510ac616b81a838ec5db903a3f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.24
|
Release files / nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_x86_64.whl
| Download URL | nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_x86_64.whl |
|---|---|
| Size | 4.2 MB |
| Tags | CPython 3.11 Linux musl 1.2+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
f33ee18da2334d5f5bb097d4180624117492aff517430c20726548d3fb1d32b8
|
|
BLAKE2b-256 checksum How to use checksums |
cff2f2e692ddfc2b39a446156bf8b0d4105b82381f9fe3c69c042bdf9e5301e2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.24
|
Release files / nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_armv7l.whl
| Download URL | nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_armv7l.whl |
|---|---|
| Size | 3.7 MB |
| Tags | CPython 3.11 Linux musl 1.2+ ARMv7l abi3 |
|
SHA-256 checksum How to use checksums |
a8448bf3a78c793179102461a3ef79ba0e0b4329f1aa16a903ef1264bc54b73b
|
|
BLAKE2b-256 checksum How to use checksums |
905af0ec71e1e8ea023621a454d2047191d29c041a37027a7302e5b5cfdc3c3c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.24
|
Release files / nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_aarch64.whl
| Download URL | nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_aarch64.whl |
|---|---|
| Size | 4.0 MB |
| Tags | CPython 3.11 Linux musl 1.2+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
452384b229bce52fbbea8a3529190836560aed009c8ab3ad66ef37bd9cf7b5af
|
|
BLAKE2b-256 checksum How to use checksums |
d96b58dd0b39c338744e29ac944849d8e6915ee84f2c416daa25a58173630d26
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.24
|
Release files / nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 4.1 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
8b043587472dbcf86ae54f1d00b9264c77d9b9671b501a72ae044a6a5039f158
|
|
BLAKE2b-256 checksum How to use checksums |
c1f5146f21e698022d3472c9a8d404e0ae2c6a6c03835e3da424e0a8333050fa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.24
|
Release files / nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
| Download URL | nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl |
|---|---|
| Size | 3.5 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ ARMv7l abi3 |
|
SHA-256 checksum How to use checksums |
56496bd78814f723880f3578e9bc328a0c36c4bf0e0b8e867d2c7a5d913cb26e
|
|
BLAKE2b-256 checksum How to use checksums |
95a243296ff6264f30c79405713ff418ee833ba02a40a917c8535c51b6656aa1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.24
|
Release files / nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 3.8 MB |
| Tags | CPython 3.11 Linux glibc 2.17+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
e9234fab1cb418db0a44aa353e6b237ee2e2d3b36df72621209a8e5613cef3c7
|
|
BLAKE2b-256 checksum How to use checksums |
1594076b6f51663c39eb571c826299393e46acb2f66dc2f109ec2e9596307921
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.24
|
Release files / nominal_streaming-0.11.0-cp311-abi3-macosx_11_0_arm64.whl
| Download URL | nominal_streaming-0.11.0-cp311-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 3.8 MB |
| Tags | CPython 3.11 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
e46631237d7648d79fa39c1e530c60f789d237c7b5a69fa99b7c31dcf9ffc2c4
|
|
BLAKE2b-256 checksum How to use checksums |
2b2cec5d7edf9f92325dd6da8034662c8de181f9d10eab102a98355fd64f966d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.24
|
Security for nominal-streaming 0.11.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.