Skip to main content

nominal-streaming Python Bindings

nominal-streaming is a thin python wrapper around the existing nominal-streaming rust crate. Usage semantics remain largely the same, but with some slight alterations to allow for a more pythonic interface.

Requires Python 3.11 or newer.

The library aims to balance three concerns:

  1. Data should exist in-memory only for a limited, configurable amount of time before it's sent to Core.
  2. Writes should fall back to disk if there are network failures.
  3. Backpressure should be applied to incoming requests when network throughput is saturated.

This library streams data to Nominal Core, to a file, or to Nominal Core with a file as backup (recommended to protect against network failures). It also provides configuration to manage the tradeoff between above listed concerns.

Usage example: streaming from memory to Nominal Core with file fallback

import pathlib
import time

from nominal.core import NominalClient
from nominal_streaming import NominalDatasetStream, PyNominalStreamOpts

if __name__ == "__main__":
    num_points = 100_000
    stream = (
        NominalDatasetStream(
            auth_header="<api key>",
            opts=PyNominalStreamOpts(),
        )
        .enable_logging("info") # can set debug, warn, etc.
        .with_core_consumer("<dataset rid>")
        .with_file_fallback(pathlib.Path("local_fallback.avro"))
    )

    with stream:
        # Stream 100_000 live readings (made up values)
        for idx in range(num_points):
            time_ns = int(time.time() * 1e9)
            value = (idx % 50) + 0.5
            stream.enqueue("channel_name", time_ns, value, tags={"tag_key": "tag_value"})

        # Stream 100_000 points in one batch
        start_time = int(time.time() * 1e9)
        timestamp_offsets = int(1e9 / 1600)
        timestamps = [start_time + timestamp_offsets * idx for idx in range(num_points)]
        values = [(idx % 50) + 0.5 for idx in range(num_points)]
        stream.enqueue_batch(
            "channel_name",
            timestamps,
            values,
            tags={"tag_key": "tag_value"}
        )

Runtime metrics

Enable PyNominalStreamOpts(track_metrics=True) (or pass track_metrics=True to NominalDatasetStream.create) to emit dictionary enqueue staleness and Core request latency metrics. Metrics are disabled by default.

NumPy batches

NumPy is optional: install nominal-streaming[numpy], or use the NumPy installation already provided by your application. Pass arrays directly to enqueue_batch:

import numpy as np

timestamps = np.array([1_700_000_000_000_000_000, 1_700_000_000_000_000_001], dtype="uint64")
values = np.array([1.25, 2.5], dtype="float64")
stream.enqueue_batch("temperature", timestamps, values)

Native-endian, aligned, one-dimensional NumPy arrays are copied into Rust-owned memory without creating a Python object for every element. The fast path supports int64/uint64 timestamps and float32/float64/int64/uint64 values, including strided, reversed, and read-only views. Integer values retain the API's existing conversion to doubles. Inputs can be changed or freed after the call returns.

Other dtypes, unaligned or non-native-endian arrays, and ndarray subclasses use the existing element-wise conversion. Masked values and datetime/timedelta values still require explicit conversion. Lists and tuples continue to work without NumPy.

Building and testing wheels

All builds use the Python 3.11 stable ABI (cp311-abi3) with native NumPy buffer support. Build a wheel with the standard workspace command:

uv build --package nominal-streaming

NumPy is never needed to import the library or use lists. CI shares one wheel across parallel Python 3.11 through 3.15 jobs. Each job exercises lists without NumPy, then runs the batch regression tests and type checks with the locked test dependencies. To run tests locally with the test dependency group:

uv run --group test python -m unittest discover -s py-nominal-streaming/tests -v

Measuring enqueue performance

With a wheel and the test dependencies installed:

python py-nominal-streaming/benchmarks/enqueue_batch.py

This compares direct arrays, pre-existing lists, and .tolist() plus the enqueue call. It reports all samples and medians and reads the local Avro output to verify point counts and value checksums. It measures the complete public enqueue call with buffer capacity available; startup and draining are excluded. It does not measure network throughput. Downstream serialization, compression, and upload backpressure can limit the overall streaming improvement.

Log streams

NominalLogStream sends timestamped string messages and per-record string arguments to an existing dataset's log channel. It uses bounded native buffering: enqueue blocks when the byte budget is full, while releasing the Python GIL. Individual writes are automatically batched into requests by Rust. Uploads use the same HTTP transport and retry policy as time series streaming. stats().requests counts logical uploads, and retries inside one upload do not add to it.

import os
from pathlib import Path
from nominal_streaming import NominalLogStream, PyNominalLogStreamOpts

opts = PyNominalLogStreamOpts(max_buffered_bytes=64 * 1024 * 1024)
with (NominalLogStream(os.environ["NOMINAL_TOKEN"], opts)
      .with_core_consumer(os.environ["NOMINAL_DATASET_RID"])
      .with_file_fallback(Path("log-backup"))) as stream:
    stream.enqueue("engine", "2026-09-14T12:00:00.123456789Z", "started",
                   args={"engine": "left"})
    stream.enqueue("engine", 1_800_000_000_000_000_001, "running",
                   args={"engine": "left", "phase": "test"})
    stream.enqueue("engine", 1_800_000_000_000_000_002, "stopped",
                   args={"engine": "left", "phase": "done"})
    print(stream.flush().acknowledged_records)

For local-only recording, use with NominalLogStream().to_file(Path("logs")) as stream:. The destination is a directory, not a telemetry Avro file. Each channel gets one append-only JSONL journal per stream, plus a manifest with its channel and nanosecond timestamp metadata. Journal field names reserved for timestamp and message cannot also be argument keys. File preservation increments backed_up_records, not acknowledged_records.

Integer timestamps are signed Unix nanoseconds. Aware datetime values use exact integer arithmetic. ISO 8601 strings require a timezone and support up to nine fractional digits; ambiguous dates and naive datetimes are rejected. tags is an alias for args; passing both is an error. All messages and argument keys/values must be strings.

close() refuses new writes, drains all accepted records, and reports delivery failures. close(wait=False) starts graceful background draining; a later close() waits and reports its result. stats() separates acceptance, acknowledgement, backup, and failure. Use explicit close or a context manager to observe failures before process exit. No global signal handlers are installed, and streams may be used from worker threads.

If a batch can be neither delivered nor written to the journal, the stream drops it, counts it in failed_records, and keeps accepting writes. The next flush() raises once to report the loss, and close() raises if no flush reported it yet. Configure a fallback directory to avoid dropping logs during network failures.

opts = (
    PyNominalLogStreamOpts()
    .with_max_request_bytes(8 * 1024 * 1024)
    .with_num_upload_workers(4)
    .with_num_runtime_workers(2)
)
stream = NominalLogStream().with_options(opts).enable_logging("info")

Configure the destination before opening the stream. Runtime workers drive asynchronous HTTP I/O; upload workers perform compression and dispatch. Logs default to two runtime workers and four upload workers. Runtime workers must be positive but need not match the upload count. Configuration is frozen once opened; use a new stream to change it.

Metadata

Release files for nominal-streaming 0.11.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for nominal-streaming 0.11.1
File Actions
nominal_streaming-0.11.1-cp311-abi3-win_amd64.whl CPython 3.11 abi3 Windows x86-64 Details for nominal_streaming-0.11.1-cp311-abi3-win_amd64.whl
nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_x86_64.whl CPython 3.11 abi3 Linux musl 1.2+ x86-64 Details for nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_x86_64.whl
nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_armv7l.whl CPython 3.11 abi3 Linux musl 1.2+ ARMv7l Details for nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_armv7l.whl
nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_aarch64.whl CPython 3.11 abi3 Linux musl 1.2+ ARM64 Details for nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_aarch64.whl
nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl CPython 3.11 abi3 Linux glibc 2.17+ x86-64 Details for nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl CPython 3.11 abi3 Linux glibc 2.17+ ARMv7l Details for nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.11 abi3 Linux glibc 2.17+ ARM64 Details for nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
nominal_streaming-0.11.1-cp311-abi3-macosx_11_0_arm64.whl CPython 3.11 abi3 macOS 11.0+ ARM64 Details for nominal_streaming-0.11.1-cp311-abi3-macosx_11_0_arm64.whl

Total release size: 31.4 MB

Release files / nominal_streaming-0.11.1-cp311-abi3-win_amd64.whl

Download URL nominal_streaming-0.11.1-cp311-abi3-win_amd64.whl
Size 4.3 MB
Tags CPython 3.11 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
3cf5c6155fb7f90c353046fbfde50b2102aee6c82bb69fc06fe9cca71bb07ca5
BLAKE2b-256 checksum
How to use checksums
34d4031cd54ab052e898c986db5cdf03d5fca64f7dac2ebceac990f5ccb18ef2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_x86_64.whl

Download URL nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_x86_64.whl
Size 4.2 MB
Tags CPython 3.11 Linux musl 1.2+ x86-64 abi3
SHA-256 checksum
How to use checksums
52b90ec0ef1d9e3bc28ec0bdeb2d1ee2d477193778d2b295b566dcaae1280622
BLAKE2b-256 checksum
How to use checksums
d1014daf2f1c8a529493aae2ffcc0f6661e1ec1baef5eb11c758a4ce997571d6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_armv7l.whl

Download URL nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_armv7l.whl
Size 3.8 MB
Tags CPython 3.11 Linux musl 1.2+ ARMv7l abi3
SHA-256 checksum
How to use checksums
1d420ee2c2d28ed7744fea841911250d1b30113df37e841d368f115be919c8f1
BLAKE2b-256 checksum
How to use checksums
7474eff02cf50eb1a2a74d9118fa57b451ae6bae2aafefd9b8b2b89c66852003
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_aarch64.whl

Download URL nominal_streaming-0.11.1-cp311-abi3-musllinux_1_2_aarch64.whl
Size 4.0 MB
Tags CPython 3.11 Linux musl 1.2+ ARM64 abi3
SHA-256 checksum
How to use checksums
e6c53b70307b44b4b7a0eb3ac1cf87b823ca570bfd2f8874fd6f4c313eeb81de
BLAKE2b-256 checksum
How to use checksums
4238e6ca7e29a9c831ffffb99f6e40a5a821211aa86ba528b65d003e6021b534
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Download URL nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Size 4.1 MB
Tags CPython 3.11 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
5e6f5458ff0d359c89f6888a4548782f9b0cde8b9c7604fb3ce9a8daf2ea790d
BLAKE2b-256 checksum
How to use checksums
3a35644b60bd3ea8519fcc767172d90a3496534d437b8fd3403ec697f27ae2ce
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl

Download URL nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Size 3.6 MB
Tags CPython 3.11 Linux glibc 2.17+ ARMv7l abi3
SHA-256 checksum
How to use checksums
e6126d7c394d862e2682c2635446e14c847f929b6e2d9268a9183f6ae544a362
BLAKE2b-256 checksum
How to use checksums
bd9583284018372c20f69ac036cb848d941e01b2d2a259d8c749d7bab659fc1b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL nominal_streaming-0.11.1-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 3.8 MB
Tags CPython 3.11 Linux glibc 2.17+ ARM64 abi3
SHA-256 checksum
How to use checksums
e0268bae8c5bdb47ef8f17a2d7c4e46a700c075790e658fff9488032c482521e
BLAKE2b-256 checksum
How to use checksums
44730d3e8b9fdffde7f2f6e068b76e024baab7ea78de90ddedabff211c2e4eda
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.1-cp311-abi3-macosx_11_0_arm64.whl

Download URL nominal_streaming-0.11.1-cp311-abi3-macosx_11_0_arm64.whl
Size 3.8 MB
Tags CPython 3.11 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
02138f8bb556b43bb6ff0316e362931c04eed6f4a413769c87b82e5b4b6c03c6
BLAKE2b-256 checksum
How to use checksums
6c1b32ce226106f376d2cb457b7022cdab0b78b17a2e9746aaef5851829f227c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Security for nominal-streaming 0.11.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.11.1 This release

8 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page