Skip to main content

nominal-streaming Python Bindings

nominal-streaming is a thin python wrapper around the existing nominal-streaming rust crate. Usage semantics remain largely the same, but with some slight alterations to allow for a more pythonic interface.

Requires Python 3.11 or newer.

The library aims to balance three concerns:

  1. Data should exist in-memory only for a limited, configurable amount of time before it's sent to Core.
  2. Writes should fall back to disk if there are network failures.
  3. Backpressure should be applied to incoming requests when network throughput is saturated.

This library streams data to Nominal Core, to a file, or to Nominal Core with a file as backup (recommended to protect against network failures). It also provides configuration to manage the tradeoff between above listed concerns.

Usage example: streaming from memory to Nominal Core with file fallback

import pathlib
import time

from nominal.core import NominalClient
from nominal_streaming import NominalDatasetStream, PyNominalStreamOpts

if __name__ == "__main__":
    num_points = 100_000
    stream = (
        NominalDatasetStream(
            auth_header="<api key>",
            opts=PyNominalStreamOpts(),
        )
        .enable_logging("info") # can set debug, warn, etc.
        .with_core_consumer("<dataset rid>")
        .with_file_fallback(pathlib.Path("local_fallback.avro"))
    )

    with stream:
        # Stream 100_000 live readings (made up values)
        for idx in range(num_points):
            time_ns = int(time.time() * 1e9)
            value = (idx % 50) + 0.5
            stream.enqueue("channel_name", time_ns, value, tags={"tag_key": "tag_value"})

        # Stream 100_000 points in one batch
        start_time = int(time.time() * 1e9)
        timestamp_offsets = int(1e9 / 1600)
        timestamps = [start_time + timestamp_offsets * idx for idx in range(num_points)]
        values = [(idx % 50) + 0.5 for idx in range(num_points)]
        stream.enqueue_batch(
            "channel_name",
            timestamps,
            values,
            tags={"tag_key": "tag_value"}
        )

Runtime metrics

Enable PyNominalStreamOpts(track_metrics=True) (or pass track_metrics=True to NominalDatasetStream.create) to emit dictionary enqueue staleness and Core request latency metrics. Metrics are disabled by default.

NumPy batches

NumPy is optional: install nominal-streaming[numpy], or use the NumPy installation already provided by your application. Pass arrays directly to enqueue_batch:

import numpy as np

timestamps = np.array([1_700_000_000_000_000_000, 1_700_000_000_000_000_001], dtype="uint64")
values = np.array([1.25, 2.5], dtype="float64")
stream.enqueue_batch("temperature", timestamps, values)

Native-endian, aligned, one-dimensional NumPy arrays are copied into Rust-owned memory without creating a Python object for every element. The fast path supports int64/uint64 timestamps and float32/float64/int64/uint64 values, including strided, reversed, and read-only views. Integer values retain the API's existing conversion to doubles. Inputs can be changed or freed after the call returns.

Other dtypes, unaligned or non-native-endian arrays, and ndarray subclasses use the existing element-wise conversion. Masked values and datetime/timedelta values still require explicit conversion. Lists and tuples continue to work without NumPy.

Building and testing wheels

All builds use the Python 3.11 stable ABI (cp311-abi3) with native NumPy buffer support. Build a wheel with the standard workspace command:

uv build --package nominal-streaming

NumPy is never needed to import the library or use lists. CI shares one wheel across parallel Python 3.11 through 3.15 jobs. Each job exercises lists without NumPy, then runs the batch regression tests and type checks with the locked test dependencies. To run tests locally with the test dependency group:

uv run --group test python -m unittest discover -s py-nominal-streaming/tests -v

Measuring enqueue performance

With a wheel and the test dependencies installed:

python py-nominal-streaming/benchmarks/enqueue_batch.py

This compares direct arrays, pre-existing lists, and .tolist() plus the enqueue call. It reports all samples and medians and reads the local Avro output to verify point counts and value checksums. It measures the complete public enqueue call with buffer capacity available; startup and draining are excluded. It does not measure network throughput. Downstream serialization, compression, and upload backpressure can limit the overall streaming improvement.

Log streams

NominalLogStream sends timestamped string messages and per-record string arguments to an existing dataset's log channel. It uses bounded native buffering: enqueue blocks when the byte budget is full, while releasing the Python GIL. Individual writes are automatically batched into requests by Rust. Uploads use the same HTTP transport and retry policy as time series streaming. stats().requests counts logical uploads, and retries inside one upload do not add to it.

import os
from pathlib import Path
from nominal_streaming import NominalLogStream, PyNominalLogStreamOpts

opts = PyNominalLogStreamOpts(max_buffered_bytes=64 * 1024 * 1024)
with (NominalLogStream(os.environ["NOMINAL_TOKEN"], opts)
      .with_core_consumer(os.environ["NOMINAL_DATASET_RID"])
      .with_file_fallback(Path("log-backup"))) as stream:
    stream.enqueue("engine", "2026-09-14T12:00:00.123456789Z", "started",
                   args={"engine": "left"})
    stream.enqueue("engine", 1_800_000_000_000_000_001, "running",
                   args={"engine": "left", "phase": "test"})
    stream.enqueue("engine", 1_800_000_000_000_000_002, "stopped",
                   args={"engine": "left", "phase": "done"})
    print(stream.flush().acknowledged_records)

For local-only recording, use with NominalLogStream().to_file(Path("logs")) as stream:. The destination is a directory, not a telemetry Avro file. Each channel gets one append-only JSONL journal per stream, plus a manifest with its channel and nanosecond timestamp metadata. Journal field names reserved for timestamp and message cannot also be argument keys. File preservation increments backed_up_records, not acknowledged_records.

Integer timestamps are signed Unix nanoseconds. Aware datetime values use exact integer arithmetic. ISO 8601 strings require a timezone and support up to nine fractional digits; ambiguous dates and naive datetimes are rejected. tags is an alias for args; passing both is an error. All messages and argument keys/values must be strings.

close() refuses new writes, drains all accepted records, and reports delivery failures. close(wait=False) starts graceful background draining; a later close() waits and reports its result. stats() separates acceptance, acknowledgement, backup, and failure. Use explicit close or a context manager to observe failures before process exit. No global signal handlers are installed, and streams may be used from worker threads.

If a batch can be neither delivered nor written to the journal, the stream drops it, counts it in failed_records, and keeps accepting writes. The next flush() raises once to report the loss, and close() raises if no flush reported it yet. Configure a fallback directory to avoid dropping logs during network failures.

opts = (
    PyNominalLogStreamOpts()
    .with_max_request_bytes(8 * 1024 * 1024)
    .with_num_upload_workers(4)
    .with_num_runtime_workers(2)
)
stream = NominalLogStream().with_options(opts).enable_logging("info")

Configure the destination before opening the stream. Runtime workers drive asynchronous HTTP I/O; upload workers perform compression and dispatch. Logs default to two runtime workers and four upload workers. Runtime workers must be positive but need not match the upload count. Configuration is frozen once opened; use a new stream to change it.

Metadata

Release files for nominal-streaming 0.10.7

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for nominal-streaming 0.10.7
File Actions
nominal_streaming-0.10.7-cp311-abi3-win_amd64.whl CPython 3.11 abi3 Windows x86-64 Details for nominal_streaming-0.10.7-cp311-abi3-win_amd64.whl
nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_x86_64.whl CPython 3.11 abi3 Linux musl 1.2+ x86-64 Details for nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_x86_64.whl
nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_armv7l.whl CPython 3.11 abi3 Linux musl 1.2+ ARMv7l Details for nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_armv7l.whl
nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_aarch64.whl CPython 3.11 abi3 Linux musl 1.2+ ARM64 Details for nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_aarch64.whl
nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl CPython 3.11 abi3 Linux glibc 2.17+ x86-64 Details for nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl CPython 3.11 abi3 Linux glibc 2.17+ ARMv7l Details for nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.11 abi3 Linux glibc 2.17+ ARM64 Details for nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
nominal_streaming-0.10.7-cp311-abi3-macosx_11_0_arm64.whl CPython 3.11 abi3 macOS 11.0+ ARM64 Details for nominal_streaming-0.10.7-cp311-abi3-macosx_11_0_arm64.whl

Total release size: 31.3 MB

Release files / nominal_streaming-0.10.7-cp311-abi3-win_amd64.whl

Download URL nominal_streaming-0.10.7-cp311-abi3-win_amd64.whl
Size 4.3 MB
Tags CPython 3.11 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
6e525049aa514c46eb22783d5d36bb4882fe38d85fcd7a6f054ed8d272dc3010
BLAKE2b-256 checksum
How to use checksums
127e1b5b9ce7b2448ba9563aaf7c04cac999e82d70ee3870d7ea925ecd7f986d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_x86_64.whl

Download URL nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_x86_64.whl
Size 4.2 MB
Tags CPython 3.11 Linux musl 1.2+ x86-64 abi3
SHA-256 checksum
How to use checksums
9e6549fbe0f50769f11700bb6a8e2f4c7c0ecfc19f98c936d312c11ce78ddd3d
BLAKE2b-256 checksum
How to use checksums
a9a9244fca78ec5b70de892e84b8cb3f3ee09e67901ac82efff4a2063270048c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_armv7l.whl

Download URL nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_armv7l.whl
Size 3.7 MB
Tags CPython 3.11 Linux musl 1.2+ ARMv7l abi3
SHA-256 checksum
How to use checksums
bf8efb90723c21d2f12d41874869ed4a476cc5d99a34a6644274b2a68d11d6f8
BLAKE2b-256 checksum
How to use checksums
7165ece731519f11bf5f741b5249a23391d4a8a05a651c0a08040cef495fc205
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_aarch64.whl

Download URL nominal_streaming-0.10.7-cp311-abi3-musllinux_1_2_aarch64.whl
Size 3.9 MB
Tags CPython 3.11 Linux musl 1.2+ ARM64 abi3
SHA-256 checksum
How to use checksums
7c4e5552bd539a0345cc0399c0ee96367acd5a4ef60af806a34e8fa49754e549
BLAKE2b-256 checksum
How to use checksums
d2f3de8f20b577d77a2d2f3b596129dee5c36be0ac61d2999472ce1b2728a584
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Download URL nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Size 4.1 MB
Tags CPython 3.11 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
3c7fc71824f7cd4c505ab7ee2a3a8a4338dd72ed2fee32cd32b77f7882fd2f53
BLAKE2b-256 checksum
How to use checksums
ec6a84fa710ecab3860695d7d29dad58854cefc2a3a5d72a37c03d9bc34fc81c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl

Download URL nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Size 3.5 MB
Tags CPython 3.11 Linux glibc 2.17+ ARMv7l abi3
SHA-256 checksum
How to use checksums
81442fc85331984ff19eae351319d4523377a7eac74d360100425cf3107f86d1
BLAKE2b-256 checksum
How to use checksums
e1f01e59cc35213a0abdb3c2e0a1035d913ffdf44f4ce03a097a8afe38d908cd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL nominal_streaming-0.10.7-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 3.8 MB
Tags CPython 3.11 Linux glibc 2.17+ ARM64 abi3
SHA-256 checksum
How to use checksums
b783eb1df23f000d9fb85cf80785bff4af1fb4769f4cd637a43d0051fbb1ef92
BLAKE2b-256 checksum
How to use checksums
821a5737dbcf6470064e6e3e86ed3e025504a831fa906f91cb9be9f2475c1d5e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.7-cp311-abi3-macosx_11_0_arm64.whl

Download URL nominal_streaming-0.10.7-cp311-abi3-macosx_11_0_arm64.whl
Size 3.8 MB
Tags CPython 3.11 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
48bd955f7dcd4fb9361d2fcf29496807d876a834a15a1356133691557c0b4d21
BLAKE2b-256 checksum
How to use checksums
8547ffcd221b486576f74dd531187b9ecda33c8f037ec497b7bb145bb4c02ace
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Security for nominal-streaming 0.10.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.10.7 This release

8 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page