Skip to main content

nominal-streaming Python Bindings

nominal-streaming is a thin python wrapper around the existing nominal-streaming rust crate. Usage semantics remain largely the same, but with some slight alterations to allow for a more pythonic interface.

Requires Python 3.11 or newer.

The library aims to balance three concerns:

  1. Data should exist in-memory only for a limited, configurable amount of time before it's sent to Core.
  2. Writes should fall back to disk if there are network failures.
  3. Backpressure should be applied to incoming requests when network throughput is saturated.

This library streams data to Nominal Core, to a file, or to Nominal Core with a file as backup (recommended to protect against network failures). It also provides configuration to manage the tradeoff between above listed concerns.

Usage example: streaming from memory to Nominal Core with file fallback

import pathlib
import time

from nominal.core import NominalClient
from nominal_streaming import NominalDatasetStream, PyNominalStreamOpts

if __name__ == "__main__":
    num_points = 100_000
    stream = (
        NominalDatasetStream(
            auth_header="<api key>",
            opts=PyNominalStreamOpts(),
        )
        .enable_logging("info") # can set debug, warn, etc.
        .with_core_consumer("<dataset rid>")
        .with_file_fallback(pathlib.Path("local_fallback.avro"))
    )

    with stream:
        # Stream 100_000 live readings (made up values)
        for idx in range(num_points):
            time_ns = int(time.time() * 1e9)
            value = (idx % 50) + 0.5
            stream.enqueue("channel_name", time_ns, value, tags={"tag_key": "tag_value"})

        # Stream 100_000 points in one batch
        start_time = int(time.time() * 1e9)
        timestamp_offsets = int(1e9 / 1600)
        timestamps = [start_time + timestamp_offsets * idx for idx in range(num_points)]
        values = [(idx % 50) + 0.5 for idx in range(num_points)]
        stream.enqueue_batch(
            "channel_name",
            timestamps,
            values,
            tags={"tag_key": "tag_value"}
        )

Runtime metrics

Enable PyNominalStreamOpts(track_metrics=True) (or pass track_metrics=True to NominalDatasetStream.create) to emit dictionary enqueue staleness and Core request latency metrics. Metrics are disabled by default.

NumPy batches

NumPy is optional: install nominal-streaming[numpy], or use the NumPy installation already provided by your application. Pass arrays directly to enqueue_batch:

import numpy as np

timestamps = np.array([1_700_000_000_000_000_000, 1_700_000_000_000_000_001], dtype="uint64")
values = np.array([1.25, 2.5], dtype="float64")
stream.enqueue_batch("temperature", timestamps, values)

Native-endian, aligned, one-dimensional NumPy arrays are copied into Rust-owned memory without creating a Python object for every element. The fast path supports int64/uint64 timestamps and float32/float64/int64/uint64 values, including strided, reversed, and read-only views. Integer values retain the API's existing conversion to doubles. Inputs can be changed or freed after the call returns.

Other dtypes, unaligned or non-native-endian arrays, and ndarray subclasses use the existing element-wise conversion. Masked values and datetime/timedelta values still require explicit conversion. Lists and tuples continue to work without NumPy.

Building and testing wheels

All builds use the Python 3.11 stable ABI (cp311-abi3) with native NumPy buffer support. Build a wheel with the standard workspace command:

uv build --package nominal-streaming

NumPy is never needed to import the library or use lists. CI shares one wheel across parallel Python 3.11 through 3.15 jobs. Each job exercises lists without NumPy, then runs the batch regression tests and type checks with the locked test dependencies. To run tests locally with the test dependency group:

uv run --group test python -m unittest discover -s py-nominal-streaming/tests -v

Measuring enqueue performance

With a wheel and the test dependencies installed:

python py-nominal-streaming/benchmarks/enqueue_batch.py

This compares direct arrays, pre-existing lists, and .tolist() plus the enqueue call. It reports all samples and medians and reads the local Avro output to verify point counts and value checksums. It measures the complete public enqueue call with buffer capacity available; startup and draining are excluded. It does not measure network throughput. Downstream serialization, compression, and upload backpressure can limit the overall streaming improvement.

Log streams

NominalLogStream sends timestamped string messages and per-record string arguments to an existing dataset's log channel. It uses bounded native buffering: enqueue blocks when the byte budget is full, while releasing the Python GIL. Individual writes are automatically batched into requests by Rust. Uploads use the same HTTP transport and retry policy as time series streaming. stats().requests counts logical uploads, and retries inside one upload do not add to it.

import os
from pathlib import Path
from nominal_streaming import NominalLogStream, PyNominalLogStreamOpts

opts = PyNominalLogStreamOpts(max_buffered_bytes=64 * 1024 * 1024)
with (NominalLogStream(os.environ["NOMINAL_TOKEN"], opts)
      .with_core_consumer(os.environ["NOMINAL_DATASET_RID"])
      .with_file_fallback(Path("log-backup"))) as stream:
    stream.enqueue("engine", "2026-09-14T12:00:00.123456789Z", "started",
                   args={"engine": "left"})
    stream.enqueue("engine", 1_800_000_000_000_000_001, "running",
                   args={"engine": "left", "phase": "test"})
    stream.enqueue("engine", 1_800_000_000_000_000_002, "stopped",
                   args={"engine": "left", "phase": "done"})
    print(stream.flush().acknowledged_records)

For local-only recording, use with NominalLogStream().to_file(Path("logs")) as stream:. The destination is a directory, not a telemetry Avro file. Each channel gets one append-only JSONL journal per stream, plus a manifest with its channel and nanosecond timestamp metadata. Journal field names reserved for timestamp and message cannot also be argument keys. File preservation increments backed_up_records, not acknowledged_records.

Integer timestamps are signed Unix nanoseconds. Aware datetime values use exact integer arithmetic. ISO 8601 strings require a timezone and support up to nine fractional digits; ambiguous dates and naive datetimes are rejected. tags is an alias for args; passing both is an error. All messages and argument keys/values must be strings.

close() refuses new writes, drains all accepted records, and reports delivery failures. close(wait=False) starts graceful background draining; a later close() waits and reports its result. stats() separates acceptance, acknowledgement, backup, and failure. Use explicit close or a context manager to observe failures before process exit. No global signal handlers are installed, and streams may be used from worker threads.

If a batch can be neither delivered nor written to the journal, the stream drops it, counts it in failed_records, and keeps accepting writes. The next flush() raises once to report the loss, and close() raises if no flush reported it yet. Configure a fallback directory to avoid dropping logs during network failures.

opts = (
    PyNominalLogStreamOpts()
    .with_max_request_bytes(8 * 1024 * 1024)
    .with_num_upload_workers(4)
    .with_num_runtime_workers(2)
)
stream = NominalLogStream().with_options(opts).enable_logging("info")

Configure the destination before opening the stream. Runtime workers drive asynchronous HTTP I/O; upload workers perform compression and dispatch. Logs default to two runtime workers and four upload workers. Runtime workers must be positive but need not match the upload count. Configuration is frozen once opened; use a new stream to change it.

Metadata

Release files for nominal-streaming 0.11.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for nominal-streaming 0.11.0
File Actions
nominal_streaming-0.11.0-cp311-abi3-win_amd64.whl CPython 3.11 abi3 Windows x86-64 Details for nominal_streaming-0.11.0-cp311-abi3-win_amd64.whl
nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_x86_64.whl CPython 3.11 abi3 Linux musl 1.2+ x86-64 Details for nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_x86_64.whl
nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_armv7l.whl CPython 3.11 abi3 Linux musl 1.2+ ARMv7l Details for nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_armv7l.whl
nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_aarch64.whl CPython 3.11 abi3 Linux musl 1.2+ ARM64 Details for nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_aarch64.whl
nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl CPython 3.11 abi3 Linux glibc 2.17+ x86-64 Details for nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl CPython 3.11 abi3 Linux glibc 2.17+ ARMv7l Details for nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.11 abi3 Linux glibc 2.17+ ARM64 Details for nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
nominal_streaming-0.11.0-cp311-abi3-macosx_11_0_arm64.whl CPython 3.11 abi3 macOS 11.0+ ARM64 Details for nominal_streaming-0.11.0-cp311-abi3-macosx_11_0_arm64.whl

Total release size: 31.4 MB

Release files / nominal_streaming-0.11.0-cp311-abi3-win_amd64.whl

Download URL nominal_streaming-0.11.0-cp311-abi3-win_amd64.whl
Size 4.3 MB
Tags CPython 3.11 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
d3e1ba0802f7b796bb159b600efafe0918834740645d2b8a2a0800d0c653cdfa
BLAKE2b-256 checksum
How to use checksums
8b1eece6a459ae85b28f582b2e27f341db5174510ac616b81a838ec5db903a3f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_x86_64.whl

Download URL nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_x86_64.whl
Size 4.2 MB
Tags CPython 3.11 Linux musl 1.2+ x86-64 abi3
SHA-256 checksum
How to use checksums
f33ee18da2334d5f5bb097d4180624117492aff517430c20726548d3fb1d32b8
BLAKE2b-256 checksum
How to use checksums
cff2f2e692ddfc2b39a446156bf8b0d4105b82381f9fe3c69c042bdf9e5301e2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_armv7l.whl

Download URL nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_armv7l.whl
Size 3.7 MB
Tags CPython 3.11 Linux musl 1.2+ ARMv7l abi3
SHA-256 checksum
How to use checksums
a8448bf3a78c793179102461a3ef79ba0e0b4329f1aa16a903ef1264bc54b73b
BLAKE2b-256 checksum
How to use checksums
905af0ec71e1e8ea023621a454d2047191d29c041a37027a7302e5b5cfdc3c3c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_aarch64.whl

Download URL nominal_streaming-0.11.0-cp311-abi3-musllinux_1_2_aarch64.whl
Size 4.0 MB
Tags CPython 3.11 Linux musl 1.2+ ARM64 abi3
SHA-256 checksum
How to use checksums
452384b229bce52fbbea8a3529190836560aed009c8ab3ad66ef37bd9cf7b5af
BLAKE2b-256 checksum
How to use checksums
d96b58dd0b39c338744e29ac944849d8e6915ee84f2c416daa25a58173630d26
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Download URL nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Size 4.1 MB
Tags CPython 3.11 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
8b043587472dbcf86ae54f1d00b9264c77d9b9671b501a72ae044a6a5039f158
BLAKE2b-256 checksum
How to use checksums
c1f5146f21e698022d3472c9a8d404e0ae2c6a6c03835e3da424e0a8333050fa
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl

Download URL nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Size 3.5 MB
Tags CPython 3.11 Linux glibc 2.17+ ARMv7l abi3
SHA-256 checksum
How to use checksums
56496bd78814f723880f3578e9bc328a0c36c4bf0e0b8e867d2c7a5d913cb26e
BLAKE2b-256 checksum
How to use checksums
95a243296ff6264f30c79405713ff418ee833ba02a40a917c8535c51b6656aa1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL nominal_streaming-0.11.0-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 3.8 MB
Tags CPython 3.11 Linux glibc 2.17+ ARM64 abi3
SHA-256 checksum
How to use checksums
e9234fab1cb418db0a44aa353e6b237ee2e2d3b36df72621209a8e5613cef3c7
BLAKE2b-256 checksum
How to use checksums
1594076b6f51663c39eb571c826299393e46acb2f66dc2f109ec2e9596307921
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.11.0-cp311-abi3-macosx_11_0_arm64.whl

Download URL nominal_streaming-0.11.0-cp311-abi3-macosx_11_0_arm64.whl
Size 3.8 MB
Tags CPython 3.11 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
e46631237d7648d79fa39c1e530c60f789d237c7b5a69fa99b7c31dcf9ffc2c4
BLAKE2b-256 checksum
How to use checksums
2b2cec5d7edf9f92325dd6da8034662c8de181f9d10eab102a98355fd64f966d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Security for nominal-streaming 0.11.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.11.0 This release

8 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page