Skip to main content

nominal-streaming Python Bindings

nominal-streaming is a thin python wrapper around the existing nominal-streaming rust crate. Usage semantics remain largely the same, but with some slight alterations to allow for a more pythonic interface.

Requires Python 3.11 or newer.

The library aims to balance three concerns:

  1. Data should exist in-memory only for a limited, configurable amount of time before it's sent to Core.
  2. Writes should fall back to disk if there are network failures.
  3. Backpressure should be applied to incoming requests when network throughput is saturated.

This library streams data to Nominal Core, to a file, or to Nominal Core with a file as backup (recommended to protect against network failures). It also provides configuration to manage the tradeoff between above listed concerns.

Usage example: streaming from memory to Nominal Core with file fallback

import pathlib
import time

from nominal.core import NominalClient
from nominal_streaming import NominalDatasetStream, PyNominalStreamOpts

if __name__ == "__main__":
    num_points = 100_000
    stream = (
        NominalDatasetStream(
            auth_header="<api key>",
            opts=PyNominalStreamOpts(),
        )
        .enable_logging("info") # can set debug, warn, etc.
        .with_core_consumer("<dataset rid>")
        .with_file_fallback(pathlib.Path("local_fallback.avro"))
    )

    with stream:
        # Stream 100_000 live readings (made up values)
        for idx in range(num_points):
            time_ns = int(time.time() * 1e9)
            value = (idx % 50) + 0.5
            stream.enqueue("channel_name", time_ns, value, tags={"tag_key": "tag_value"})

        # Stream 100_000 points in one batch
        start_time = int(time.time() * 1e9)
        timestamp_offsets = int(1e9 / 1600)
        timestamps = [start_time + timestamp_offsets * idx for idx in range(num_points)]
        values = [(idx % 50) + 0.5 for idx in range(num_points)]
        stream.enqueue_batch(
            "channel_name",
            timestamps,
            values,
            tags={"tag_key": "tag_value"}
        )

Runtime metrics

Enable PyNominalStreamOpts(track_metrics=True) (or pass track_metrics=True to NominalDatasetStream.create) to emit dictionary enqueue staleness and Core request latency metrics. Metrics are disabled by default.

NumPy batches

NumPy is optional: install nominal-streaming[numpy], or use the NumPy installation already provided by your application. Pass arrays directly to enqueue_batch:

import numpy as np

timestamps = np.array([1_700_000_000_000_000_000, 1_700_000_000_000_000_001], dtype="uint64")
values = np.array([1.25, 2.5], dtype="float64")
stream.enqueue_batch("temperature", timestamps, values)

Native-endian, aligned, one-dimensional NumPy arrays are copied into Rust-owned memory without creating a Python object for every element. The fast path supports int64/uint64 timestamps and float32/float64/int64/uint64 values, including strided, reversed, and read-only views. Integer values retain the API's existing conversion to doubles. Inputs can be changed or freed after the call returns.

Other dtypes, unaligned or non-native-endian arrays, and ndarray subclasses use the existing element-wise conversion. Masked values and datetime/timedelta values still require explicit conversion. Lists and tuples continue to work without NumPy.

Building and testing wheels

All builds use the Python 3.11 stable ABI (cp311-abi3) with native NumPy buffer support. Build a wheel with the standard workspace command:

uv build --package nominal-streaming

NumPy is never needed to import the library or use lists. CI shares one wheel across parallel Python 3.11 through 3.15 jobs. Each job exercises lists without NumPy, then runs the batch regression tests and type checks with the locked test dependencies. To run tests locally with the test dependency group:

uv run --group test python -m unittest discover -s py-nominal-streaming/tests -v

Measuring enqueue performance

With a wheel and the test dependencies installed:

python py-nominal-streaming/benchmarks/enqueue_batch.py

This compares direct arrays, pre-existing lists, and .tolist() plus the enqueue call. It reports all samples and medians and reads the local Avro output to verify point counts and value checksums. It measures the complete public enqueue call with buffer capacity available; startup and draining are excluded. It does not measure network throughput. Downstream serialization, compression, and upload backpressure can limit the overall streaming improvement.

Log streams

NominalLogStream sends timestamped string messages and per-record string arguments to an existing dataset's log channel. It uses bounded native buffering: enqueue blocks when the byte budget is full, while releasing the Python GIL. Individual writes are automatically batched into requests by Rust. Uploads use the same HTTP transport and retry policy as time series streaming. stats().requests counts logical uploads, and retries inside one upload do not add to it.

import os
from pathlib import Path
from nominal_streaming import NominalLogStream, PyNominalLogStreamOpts

opts = PyNominalLogStreamOpts(max_buffered_bytes=64 * 1024 * 1024)
with (NominalLogStream(os.environ["NOMINAL_TOKEN"], opts)
      .with_core_consumer(os.environ["NOMINAL_DATASET_RID"])
      .with_file_fallback(Path("log-backup"))) as stream:
    stream.enqueue("engine", "2026-09-14T12:00:00.123456789Z", "started",
                   args={"engine": "left"})
    stream.enqueue("engine", 1_800_000_000_000_000_001, "running",
                   args={"engine": "left", "phase": "test"})
    stream.enqueue("engine", 1_800_000_000_000_000_002, "stopped",
                   args={"engine": "left", "phase": "done"})
    print(stream.flush().acknowledged_records)

For local-only recording, use with NominalLogStream().to_file(Path("logs")) as stream:. The destination is a directory, not a telemetry Avro file. Each channel gets one append-only JSONL journal per stream, plus a manifest with its channel and nanosecond timestamp metadata. Journal field names reserved for timestamp and message cannot also be argument keys. File preservation increments backed_up_records, not acknowledged_records.

Integer timestamps are signed Unix nanoseconds. Aware datetime values use exact integer arithmetic. ISO 8601 strings require a timezone and support up to nine fractional digits; ambiguous dates and naive datetimes are rejected. tags is an alias for args; passing both is an error. All messages and argument keys/values must be strings.

close() refuses new writes, drains all accepted records, and reports delivery failures. close(wait=False) starts graceful background draining; a later close() waits and reports its result. stats() separates acceptance, acknowledgement, backup, and failure. Use explicit close or a context manager to observe failures before process exit. No global signal handlers are installed, and streams may be used from worker threads.

If a batch can be neither delivered nor written to the journal, the stream drops it, counts it in failed_records, and keeps accepting writes. The next flush() raises once to report the loss, and close() raises if no flush reported it yet. Configure a fallback directory to avoid dropping logs during network failures.

opts = (
    PyNominalLogStreamOpts()
    .with_max_request_bytes(8 * 1024 * 1024)
    .with_num_upload_workers(4)
    .with_num_runtime_workers(2)
)
stream = NominalLogStream().with_options(opts).enable_logging("info")

Configure the destination before opening the stream. Runtime workers drive asynchronous HTTP I/O; upload workers perform compression and dispatch. Logs default to two runtime workers and four upload workers. Runtime workers must be positive but need not match the upload count. Configuration is frozen once opened; use a new stream to change it.

Metadata

Release files for nominal-streaming 0.10.6

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Built distributions (wheels)

Table of built distributions (wheels) for nominal-streaming 0.10.6
File Actions
nominal_streaming-0.10.6-cp311-abi3-win_amd64.whl CPython 3.11 abi3 Windows x86-64 Details for nominal_streaming-0.10.6-cp311-abi3-win_amd64.whl
nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_x86_64.whl CPython 3.11 abi3 Linux musl 1.2+ x86-64 Details for nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_x86_64.whl
nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_armv7l.whl CPython 3.11 abi3 Linux musl 1.2+ ARMv7l Details for nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_armv7l.whl
nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_aarch64.whl CPython 3.11 abi3 Linux musl 1.2+ ARM64 Details for nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_aarch64.whl
nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl CPython 3.11 abi3 Linux glibc 2.17+ x86-64 Details for nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl CPython 3.11 abi3 Linux glibc 2.17+ ARMv7l Details for nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl CPython 3.11 abi3 Linux glibc 2.17+ ARM64 Details for nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
nominal_streaming-0.10.6-cp311-abi3-macosx_11_0_arm64.whl CPython 3.11 abi3 macOS 11.0+ ARM64 Details for nominal_streaming-0.10.6-cp311-abi3-macosx_11_0_arm64.whl

Total release size: 31.1 MB

Release files / nominal_streaming-0.10.6-cp311-abi3-win_amd64.whl

Download URL nominal_streaming-0.10.6-cp311-abi3-win_amd64.whl
Size 4.2 MB
Tags CPython 3.11 Windows x86-64 abi3
SHA-256 checksum
How to use checksums
7ecf9c97f3711538e0ed95f54d035727ffe3e2525b4d801f5aa08f45e075c008
BLAKE2b-256 checksum
How to use checksums
7b011da3b176fae954f6d0de427bc6a85e8e766af6f834b9af69a7d6c75eaaec
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_x86_64.whl

Download URL nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_x86_64.whl
Size 4.2 MB
Tags CPython 3.11 Linux musl 1.2+ x86-64 abi3
SHA-256 checksum
How to use checksums
f767f607ba9e260c909d2febd5e6a20bb4f26ddd2b2f1aad5841bd41e8ff391c
BLAKE2b-256 checksum
How to use checksums
4546eeb2afd951ff647ac15bf5cdc696126455d7e19561a6dd0cad47540fa592
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_armv7l.whl

Download URL nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_armv7l.whl
Size 3.7 MB
Tags CPython 3.11 Linux musl 1.2+ ARMv7l abi3
SHA-256 checksum
How to use checksums
39335d4c9eb0cfe815adc0d2703166be54e5c0fb43e24e7a5e3c4d96fbd72cfd
BLAKE2b-256 checksum
How to use checksums
b27f831fcf2dca6679649bbdccff1dcc43bece701b5023b222ce7d06593e287c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_aarch64.whl

Download URL nominal_streaming-0.10.6-cp311-abi3-musllinux_1_2_aarch64.whl
Size 3.9 MB
Tags CPython 3.11 Linux musl 1.2+ ARM64 abi3
SHA-256 checksum
How to use checksums
fc0331e8d3959caf28988bc9b11bf4fd3f58936cbcb6ea5b87c6e8f39ab2ce92
BLAKE2b-256 checksum
How to use checksums
8ce708ea423a5482ef6333b48615c950f4d30c762816b054a09a71c210e39095
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl

Download URL nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Size 4.0 MB
Tags CPython 3.11 Linux glibc 2.17+ x86-64 abi3
SHA-256 checksum
How to use checksums
109f383b1675ea4d215136ef36f2092fb7db8e6171987ac4adaba4d992eecce8
BLAKE2b-256 checksum
How to use checksums
22ac7ca15b5658336e5c8e12cf7f96f007d52efd49f7a287b1ba85ca720f2c86
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl

Download URL nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl
Size 3.5 MB
Tags CPython 3.11 Linux glibc 2.17+ ARMv7l abi3
SHA-256 checksum
How to use checksums
dc7900681840b9a66bcb0d6a36f33bcc42a375d43c6e1fdbd86eb2c5d5b70549
BLAKE2b-256 checksum
How to use checksums
f9c1efa933004c770be7fdeb84f32c0ee556f0ee841a662a89c1b2305a643fb0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl

Download URL nominal_streaming-0.10.6-cp311-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Size 3.8 MB
Tags CPython 3.11 Linux glibc 2.17+ ARM64 abi3
SHA-256 checksum
How to use checksums
db3a0b483707f9bac3817e024f789fb90fb3879607214eaca6bf51b7b3aa95aa
BLAKE2b-256 checksum
How to use checksums
e746935c5e36fec2d14b2663b888b397bc59d62018b67a722e30c6547f71a118
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Release files / nominal_streaming-0.10.6-cp311-abi3-macosx_11_0_arm64.whl

Download URL nominal_streaming-0.10.6-cp311-abi3-macosx_11_0_arm64.whl
Size 3.8 MB
Tags CPython 3.11 abi3 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
2d0bb6cbff66c75b569fca81537e44a4cf40efeed23a3956b2b6a5ac7d910056
BLAKE2b-256 checksum
How to use checksums
d72c844a999fbdf744cdb7ad58ded5ca4b4372c7393463d5736d9e069358eb5c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.24

Security for nominal-streaming 0.10.6

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.10.6 This release

8 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page