Skip to main content

zeitreiher-models

Go Tests Go Lint Go Reference Python Tests Python Coverage Python Lint Python Formatting Proto

Public data models of the Zeitreiher API, version 1: a read-only API over SAP IS-U EDM profile values (Lastgänge) — profile metadata plus many [from, to) intervals with values.

This repository holds

  • Go structs (package zeitreihermodels, module github.com/hochfrequenz/zeitreiher-models),
  • Python frozen pydantic v2 models (package zeitreiher_models),
  • the shared fixtures in testdata/ that define the JSON contract, and
  • the gRPC contract in proto/zeitreiher/v1.

Everything is written by hand. No code is generated and no generated code is committed: every type, validator and (un)marshaller is ordinary, readable code. Both languages parse every valid fixture, write it back and compare the result with the fixture, and reject every invalid fixture — so Go and Python agree on the contract by construction.

The contract in short

Topic Rule
Intervals Half-open everywhere: [from, to); from must be before to.
Request timestamps RFC 3339, whole seconds, explicit offset (Z or ±hh:mm), ASCII digits only, hours 00–23. The instant must lie in the years 1900–9999 in UTC, so that it can be echoed in a response (1900-01-01T00:00:00+01:00 is 1899 in UTC and rejected). Kept as sent (+02:00 stays +02:00). A missing offset (2026-01-01T00:00:00) and the "unknown offset" -00:00 are rejected.
Response timestamps RFC 3339, whole seconds, always UTC (Z). Instants built in code must survive the round trip (no offsets with seconds, years 1900–9999).
Values Already-scaled decimal strings in their shortest canonical form ("160", "0.022", "25.25", "-3.5"), never JSON numbers. No exponent, no plus sign, no leading zeros, no trailing zeros after the decimal point ("160.0" is rejected: after parsing it equals "160"), no negative zero.
Profile numbers In requests 1–18 digits, leading zeros allowed, not all zeros; in responses without leading zeros.
Units unit is a canonical unit code: a UN/ECE Recommendation 20 code such as KWH, MWH or K3 (K3 = kvarh) (2 or 3 upper-case ASCII letters or digits, strict). Requests send canonical codes; the server maps them to the customer's SAP unit (an unknown code is a 400; GET /api/v1/system lists the codes the system can map in units). In series records unit is always present and null when the SAP unit has no mapping; the series is delivered anyway. unit_raw (series records and resolve/versions assignments) is the SAP unit key (T006 MSEHI) as SAP delivers it, mapped or not, read leniently: any non-empty text of at most 30 characters (code points) without control characters (Unicode Cc). See the paragraph below the table for a missing raw unit and for IsUnitCode.
Series role Every series record and every resolve/versions assignment without error carries role, the role of its profile assignment (SAP EPROFASS role, e.g. "0001"; any non-empty text), or null when the series was not resolved through a role (e.g. by: profile). For by: malo the roles to deliver are configurable (default: all).
Assignments Resolve and versions assignments describe their profile like the series record: role and unit are always written (null when unset) and unit_raw is required, with the same shapes as in series. An assignment with error may omit all three.
System info GET /api/v1/system reports units (the canonical unit codes the system can map) and caps (limits, all at least 1: max_window_seconds and max_values bound one internal SAP values call and the service splits a query window accordingly; max_resolve_window_seconds is the longest request window the service accepts for query, resolve and versions, a longer window is a 400 invalid_request; versions and resolve are not split and have no value cap; window_too_large never reaches public clients). The optional abap_* members are described below the table.
Selectors {"by":"profile","profile":"…"}, {"by":"malo","malo":"<11 digits>"}, or a customer resolver {"by":"<snake_case name>","params":{"<key>":"<string>"}} (by is 1 to 30 characters).
Version selector Exactly one of {"current":{}} (default) or {"number":N}; no null members. v1 has no as-of selector: {"as_of":…} is rejected. {"number":N} returns the current values overlaid with the stored values of versions ≥ N (the lowest wins), only inside N's covered range (gaps outside). Each value record says its source: "version":<n> for a stored version row, "version":null for a current value.
Intervals (interval) Sub-daily ISO 8601 durations in minutes: PT15M, PT60M, … Hours are written in minutes: PT60M, never PT1H (rejected, so that every interval has one spelling).
Text Every string is valid Unicode and is written unchanged. Parsers reject invalid UTF-8 and unpaired \uD800–\uDFFF escapes. Go refuses to marshal a string that is not valid UTF-8 (instead of writing U+FFFD); Python rejects a string with a lone surrogate when the model is built (ValidationError).
Unknown keys Requests reject them (a typo fails loudly). Responses ignore them (additive server changes do not break older clients).
Duplicate keys Not rejected: when an object repeats a key, the last occurrence wins and earlier ones are ignored without validation ({"unit":"K H","unit":"MWH"} is read as "MWH"), in both languages and on both sides. Producers must not write duplicate keys; the models never do. Pinned by the *duplicate_key_last_wins* fixtures.
null Request fields that are optional may be omitted or null. In responses, series.role, series.unit, assignment.role and assignment.unit (without error), value.version, end.next, version.to and version.created_at are always present and may be null; other optional response fields are omitted when unset.
Error codes Request-level codes in the error envelope {"error":{"code","message","request_id"?}} — btpingo's vocabulary (invalid_request, unauthorized, forbidden, not_found, method_not_allowed, request_too_large, upstream_unreachable, internal), shared by the Hochfrequenz BTP services, plus cursor_invalid, cursor_mismatch, cursor_expired, cursor_stale and upstream_timeout; record-level codes in error records and resolve/versions results. The known codes are listed in testdata/error_codes.json. Strict on write, lenient on read: parsers accept unknown snake_case codes so that a newer server does not break older clients (Go IsKnown(), Python `RecordErrorCode
NDJSON One record per line, lines end with \n (the last one optionally), no \r, no blank lines. A response that does not end with exactly one end record was cut off and must be treated as a failure. The end record's series, values and errors count the records on this page only; a series record re-sent at the start of a continuation page counts again.

Units, in detail: unit_raw is always required and never null. A backend unit key that is empty (or only control characters) cannot be represented in a series record, so that series is reported as an error record with the code unit_missing instead. The series_unit_raw_empty, series_unit_raw_null and series_unit_raw_control_char fixtures in testdata/records/invalid/ pin the rejected shapes, and testdata/records/valid/error_unit_missing.json shows the error record. Consumers can check the canonical unit rule with Go IsUnitCode(s) and Python is_unit_code(value), the same check the models apply to unit. The shared cases are in testdata/unit_codes.json.

System info, the ABAP version state: The three abap_* members pass the version state of the ABAP backend through. All three are optional: older services and backends omit them (an explicit null counts as absent), unknown members stay ignored, and an unset member is omitted when writing. abap_last_source_change_utc is an instant (RFC 3339 with Z, whole seconds, like every response instant); absent and null both mean unknown. abap_inactive_objects is {"available": true, "count": n, "names": [...]} (count at least 0 and equal to the number of names) or {"available": false, "reason": "..."} (non-empty reason, no count or names). abap_source_fingerprint is the aggregate SHA-256 of the ABAP sources: exactly 64 lower-case hex characters. In Go they are SystemInfo.AbapLastSourceChangeUTC (*time.Time), SystemInfo.AbapInactiveObjects (*InactiveObjects) and SystemInfo.AbapSourceFingerprint (*string); in Python abap_last_source_change_utc, abap_inactive_objects (InactiveObjects) and abap_source_fingerprint. In SystemResponse they are fields 10 to 12 (a Timestamp, an InactiveObjects message and an optional string).

Every public model and field is documented: Python fields carry Field(description=...) (so the descriptions land in the JSON schema from model_json_schema()), Go types, fields and constants carry godoc comments (revive enforces them on exported types, functions and constants; field docs are a convention kept by review), and the two languages are kept in step.

The client-relevant decisions are all in the table above, "The contract in short".

Go

go get github.com/hochfrequenz/zeitreiher-models
import zrm "github.com/hochfrequenz/zeitreiher-models"

cet := time.FixedZone("CET", 3600)
req := zrm.QueryRequest{
	Select:   []zrm.Selector{zrm.MaLoSelector("87253749908"), zrm.ProfileSelector("50000000000056")},
	From:     time.Date(2026, 1, 1, 0, 0, 0, 0, cet),
	To:       time.Date(2026, 2, 1, 0, 0, 0, 0, cet),
	Interval: "PT60M",
}
body, err := json.Marshal(req) // validates; an invalid request does not marshal

records, err := zrm.ReadStream(resp.Body) // errors.Is(err, zrm.ErrMissingEnd) for a cut-off stream
for _, rec := range records {
	switch r := rec.(type) {
	case *zrm.SeriesRecord:
	case *zrm.ValueRecord: // r.Value is a zrm.Decimal (string)
	case *zrm.ErrorRecord: // r.Code.IsKnown()
	case *zrm.EndRecord: // r.Next != nil → request the next page with Cursor = *r.Next
	}
}

ReadRecords(io.Reader) yields records one by one (iter.Seq2[Record, error]) for streaming; WriteRecord writes one NDJSON line. Every type validates itself in UnmarshalJSON and refuses to marshal when it is invalid.

Python

pip install zeitreiher-models  # once published to PyPI
from datetime import datetime, timedelta, timezone

from zeitreiher_models import MaLoSelector, QueryRequest, ValueRecord, iter_records, read_stream

cet = timezone(timedelta(hours=1))
request = QueryRequest(
    select=[MaLoSelector(malo="87253749908")],
    from_=datetime(2026, 1, 1, tzinfo=cet),
    to=datetime(2026, 2, 1, tzinfo=cet),
    interval="PT60M",
)
body = request.model_dump_json()  # {"select":[…],"from":"2026-01-01T00:00:00+01:00",…}

for record in read_stream(response_bytes):  # raises MissingEndError for a cut-off stream
    if isinstance(record, ValueRecord):
        print(record.from_, record.decimal)  # decimal.Decimal, no float rounding

from is a Python keyword, so the attribute is from_; the JSON key is always from.

Fixtures

testdata/
  error_codes.json                       the known request- and record-level codes
  unit_codes.json                        valid and invalid canonical unit codes, shared by IsUnitCode and is_unit_code
  query_request/{valid,invalid}/         request bodies (DST days, +02:00 vs Z, naive timestamps, custom selectors, versions, …)
  resolve_request/, versions_request/
  records/{valid,invalid}/               single NDJSON records, one valid fixture per record-level error code
  streams/{valid,invalid}/               whole NDJSON responses (DST days with 92/100 quarter hours, gaps, versions, paging, …)
  resolve_response/, versions_response/
  error_envelope/{valid,invalid}/        one valid fixture per request-level error code
  records/read_only/, error_envelope/read_only/
                                         unknown error codes: they parse, but writing them is refused
  system_info/

X.canonical.json next to X.json is the expected re-serialization when it differs from the input (e.g. +00:00 becomes Z, "cursor": null is omitted, unknown response keys are dropped). Consumers in other languages (the Go service, future Rust/C#/TS models) should run the same fixtures.

testdata/serialized.golden pins the exact bytes both languages write for every valid fixture (<fixture>\t<JSON> per line), so Go and Python output is byte-for-byte identical: compact JSON, keys in model order, map keys (params, attributes) sorted, and Go's string escaping — <, >, &, U+2028 and U+2029 are written as <, >, &, 
, 
; all other non-ASCII characters are written as UTF-8. Python's model_dump_json() and dump_record() apply that escaping; in Go, json.Marshal, MarshalRecord and a default json.Encoder produce it (an encoder with SetEscapeHTML(false) would not).

Every invalid fixture has an entry in testdata/invalid_reasons.json, so that a fixture cannot pass by being rejected for the wrong reason:

  • go: a substring of the Go error (the field path and the failed rule);
  • python: the pydantic error that must be among ValidationError.errors() — its loc (joined with .), its type and, only for the error types whose messages the models write themselves (value_error, instant, unknown_key, version_selector), msg, a substring of its message. pydantic's own messages are not compared, so rewording them in a pydantic upgrade does not break the fixtures.

Regenerating the pinned files

  • serialized.golden — after an intended change to the output (or a new valid fixture), run go test -run Golden -update ., review the diff, then uv run pytest unittests to confirm that Python writes the same bytes. The file is only ever written by Go; Python compares against it.
  • invalid_reasons.json — maintained by hand, not generated, because each entry states the intended reason. For a new invalid fixture, the failing Go test prints the actual error (go test -run InvalidFixtures .) and the failing Python test prints a ready-made python entry (uv run pytest unittests -k invalid_fixture); check that both name the rule the fixture is about before adding them. Keep the file sorted, two-space indented and ASCII-only (json.dumps(..., indent=2, sort_keys=True, ensure_ascii=True)).
  • After a pydantic upgrade — run uv run pytest unittests. A failing invalid_fixture test means that pydantic changed an error's loc or type (messages are not compared), or that a fixture is now rejected for a different reason; compare the printed errors() with the entry and update the entry only if the fixture is still rejected for the intended reason. serialized.golden failures after an upgrade mean that the output bytes changed and need a fix in the models, not a new golden file.

gRPC

proto/zeitreiher/v1/*.proto (package zeitreiher.v1, service SeriesService with Query, Resolve, ListVersions, System) is the published gRPC contract. Field names equal the JSON keys. Request timestamps are RFC 3339 strings (so the explicit-offset rule holds for gRPC too); response timestamps are google.protobuf.Timestamp. Conformance with the JSON contract is defined through the hand-written models (fixture → model → proto → model → JSON), not through protojson. Go and other languages generate code in the consuming project, e.g. with buf managed mode; Python consumers use the grpc extra (see below) instead.

A failed call (and a Query stream cut off before its EndRecord) ends with a gRPC status that carries a google.rpc.ErrorInfo detail with reason = the public request-level code and domain = "zeitreiher.hochfrequenz.de"; series_service.proto lists the status code for each code. Clients switch on reason when it is present. Errors of the RPC framework itself (malformed message INVALID_ARGUMENT, oversized body RESOURCE_EXHAUSTED, unknown method UNIMPLEMENTED) and calls ended by the client (CANCELED, a client-set DEADLINE_EXCEEDED) carry no ErrorInfo; clients switch on the status code for those.

gRPC from Python

pip install "zeitreiher-models[grpc]"

The extra ships the generated stubs (zeitreiher_models.grpc.v1) and converters between the proto messages and the models (zeitreiher_models.grpc: query_request_to_proto, record_from_proto, resolve_response_from_proto, ... in both directions, and error_reason). The base install stays pydantic-only and never imports grpc or protobuf. Request timestamps stay RFC 3339 strings, response timestamps become aware UTC datetime, decimal values stay strings.

import grpc

from zeitreiher_models import EndRecord, ErrorRecord, QueryRequest, SeriesRecord, ValueRecord
from zeitreiher_models.grpc import error_reason, query_request_to_proto, record_from_proto
from zeitreiher_models.grpc.v1 import series_service_pb2_grpc

# h2c without TLS (e.g. in a test); for the BTP use grpc.secure_channel(host, grpc.ssl_channel_credentials())
channel = grpc.insecure_channel("localhost:8080")
stub = series_service_pb2_grpc.SeriesServiceStub(channel)
request = QueryRequest.model_validate(
    {
        "select": [{"by": "profile", "profile": "50000000000056"}],
        "from": "2026-01-14T00:00:00+01:00",
        "to": "2026-01-15T00:00:00+01:00",
    }
)
metadata = (("authorization", "Bearer " + token),)  # the token is yours to obtain
complete = False
try:
    for message in stub.Query(query_request_to_proto(request), metadata=metadata):
        record = record_from_proto(message)
        if isinstance(record, EndRecord):
            complete = True  # record.next is the cursor of the next page, or None
        elif isinstance(record, SeriesRecord | ValueRecord | ErrorRecord):
            ...  # collect
except grpc.RpcError as err:
    print(err.code(), error_reason(err))  # e.g. cursor_stale; None if the status carries no ErrorInfo
    raise
if not complete:
    raise RuntimeError("the stream ended without an EndRecord: treat the response as failed")

error_reason reads the google.rpc.ErrorInfo (domain zeitreiher.hochfrequenz.de) of the grpc-status-details-bin trailer and returns its reason; it returns None when there is none (a cancelled call, a client deadline, a transport error). Converting a request from proto (query_request_from_proto) raises the models' pydantic.ValidationError for an invalid message. Token handling, paging and retries stay with the consumer.

The stubs are generated and committed; regenerate them after a proto change with uv sync --locked --group codegen --extra grpc && uv run --no-sync python scripts/generate_grpc.py (the Python version and the exact generator versions pinned in the repo are required; the script refuses anything else).

Versioning

v1 is in the URL path (/api/v1/…), the proto package (zeitreiher.v1) and the models' major version. Additive changes (new optional fields, new error codes) stay in v1; breaking changes need v2.

The Go package zeitreihermodels lives at the repository root (import path github.com/hochfrequenz/zeitreiher-models); there is no v1 sub-package. Its major version follows Go module versioning. While the major version is 0, minor releases may contain breaking changes (announced in the release notes); 1.0 freezes the v1 contract. From 1.0 on, releases are tags v1.x.y of this module path, and a breaking v2 becomes the module github.com/hochfrequenz/zeitreiher-models/v2 (tags v2.x.y). The Python package takes its version from the same tags.

Development

go test ./...                      # Go
golangci-lint run && golangci-lint fmt --diff
uv sync --group dev --extra grpc  # Python
uv run pytest unittests
uv run ruff check src unittests && uv run ruff format --check .
uv run mypy --strict src/zeitreiher_models unittests
uv sync --locked --group codegen --extra grpc && uv run --no-sync python scripts/generate_grpc.py  # regenerate the gRPC stubs
buf lint && buf build && buf format --diff --exit-code   # buf version as in CI

The Go tests (fixtures_test.go) and the Python tests (unittests/test_fixtures.py) use the same table of fixture kinds; a new fixture directory must be added to both. unittests/test_proto_contract.py fails when a model and its proto message disagree on field names.

Regenerating the stubs (scripts/generate_grpc.py) requires the codegen dependency group from the lockfile (--locked) and the pinned Python version; the script checks the exact versions of grpcio-tools and mypy-protobuf. CI regenerates and fails on any diff in src/zeitreiher_models/grpc/v1. unittests/test_grpc_conformance.py ports the Go conformance test of the proto mapping.

  • Hochfrequenz/mdq-models (public, MIT) holds the canonical schema of the Messdatenqualitätsagent (MQA), written by hand in the same way as this repository.

License

MIT

Metadata

Release files for zeitreiher-models 0.6.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for zeitreiher-models 0.6.0
File Size Uploaded Provenance Actions
zeitreiher_models-0.6.0.tar.gz 197.0 kB With provenance Details for zeitreiher_models-0.6.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for zeitreiher-models 0.6.0
File Interpreter ABI Platform Actions
zeitreiher_models-0.6.0-py3-none-any.whl Python 3 none any Details for zeitreiher_models-0.6.0-py3-none-any.whl

Total release size: 258.6 kB

Release files / zeitreiher_models-0.6.0.tar.gz

Download URL zeitreiher_models-0.6.0.tar.gz
Size 197.0 kB
Tags Source
SHA-256 checksum
How to use checksums
148beb750b967e4e46f7f4ec705eea8b39d01bcc7df614596d32228570e63bf2
BLAKE2b-256 checksum
How to use checksums
c8f04e145fd0ef729badf8be6aec4f220406d1ff9698f227405a4115cdd01eb6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.

Transparency log

Release files / zeitreiher_models-0.6.0-py3-none-any.whl

Download URL zeitreiher_models-0.6.0-py3-none-any.whl
Size 61.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
9581d22939ffbfe624399cd77451929a735a6ecfeb8f1ae5c963905eef992b03
BLAKE2b-256 checksum
How to use checksums
0263f44de18e10da1e8ead7be0f967f9ab67b07b61bb06dfec58898f0adbef61
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.

Transparency log

Security for zeitreiher-models 0.6.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for zeitreiher_models-0.6.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page