zeitreiher-models
Public data models of the Zeitreiher API, version 1: a read-only API over SAP IS-U EDM profile
values (Lastgänge) — profile metadata plus many [from, to) intervals with values.
This repository holds
- Go structs (package
zeitreihermodels, modulegithub.com/hochfrequenz/zeitreiher-models), - Python frozen pydantic v2 models (package
zeitreiher_models), - the shared fixtures in
testdata/that define the JSON contract, and - the gRPC contract in
proto/zeitreiher/v1.
Everything is written by hand. No code is generated and no generated code is committed: every type, validator and (un)marshaller is ordinary, readable code. Both languages parse every valid fixture, write it back and compare the result with the fixture, and reject every invalid fixture — so Go and Python agree on the contract by construction.
The contract in short
| Topic | Rule |
|---|---|
| Intervals | Half-open everywhere: [from, to); from must be before to. |
| Request timestamps | RFC 3339, whole seconds, explicit offset (Z or ±hh:mm), ASCII digits only, hours 00–23. The instant must lie in the years 1900–9999 in UTC, so that it can be echoed in a response (1900-01-01T00:00:00+01:00 is 1899 in UTC and rejected). Kept as sent (+02:00 stays +02:00). A missing offset (2026-01-01T00:00:00) and the "unknown offset" -00:00 are rejected. |
| Response timestamps | RFC 3339, whole seconds, always UTC (Z). Instants built in code must survive the round trip (no offsets with seconds, years 1900–9999). |
| Values | Already-scaled decimal strings in their shortest canonical form ("160", "0.022", "25.25", "-3.5"), never JSON numbers. No exponent, no plus sign, no leading zeros, no trailing zeros after the decimal point ("160.0" is rejected: after parsing it equals "160"), no negative zero. |
| Profile numbers | In requests 1–18 digits, leading zeros allowed, not all zeros; in responses without leading zeros. |
| Units | unit is a canonical unit code: a UN/ECE Recommendation 20 code such as KWH, MWH or K3 (K3 = kvarh) (2 or 3 upper-case ASCII letters or digits, strict). Requests send canonical codes; the server maps them to the customer's SAP unit (an unknown code is a 400; GET /api/v1/system lists the codes the system can map in units). In series records unit is always present and null when the SAP unit has no mapping; the series is delivered anyway. unit_raw (series records and resolve/versions assignments) is the SAP unit key (T006 MSEHI) as SAP delivers it, mapped or not, read leniently: any non-empty text of at most 30 characters (code points) without control characters (Unicode Cc). See the paragraph below the table for a missing raw unit and for IsUnitCode. |
| Series role | Every series record and every resolve/versions assignment without error carries role, the role of its profile assignment (SAP EPROFASS role, e.g. "0001"; any non-empty text), or null when the series was not resolved through a role (e.g. by: profile). For by: malo the roles to deliver are configurable (default: all). |
| Assignments | Resolve and versions assignments describe their profile like the series record: role and unit are always written (null when unset) and unit_raw is required, with the same shapes as in series. An assignment with error may omit all three. |
| System info | GET /api/v1/system reports units (the canonical unit codes the system can map) and caps (limits, all at least 1: max_window_seconds and max_values bound one internal SAP values call and the service splits a query window accordingly; max_resolve_window_seconds is the longest request window the service accepts for query, resolve and versions, a longer window is a 400 invalid_request; versions and resolve are not split and have no value cap; window_too_large never reaches public clients). |
| Selectors | {"by":"profile","profile":"…"}, {"by":"malo","malo":"<11 digits>"}, or a customer resolver {"by":"<snake_case name>","params":{"<key>":"<string>"}} (by is 1 to 30 characters). |
| Version selector | Exactly one of {"current":{}} (default) or {"number":N}; no null members. v1 has no as-of selector: {"as_of":…} is rejected. {"number":N} returns the current values overlaid with the stored values of versions ≥ N (the lowest wins), only inside N's covered range (gaps outside). Each value record says its source: "version":<n> for a stored version row, "version":null for a current value. |
Intervals (interval) |
Sub-daily ISO 8601 durations in minutes: PT15M, PT60M, … Hours are written in minutes: PT60M, never PT1H (rejected, so that every interval has one spelling). |
| Text | Every string is valid Unicode and is written unchanged. Parsers reject invalid UTF-8 and unpaired \uD800–\uDFFF escapes. Go refuses to marshal a string that is not valid UTF-8 (instead of writing U+FFFD); Python rejects a string with a lone surrogate when the model is built (ValidationError). |
| Unknown keys | Requests reject them (a typo fails loudly). Responses ignore them (additive server changes do not break older clients). |
| Duplicate keys | Not rejected: when an object repeats a key, the last occurrence wins and earlier ones are ignored without validation ({"unit":"K H","unit":"MWH"} is read as "MWH"), in both languages and on both sides. Producers must not write duplicate keys; the models never do. Pinned by the *duplicate_key_last_wins* fixtures. |
null |
Request fields that are optional may be omitted or null. In responses, series.role, series.unit, assignment.role and assignment.unit (without error), value.version, end.next, version.to and version.created_at are always present and may be null; other optional response fields are omitted when unset. |
| Error codes | Request-level codes in the error envelope {"error":{"code","message","request_id"?}} — btpingo's vocabulary (invalid_request, unauthorized, forbidden, not_found, method_not_allowed, request_too_large, upstream_unreachable, internal), shared by the Hochfrequenz BTP services, plus cursor_invalid, cursor_mismatch, cursor_expired, cursor_stale and upstream_timeout; record-level codes in error records and resolve/versions results. The known codes are listed in testdata/error_codes.json. Strict on write, lenient on read: parsers accept unknown snake_case codes so that a newer server does not break older clients (Go IsKnown(), Python `RecordErrorCode |
| NDJSON | One record per line, lines end with \n (the last one optionally), no \r, no blank lines. A response that does not end with exactly one end record was cut off and must be treated as a failure. The end record's series, values and errors count the records on this page only; a series record re-sent at the start of a continuation page counts again. |
Units, in detail:
unit_raw is always required and never null.
A backend unit key that is empty (or only control characters) cannot be represented in a series record, so that series is reported as an error record with the code unit_missing instead.
The series_unit_raw_empty, series_unit_raw_null and series_unit_raw_control_char fixtures in testdata/records/invalid/ pin the rejected shapes, and testdata/records/valid/error_unit_missing.json shows the error record.
Consumers can check the canonical unit rule with Go IsUnitCode(s) and Python is_unit_code(value), the same check the models apply to unit.
The shared cases are in testdata/unit_codes.json.
Every public model and field is documented: Python fields carry Field(description=...) (so the descriptions land in the JSON schema from model_json_schema()), Go types, fields and constants carry godoc comments (revive enforces them on exported types, functions and constants; field docs are a convention kept by review), and the two languages are kept in step.
The client-relevant decisions are all in the table above, "The contract in short".
Go
go get github.com/hochfrequenz/zeitreiher-models
import zrm "github.com/hochfrequenz/zeitreiher-models"
cet := time.FixedZone("CET", 3600)
req := zrm.QueryRequest{
Select: []zrm.Selector{zrm.MaLoSelector("87253749908"), zrm.ProfileSelector("50000000000056")},
From: time.Date(2026, 1, 1, 0, 0, 0, 0, cet),
To: time.Date(2026, 2, 1, 0, 0, 0, 0, cet),
Interval: "PT60M",
}
body, err := json.Marshal(req) // validates; an invalid request does not marshal
records, err := zrm.ReadStream(resp.Body) // errors.Is(err, zrm.ErrMissingEnd) for a cut-off stream
for _, rec := range records {
switch r := rec.(type) {
case *zrm.SeriesRecord:
case *zrm.ValueRecord: // r.Value is a zrm.Decimal (string)
case *zrm.ErrorRecord: // r.Code.IsKnown()
case *zrm.EndRecord: // r.Next != nil → request the next page with Cursor = *r.Next
}
}
ReadRecords(io.Reader) yields records one by one (iter.Seq2[Record, error]) for streaming;
WriteRecord writes one NDJSON line. Every type validates itself in UnmarshalJSON and refuses
to marshal when it is invalid.
Python
pip install zeitreiher-models # once published to PyPI
from datetime import datetime, timedelta, timezone
from zeitreiher_models import MaLoSelector, QueryRequest, ValueRecord, iter_records, read_stream
cet = timezone(timedelta(hours=1))
request = QueryRequest(
select=[MaLoSelector(malo="87253749908")],
from_=datetime(2026, 1, 1, tzinfo=cet),
to=datetime(2026, 2, 1, tzinfo=cet),
interval="PT60M",
)
body = request.model_dump_json() # {"select":[…],"from":"2026-01-01T00:00:00+01:00",…}
for record in read_stream(response_bytes): # raises MissingEndError for a cut-off stream
if isinstance(record, ValueRecord):
print(record.from_, record.decimal) # decimal.Decimal, no float rounding
from is a Python keyword, so the attribute is from_; the JSON key is always from.
Fixtures
testdata/
error_codes.json the known request- and record-level codes
unit_codes.json valid and invalid canonical unit codes, shared by IsUnitCode and is_unit_code
query_request/{valid,invalid}/ request bodies (DST days, +02:00 vs Z, naive timestamps, custom selectors, versions, …)
resolve_request/, versions_request/
records/{valid,invalid}/ single NDJSON records, one valid fixture per record-level error code
streams/{valid,invalid}/ whole NDJSON responses (DST days with 92/100 quarter hours, gaps, versions, paging, …)
resolve_response/, versions_response/
error_envelope/{valid,invalid}/ one valid fixture per request-level error code
records/read_only/, error_envelope/read_only/
unknown error codes: they parse, but writing them is refused
system_info/
X.canonical.json next to X.json is the expected re-serialization when it differs from the input
(e.g. +00:00 becomes Z, "cursor": null is omitted, unknown response keys are dropped).
Consumers in other languages (the Go service, future Rust/C#/TS models) should run the same fixtures.
testdata/serialized.golden pins the exact bytes both languages write for every valid fixture
(<fixture>\t<JSON> per line), so Go and Python output is byte-for-byte identical: compact JSON,
keys in model order, map keys (params, attributes) sorted, and Go's string escaping — <, >,
&, U+2028 and U+2029 are written as <, >, &,
,
; all other
non-ASCII characters are written as UTF-8. Python's model_dump_json() and dump_record() apply
that escaping; in Go, json.Marshal, MarshalRecord and a default json.Encoder produce it (an
encoder with SetEscapeHTML(false) would not).
Every invalid fixture has an entry in testdata/invalid_reasons.json, so that a fixture cannot pass by
being rejected for the wrong reason:
go: a substring of the Go error (the field path and the failed rule);python: the pydantic error that must be amongValidationError.errors()— itsloc(joined with.), itstypeand, only for the error types whose messages the models write themselves (value_error,instant,unknown_key,version_selector),msg, a substring of its message. pydantic's own messages are not compared, so rewording them in a pydantic upgrade does not break the fixtures.
Regenerating the pinned files
serialized.golden— after an intended change to the output (or a new valid fixture), rungo test -run Golden -update ., review the diff, thenuv run pytest unitteststo confirm that Python writes the same bytes. The file is only ever written by Go; Python compares against it.invalid_reasons.json— maintained by hand, not generated, because each entry states the intended reason. For a new invalid fixture, the failing Go test prints the actual error (go test -run InvalidFixtures .) and the failing Python test prints a ready-madepythonentry (uv run pytest unittests -k invalid_fixture); check that both name the rule the fixture is about before adding them. Keep the file sorted, two-space indented and ASCII-only (json.dumps(..., indent=2, sort_keys=True, ensure_ascii=True)).- After a pydantic upgrade — run
uv run pytest unittests. A failinginvalid_fixturetest means that pydantic changed an error'slocortype(messages are not compared), or that a fixture is now rejected for a different reason; compare the printederrors()with the entry and update the entry only if the fixture is still rejected for the intended reason.serialized.goldenfailures after an upgrade mean that the output bytes changed and need a fix in the models, not a new golden file.
gRPC
proto/zeitreiher/v1/*.proto (package zeitreiher.v1, service SeriesService with Query,
Resolve, ListVersions, System) is the published gRPC contract. Field names equal the JSON keys.
Request timestamps are RFC 3339 strings (so the explicit-offset rule holds for gRPC too); response
timestamps are google.protobuf.Timestamp. Conformance with the JSON contract is defined through
the hand-written models (fixture → model → proto → model → JSON), not through protojson. Go and
other languages generate code in the consuming project, e.g. with buf managed mode; Python consumers
use the grpc extra (see below) instead.
A failed call (and a Query stream cut off before its EndRecord) ends with a gRPC status that
carries a google.rpc.ErrorInfo detail with reason = the public request-level code and
domain = "zeitreiher.hochfrequenz.de"; series_service.proto lists the status code for each
code. Clients switch on reason.
gRPC from Python
pip install "zeitreiher-models[grpc]"
The extra ships the generated stubs (zeitreiher_models.grpc.v1) and converters between the proto
messages and the models (zeitreiher_models.grpc: query_request_to_proto, record_from_proto,
resolve_response_from_proto, ... in both directions, and error_reason). The base install stays
pydantic-only and never imports grpc or protobuf. Request timestamps stay RFC 3339 strings, response
timestamps become aware UTC datetime, decimal values stay strings.
import grpc
from zeitreiher_models import EndRecord, ErrorRecord, QueryRequest, SeriesRecord, ValueRecord
from zeitreiher_models.grpc import error_reason, query_request_to_proto, record_from_proto
from zeitreiher_models.grpc.v1 import series_service_pb2_grpc
# h2c without TLS (e.g. in a test); for the BTP use grpc.secure_channel(host, grpc.ssl_channel_credentials())
channel = grpc.insecure_channel("localhost:8080")
stub = series_service_pb2_grpc.SeriesServiceStub(channel)
request = QueryRequest.model_validate(
{
"select": [{"by": "profile", "profile": "50000000000056"}],
"from": "2026-01-14T00:00:00+01:00",
"to": "2026-01-15T00:00:00+01:00",
}
)
metadata = (("authorization", "Bearer " + token),) # the token is yours to obtain
complete = False
try:
for message in stub.Query(query_request_to_proto(request), metadata=metadata):
record = record_from_proto(message)
if isinstance(record, EndRecord):
complete = True # record.next is the cursor of the next page, or None
elif isinstance(record, SeriesRecord | ValueRecord | ErrorRecord):
... # collect
except grpc.RpcError as err:
print(err.code(), error_reason(err)) # e.g. cursor_stale; None if the status carries no ErrorInfo
raise
if not complete:
raise RuntimeError("the stream ended without an EndRecord: treat the response as failed")
error_reason reads the google.rpc.ErrorInfo (domain zeitreiher.hochfrequenz.de) of the
grpc-status-details-bin trailer and returns its reason; it returns None when there is none (a
cancelled call, a client deadline, a transport error). Converting a request from proto
(query_request_from_proto) raises the models' pydantic.ValidationError for an invalid message.
Token handling, paging and retries stay with the consumer.
The stubs are generated and committed; regenerate them after a proto change with
uv sync --locked --group codegen --extra grpc && uv run --no-sync python scripts/generate_grpc.py
(Python 3.12 and the exact generator versions are required; the script refuses anything else).
Versioning
v1 is in the URL path (/api/v1/…), the proto package (zeitreiher.v1) and the models' major
version. Additive changes (new optional fields, new error codes) stay in v1; breaking changes need v2.
The Go package zeitreihermodels lives at the repository root (import path
github.com/hochfrequenz/zeitreiher-models); there is no v1 sub-package. Its major version follows
Go module versioning.
While the major version is 0, minor releases may contain breaking changes (announced in the release notes); 1.0 freezes the v1 contract.
From 1.0 on, releases are tags v1.x.y of this module path, and a breaking v2 becomes
the module github.com/hochfrequenz/zeitreiher-models/v2 (tags v2.x.y).
The Python package takes its version from the same tags.
Development
go test ./... # Go
golangci-lint run && golangci-lint fmt --diff
uv sync --group dev --extra grpc # Python
uv run pytest unittests
uv run ruff check src unittests && uv run ruff format --check .
uv run mypy --strict src/zeitreiher_models unittests
uv sync --locked --group codegen --extra grpc && uv run --no-sync python scripts/generate_grpc.py # regenerate the gRPC stubs
buf lint && buf build && buf format --diff --exit-code # buf v1.75.0
The Go tests (fixtures_test.go) and the Python tests (unittests/test_fixtures.py) use the same
table of fixture kinds; a new fixture directory must be added to both. unittests/test_proto_contract.py
fails when a model and its proto message disagree on field names.
Regenerating the stubs (scripts/generate_grpc.py) requires the codegen dependency group from the
lockfile (--locked) and Python 3.12; the script checks the exact versions of grpcio-tools and
mypy-protobuf. CI regenerates and fails on any diff in src/zeitreiher_models/grpc/v1.
unittests/test_grpc_conformance.py ports the Go conformance test of the proto mapping.
Related
Hochfrequenz/mdq-models(public, MIT) holds the canonical schema of the Messdatenqualitätsagent (MQA), written by hand in the same way as this repository.
License
MIT
Metadata
Release files for zeitreiher-models 0.5.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| zeitreiher_models-0.5.1.tar.gz | 200.7 kB | With provenance | Details for zeitreiher_models-0.5.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| zeitreiher_models-0.5.1-py3-none-any.whl | Python 3 | none | any | Details for zeitreiher_models-0.5.1-py3-none-any.whl |
Total release size: 258.8 kB
Release files / zeitreiher_models-0.5.1.tar.gz
| Download URL | zeitreiher_models-0.5.1.tar.gz |
|---|---|
| Size | 200.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
cd9b2f36fed67b1eb4fa937deeebf152e2c653178f8024342a8131ddadef6e76
|
|
BLAKE2b-256 checksum How to use checksums |
da70cc5200a0925e35456ffc1df4b66f64e5f51bbd490884eb060e9ead081978
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 5, 2026.
Transparency logRelease files / zeitreiher_models-0.5.1-py3-none-any.whl
| Download URL | zeitreiher_models-0.5.1-py3-none-any.whl |
|---|---|
| Size | 58.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
af88038799e5df1d83408691d6487c2cacdc270e9db7c3223bb4f13d15b47386
|
|
BLAKE2b-256 checksum How to use checksums |
d274295aad11c1be79bb316b508ba563c34a70e50ec62fbc04e4ee68e031a09d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 5, 2026.
Transparency logSecurity for zeitreiher-models 0.5.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for zeitreiher_models-0.5.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 5, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.