pamoja-codec

CBOR, JSON, and raw codecs behind one trait, delta and varint batch packing, and an f32 quantizer for metered links. One capability of pamoja, one memory-safe Rust core with bindings for TypeScript, Python, and C#.

Install

pip install pamoja-codec

from pamoja import codec

This pulls in pamoja-native, the compiled engine. pip install pamoja is the whole framework in one package.

Example

The script the test suite runs, spliced here as it ran.

From bindings/python/guides/codec.py:

import json

from pamoja.codec import Quantizer, from_cbor, pack_samples, to_cbor, unpack_samples

# The same reading as JSON and as CBOR. Nothing is lost, and 21.5 rides as a
# half-precision float, the shortest form RFC 8949 allows for it.
reading = {"c": 21.5, "ok": True}
as_json = json.dumps(reading, separators=(",", ":")).encode()
cbor = to_cbor(reading)
print(f"json      {len(as_json)} bytes")
print(f"cbor      {len(cbor)} bytes")

# A gateway that speaks JSON gets it back unchanged, so the compact form is a transport
# choice rather than a different data model.
restored = from_cbor(cbor)
print(f"back to json, unchanged: {restored == reading}")

# A batch of readings packs to a count, then the difference between each sample and the
# one before it. Successive readings differ by very little, so the differences cost about
# a byte each where the samples would cost eight.
samples = [10, 11, 13, 12, 900]
packed = pack_samples(samples)
print(f"batch     {len(samples)} samples in {len(packed)} bytes")
print(f"unpacked  {unpack_samples(packed)}")

# Readings that arrive as floats pack the same way once a scale is chosen. Nothing in the
# bytes records that scale, so the sender and the receiver have to agree on it.
quantizer = Quantizer(100)
celsius = [20.0, 20.1, 20.2, 20.3]
packed_celsius = quantizer.encode(celsius)
recovered = quantizer.decode(packed_celsius)
print(f"degrees   {len(celsius)} readings in {len(packed_celsius)} bytes")
print(f"recovered {[round(value, 1) for value in recovered]}")

The same capability in every language

Documentation

• pamoja.codec reference, every class and function in this module.
• The Codecs guide, with the same example in Rust, TypeScript, and C#.
• Every capability, and the install page.

License

MIT