SPOS stands for Small Payload Object Serializer
Project description
SPOS
SPOS stands for Small Payload Object Serializer.
SPOS
is a tool for serializing objects. This tool focuses in
maintaining a consistent payload size while sacrificing precision.
Applications with limited bandwidth like LoRa
or Globalstar are ideal candidates
for SPOS
. SPOS
is built as a library for python3
and a command line
tool.
Quick Start
To encode data, SPOS
needs two arguments to serialize the data: The
payload_data
to be serialized and the payload specification.
import spos
payload_spec = {
"name": "example payload",
"version": "1.0.0",
"items": [{
"type": "integer",
"key": "payload_version",
"value": 1, # 01
"bits": 2
}, {
"type": "integer",
"key": "int_data",
"bits": 6
}, {
"type": "float",
"key": "float_data",
"bits": 6
}]
payload_data = {
"int_data": 13, # 001101
"float_data": 0.6 # 010011 (19/32 or 0.59375)
}
message = spos.encode(payload_data, payload_spec)
"0b01001101010011"
Decoding data
import spos
payload_spec = {
"name": "example payload",
"version": "1.0.0",
"items": [{
"type": "integer",
"key": "payload_version",
"value": 1, # 01
"bits": 2
}, {
"type": "integer",
"key": "int_data",
"bits": 6
}, {
"type": "float",
"key": "float_data",
"bits": 6
}]
message = "0b01001101010011"
payload_data = spos.decode(message, payload_spec)
{
"payload_version": 1,
"int_data": 13,
"float_data": 0.59375
}
Payload Specification
The payload specification must contain the key items
, which must be
an array containing objects that describe each block
of the
serialized message.
payload_spec = {
"items": [{
"type:": "integer",
"key": "temperature",
"bits": 6,
"offset": 273
}]}
Additional keys can be provided and keys not listed are ignored.
Payload Specification keys
items
The array of blocks
describing the message.
name
Message name
version
Message version
Block
The required keys for block
objects are: key
and type
. value
is
an optional key. For each type
there might be aditional required keys
and/or optional keys.
The value to be encoded is either a key
in the payload_data
object or
a statuc value
if present.
The encoded data is big-endian and truncations occour in the least significant bits when applicable. Data overflow is set to the maximum value and underflow to the minimum.
Block keys
key
The key to get the value of the block
in payload_data
.
value
A static value for the block
(optional).
type
There are 11 types avaliable for serializing data: boolean
, binary
,
integer
, float
, pad
, array
, object
, string
, steps
,
categories
, and crc8
.
The basic types are:
boolean
Boolean value.
Input: boolean
.
Additional keys: None
.
binary
Binary value. The data can be a binary string or an hex string. Eg
"0b10101010" # binary
"0xdeadbeef" # hex
This data is truncated in the least significant bits if the size of
the string in binary is bigger than bits
.
Input: string
.
Additional keys:
bits
(int): length of the block in bits
integer
Integer value.
Input: integer
.
Additional key:
bits
(int): length of the block in bitsoffset
(int): An integer to offset the final value. Default: 0.
float
Float value.
This type divides the interval between the lower
and upper
boundaries in equal parts according to the avaliable bits
. The
serialized value is the closest to the real one by default
("approximation": "round").
Input: int|float
.
Additional keys:
bits
(int): length of the block in bitslower
(int|float), optional: Float lower boundary. Default 0.upper
(int|float), optional: Float upper boundary. Default 1.approximation
(str), optional: Float approximation method. Values can be: "round", "floor", "ceil". Default: "round"
pad
Pads the message. No data is collected from this block.
Input: None
.
Additional keys:
bits
(int): length of the block in bits
Advanced types:
array
An array containing block
values.
The size in bits of this type is bits
+ length
* blocks
→ bits
.
Input: An array
of values allowed for blocks
.
Additional keys:
bits
(int): Number of bits to store the maximum length of the array.blocks
(block): Theblock
specification of the objects in the array.
object
Object value. Maps the data to an object.
The size in bits of this type is the sum of sizes of blocks declared
for this block
.
Input: object
.
Additional keys:
items
(items): Thearray
ofblocks
describing the object.
string
String value.
This data type encodes the string to base64. Characters outside the
base64 index table
are replaced with /
(index 62) and spaces are replaced with +
(index 63).
The size in bits of this type is 6 * length
.
Input: string
.
Additional keys:
length
(int): Strings maximum length.custom_alphabeth
(object), optional: Remaps the characters to another index. eg: Adding support for ajson
string but sacrificing the first 7 indexes (ABCDEFG).
payload_spec = {
"items": [{
"type:": "string",
"key": "text",
"length": 128,
"custom_alphabeth": {
0: "{",
1: "}",
2: "[",
3: "]",
4: '"',
5: ',',
6: '.',
}
}]
steps
Maps a numeric value to named steps. Eg:
payload_spec = {
"items": [{
"type:": "steps",
"key": "battery",
"steps": [0.1, 0.6, 0.95],
"steps_names": ["critical", "low", "discharging", "charged"]
# [-Inf, 0.1) critical, [0.1, 0.6) low, [0.6, 0.95) discharging, [0.95, Inf) charged
}]
payload_data = {"bat": 0.3} # low
The number of bits for this type is the exponent of the next power of
two of the length of steps
+ 1. In the example above it is 2 bits.
Input: int|float
.
Additional keys:
steps
(array): Array listing the boundaries of each step.steps_names
(array), optional: Names for each step. If not provided the names are created based on steps.
categories
Maps strings to categories: Eg:
payload_spec = {
"items": [{
"type:": "categories",
"key": "color",
"categories": ["red", "green", "blue", "iridescent"],
}]
payload_data = {"color": "red"} # low
The number of bits for this type is the exponent of the next power of
two of the length of steps
+ 1. In the example above it is 2 bits.
The category "unknown" is added to represent data that are not present
in the categories
array.
Input: string
.
Additional keys:
categories
(array): The array of categories strings.
crc8
Calculates the CRC8
for the message. The size in bits for this block
is always 8.
Input: None
.
Additional keys: None
.
Functions
def encode(payload_data, payload_spec):
"""
Encodes a message from payload_data according to payload_spec.
Returns the message as a binary string.
Args:
payload_data (dict): The list of values to encode.
payload_spec (dict): Payload specifications.
Returns:
message (str): Binary string of the message.
"""
def decode(message, payload_spec):
"""
Decodes a binary message according to payload_spec.
Args:
message (str): Binary string of the message.
payload_spec (dict): Payload specifications.
Returns:
payload_data (dict): Payload data.
"""
def hex_encode(payload_data, payload_spec):
"""
Encodes a message from payload_data according to payload_spec.
Returns the message as an hex string.
Args:
payload_data (dict): The list of values to encode.
payload_spec (dict): Payload specifications.
Returns:
message (str): Binary string of the message.
"""
def hex_decode(message, payload_spec):
"""
Decodes an hex message according to payload_spec.
Args:
message (str): Hex string of the message.
payload_spec (dict): Payload specifications.
Returns:
payload_data (dict): Payload data.
"""
Command line usage
# Encode data
cat payload_data | spos -p payload_spec.json
# Decode data
cat message | spos -d -p payload_spec.json
Contributors
License
SPOS - Small Payload Object Serializer Copyright (C) 2020 Luiz Eduardo Amaral luizamaral306@gmail.com
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.