pygar-client 1.5.1

A Python client for the GAR protocol

GAR Protocol

Messages are serialized using an underlying transport — currently ZeroMQ.

Binary or JSON message format is supported by the server.

Format Detection

• JSON is inferred if the first character of the introduction message is {.
• Otherwise, binary is assumed.

Binary Messages

Start with a message_type enum, followed by the appropriate C struct. These are aligned C structures generated by trscc. See gar_proto.h.

JSON Messages

JSON messages use the format:

{
  "message_type": "<message_type enum name>",
  "value": { ... }
}

Example:

{
  "message_type": "Subscribe",
  "value": {
    "subscription_mode": "Snapshot",
    "nagle_interval": 0,
    "name": "S1",
    "key_id": 0,
    "topic_id": 0,
    "_class": "Underlier",
    "key_filter": null,
    "topic_filter": null
  }
}

Session Structure

• The first message must be an Introduction.
• Heartbeats must be sent regularly within heartbeat_timeout_interval.
• First heartbeat gets a 10x grace interval.

Enumeration

• Topics and Keys must be introduced before publishing records.
• Clients and servers may use different enumerations.
• 0 is an invalid ID.
• Clients must map IDs between client and server contexts.

Example:
Client assigns key "AAPL" → ID 2
Server assigns same → ID 3
Client sends DeleteKey with ID 2
Receives DeleteKey from server with ID 3

Records

• NewRecord must precede updates.
• Records may exist without values.
• Use DeleteRecord to remove empty ones.

Record Updates

Binary:

• Use FixedLengthRecordUpdate or VariableLengthRecordUpdate
• Fixed: starts with record_id followed by data
• Variable: serializes binary_record_update, includes size in first member

JSON:

{
  "message_type": "JSONRecordUpdate",
  "value": {
    "record_id": { "key_id": 1, "topic_id": 22 },
    "value": 0.3
  }
}

Example Session

Sent: {"message_type": "Introduction", "value": { "version": 650269, "heartbeat_timeout_interval": 3000, "user": "jonh" }}
Received: {"message_type": "Introduction", "value": { "version": 650269, "heartbeat_timeout_interval": 3000, "user": "jserver" }}
Sent: {"message_type": "Subscribe", "value": { "subscription_mode": "Streaming", "nagle_interval": 0, "name": "S1", "key_id": 0, "topic_id": 0, "_class": "Underlier", "key_filter": null, "topic_filter": null }}
Received: Topic and Key Introductions
Received: NewRecord, JSONRecordUpdate
Received: SnapshotComplete
Sent/Received: Heartbeats
Received: DeleteRecord, DeleteKey
Sent: Logoff

Publication Rules

• Server publishes only subscribed messages.
• Clients may publish any messages but must introduce topics/keys first.

---

Protocol Schema

enum message_type : int64_t {
    Introduction = 'A',
    Heartbeat = 'B',
    Logoff = 'C',
    TopicIntroduction = 'D',
    KeyIntroduction = 'E',
    DeleteKey = 'F',
    Subscribe = 'G',
    SnapshotComplete = 'H',
    Unsubscribe = 'I',
    NewRecord = 'J',
    DeleteRecord = 'K',
    FixedLengthRecordUpdate = 'L',
    VariableLengthRecordUpdate = 'M',
    JSONRecordUpdate = 'N',
    Shutdown = 'O'
};

type u_milliseconds = uint64_t;

struct introduction {
    version : int;
    heartbeat_timeout_interval : u_milliseconds;
    user : string;
    schema : optional string;
};

type topic_id = unsigned;

struct topic_introduction {
    topic_id;
    name : string;
};

type key_id = unsigned;

struct key_introduction {
    key_id;
    name : string;
    _class : optional string;
};

struct delete_key {
    key_id;
};

enum subscription_mode { *None, Snapshot, Streaming, Throttled };

type regex text;

struct subscribe {
    subscription_mode;
    nagle_interval : u_milliseconds;
    name : string;
    key_id;
    topic_id;
    _class : optional string;
    key_filter : optional regex;
    topic_filter : optional regex;
};

struct snapshot_complete {
    name : string;
};

struct record_id {
    key_id;
    topic_id;
};

type byte = unsigned char;

struct json_record_update {
    record_id;
    value : json;
};

struct binary_record_update {
    record_id;
    data : byte[];
};

struct unsubscribe {
    name : string;
};