pbtools 0.36.0

Google Protocol Buffers tools.

About

Google Protocol Buffers tools in Python 3.6+.

• C source code generator.

• proto3 language parser.

Known limitations:

• Options, services (gRPC) and reserved fields are ignored.

• Public imports are not implemented.

Project homepage: https://github.com/eerimoq/pbtools

Documentation: https://pbtools.readthedocs.io

Installation

pip install pbtools

C source code design

The C source code is designed with the following in mind:

• Clean and easy to use API.

• No malloc/free. Uses a workspace/arena for memory allocations.

• Fast encoding and decoding.

• Small memory footprint.

• Thread safety.

Known limitations:

• char must be 8 bits.

Memory management

A workspace, or arena, is used to allocate memory when encoding and decoding messages. For simplicity, allocated memory can’t be freed, which puts restrictions on how a message can be modified between encodings (if one want to do that). Scalar value type fields (ints, strings, bytes, etc.) can be modified, but the length of repeated fields can’t.

Benchmark

See benchmark for a benchmark of a few C/C++ protobuf libraries.

Example usage

C source code

In this example we use the simple proto-file hello_world.proto.

syntax = "proto3";

package hello_world;

message Foo {
    int32 bar = 1;
}

Generate C source code from the proto-file.

$ pbtools generate_c_source examples/hello_world/hello_world.proto

See hello_world.h and hello_world.c for the contents of the generated files.

We’ll use the generated types and functions below.

struct hello_world_foo_t {
   struct pbtools_message_base_t base;
   int32_t bar;
};

struct hello_world_foo_t *hello_world_foo_new(
    void *workspace_p,
    size_t size);

int hello_world_foo_encode(
    struct hello_world_foo_t *self_p,
    void *encoded_p,
    size_t size);

int hello_world_foo_decode(
    struct hello_world_foo_t *self_p,
    const uint8_t *encoded_p,
    size_t size);

Encode and decode the Foo-message in main.c.

#include <stdio.h>
#include "hello_world.h"

int main(int argc, const char *argv[])
{
    int size;
    uint8_t workspace[64];
    uint8_t encoded[16];
    struct hello_world_foo_t *foo_p;

    /* Encode. */
    foo_p = hello_world_foo_new(&workspace[0], sizeof(workspace));

    if (foo_p == NULL) {
        return (1);
    }

    foo_p->bar = 78;
    size = hello_world_foo_encode(foo_p, &encoded[0], sizeof(encoded));

    if (size < 0) {
        return (2);
    }

    printf("Successfully encoded Foo into %d bytes.\n", size);

    /* Decode. */
    foo_p = hello_world_foo_new(&workspace[0], sizeof(workspace));

    if (foo_p == NULL) {
        return (3);
    }

    size = hello_world_foo_decode(foo_p, &encoded[0], size);

    if (size < 0) {
        return (4);
    }

    printf("Successfully decoded %d bytes into Foo.\n", size);
    printf("Foo.bar: %d\n", foo_p->bar);

    return (0);
}

Build and run the program.

$ gcc -I lib/include main.c hello_world.c lib/src/pbtools.c -o main
$ ./main
Successfully encoded Foo into 2 bytes.
Successfully decoded 2 bytes into Foo.
Foo.bar: 78

See examples/hello_world for all files used in this example.

Command line tool

The generate C source subcommand

Below is an example of how to generate C source code from a proto-file.

$ pbtools generate_c_source examples/address_book/address_book.proto

See address_book.h and address_book.c for the contents of the generated files.