Tuber Server and Client

Tuber is a C++ server and Python client for exposing an instrumentation control plane across a network.

On a client, you can write Python code like this:

>>> some_resource.increment([1, 2, 3, 4, 5])
[2, 3, 4, 5, 6]

…and end up with a remote method call on a networked resource written in Python or (more usually) C++. The C++ implementation might look like this:

class SomeResource {
public:
    std::vector<int> increment(std::vector<int> x) {
        std::ranges::for_each(x, [](int &n) { n++; });
        return x;
    };
};

On the client side, Python needs to know where to find the server. On the server side, the C++ code must be registered with pybind11 (just as any other pybind11 code) and the tuber server. Other than that, however, there is no ceremony and no boilerplate.

Its main features and design principles are:

• Pythonic call styles, including *args, **kwargs, and DocStrings.

• JSON and CBOR support for efficient and friendly serialization of return values.

• “Less-is-more” approach to code. For example, Tuber uses pybind11 and C++ as a shim between C and Python, because the combination gives us the shortest and most expressive way to produce the results we want. It pairs excellently with orjson (as a JSON interface) or cbor2 (as a CBOR interface), which efficiently serialize (for example) NumPy arrays created in C++ across the network.

• Schema-less RPC using standard-ish protocols (HTTP 1.1, JSON, CBOR, and something like JSON-RPC). Avoiding a schema allows server and client code to be independently and seamlessly up- and downgraded, with differences between exposed APIs only visible at the sites where RPC calls are made.

• A mature, quick-ish, third-party, low-overhead, low-prerequisite embedded webserver. Tuber uses cpp-httplib, a single-header C++ library vendored into the tree, so the server builds without any external HTTP dependencies. Its blocking, thread-per-connection style suits us because a single keep-alive connection with a single client is the expected “hot path”; C10K-style server architectures wouldn’t be better.

• High performance when communicating with RPC endpoints, using:

  • HTTP/1.1 Keep-Alive to avoid single-connection-per-request overhead. See this Wikipedia page for details.

  • A call API that (optionally) allows multiple RPC calls to be combined and dispatched together.

  • Client-side caches for remote properties (server-side constants)

  • Python 3.x’s aiohttp/asyncio libraries to asynchronously dispatch across multiple endpoints (e.g. multiple boards in a crate, each of which is an independent Tuber endpoint.)

• A friendly interactive experience using Jupyter/IPython-style REPL environments. Tuber servers export metadata that can be used to provide DocStrings and tab-completion for RPC resources.

• The ability to serve a web-based UI using static JavaScript, CSS, and HTML.

Anti-goals of this Tuber server include the following:

• No authentication/encryption is used. For now, network security is strictly out-of-scope. (Yes, it feels naïve to write this in 2022.)

• The additional complexity of HTTP/2 and HTTP/3 protocols are not justified. HTTP/1.1 keep-alive obviates much of the performance gains promised by connection multiplexing.

• The performance gains possible using a binary RPC protocol do not justify the loss of a human-readable, browser-accessible JSON protocol.

• The use of newer, better languages than C++ (server side) or Python (client side). The instruments Tuber targets are likely to be a polyglot stew, and I am mindful that every additional language or runtime reduces the project’s accessibility to newcomers. Perhaps pybind11 will be eclipsed by something in Rust one day - for now, the ability to make casual cross-language calls is essential to keeping Tuber small. (Exception: the orjson JSON library is a wonderful complement to tuber and I recommend using them together!)

Although the Tuber server hosts an embedded Python interpreter and can expose embedded resources coded in ordinary Python, it is intended to expose C/C++ code. The Python interpeter provides a convenient, Pythonic approach to attribute and method lookup and dispatch without the overhead of a fully interpreted embedded runtime.

Tuber is licensed using the 3-clause BSD license (BSD-3-Clause). This software is intended to be useful, and its licensing is intended to be pragmatic. If licensing is a stumbling block for you, please contact me at gsmecher@threespeedlogic.com.

pip install tuberd

pip install tuberd[client,async,cbor,numpy,orjson]

pip install tuberd[dev]

pip install tuber-client

pip install tuber-client[async,cbor,numpy]

$ sudo apt-get install apache2-utils
$ echo '{ "object":"Wrapper", "method":"increment", "args":[[
     1,2,3,4,5,6,7,8,9,10,
     1,2,3,4,5,6,7,8,9,10,
     1,2,3,4,5,6,7,8,9,10,
     1,2,3,4,5,6,7,8,9,10,
     1,2,3,4,5,6,7,8,9,10,
     1,2,3,4,5,6,7,8,9,10,
     1,2,3,4,5,6,7,8,9,10,
     1,2,3,4,5,6,7,8,9,10,
     1,2,3,4,5,6,7,8,9,10,
     1,2,3,4,5,6,7,8,9,10 ]]}' > benchmark.json
$ for n in `seq 100`
  do
      ab -q -k -n 10000 -c 1 -p benchmark.json -T application/json http://localhost:8080/tuber
  done | awk '
     BEGIN { delete A }
     /Time taken/ { A[length(A)+1] = $5; }
     END { printf("x = [ "); for(i in A) printf(A[i] ", "); print "];" }'

import matplotlib.pyplot as plt
plt.hist(x)
plt.legend()
plt.grid(True)
plt.savefig('histogram.png')