Skip to main content

An easy to use rpc framework for enabling fast inter-process, inter-host communication

Project description

easyrpc

An easy to use rpc framework for enabling fast inter-process, inter-host communication

Easily share functions between hosts, processes, containers without the complexity of defining non-native python types or proxy modules.

Key Features

  • No need to predefine proxy functions at the remote endpoints
  • Proxy functions parameters are validated as if defined locally.
  • Optional pre-flight encyrption
  • The use of websockets means a single TCP stream can be used or all functions, which means less time waiting for TCP to get to know each other via 3-way-handshake(i.e 2 RTTs).
  • Data can be transported via any JSON serializable data-types, with no limit on message size or nested depth.

Quick Start

$ virtualenv -p python3.7 easy-rpc-env

$ source easy-rpc-env/bin/activate

(easy-rpc-env)$ pip install easyrpc

Basic Usage:

# server.py
from fastapi import FastAPI
from easyrpc.server import EasyRpcServer

server = FastAPI()

ws_server_a = EasyRpcServer(server, '/ws/server_a', server_secret='abcd1234')

@ws_server_a.origin
def good_func_a(a, b, c):
    print(f"good_func_a {a} {b} {c}")
    return {"good_func_a": [a, b, c]}

# client.py
import asyncio
from easyrpc.proxy import EasyRpcProxy

async def main():
    p = await EasyRpcProxy.create('0.0.0.0', 8090, '/ws/server_a', server_secret='abcd1234')

    good_func_a = p.proxy_funcs['good_func_a']
    result = await good_func_a(1, 5, 7)
    print(result)

asyncio.run(main())

Advanced Usage

# server.py
from fastapi import FastAPI
from easyrpc.server import EasyRpcServer

server = FastAPI()

ws_server_a = EasyRpcServer(server, '/ws/server_a', server_secret='abcd1234')

ws_server_b = EasyRpcServer(server, '/ws/server_b', server_secret='abcd1234')

@ws_server_a.origin
def good_func_a(a, b, c):
    print(f"good_func_a {a} {b} {c}")
    return {"good_func_a": [a, b, c]}

@ws_server_b.origin
def good_func_b(a, b, c):
    print(f"good_func_b {a} {b} {c}")
    return {"good_func_b": [a, b, c]}

# Register to both EasyRpcServer servers
# availabe for both /ws/server_a & /ws/server_b proxies

@ws_server_a.origin
@ws_server_b.origin
def good_func_c(a, **kw):
    print(f"good_func_c {a} {kw}")
    return {"good_func_c": [a, kw]}

# client.py
import asyncio
from easyrpc.proxy import EasyRpcProxy

async def main():
    p = await EasyRpcProxy.create('0.0.0.0', 8090, '/ws/server_a', server_secret='abcd1234')

    good_func_a = p.proxy_funcs['good_func_a']
    result = await good_func_a(1, 5, 7)
    print(result)

    result = await p.proxy_funcs['good_func_c'](1, keyword='value')
    print(result)

    print(p.proxy_funcs)

asyncio.run(main())

Output:

# Start server
(easy-rpc-env)$ uvicorn --host 0.0.0.0 --port 8090 test_server:server
10-14 23:18 wsRpc-server /ws/server_a WARNING  ORIGIN - registered function get_registered_functions 
10-14 23:18 wsRpc-server /ws/server_b WARNING  ORIGIN - registered function get_registered_functions 
10-14 23:18 wsRpc-server /ws/server_a WARNING  ORIGIN - registered function good_func_a 
10-14 23:18 wsRpc-server /ws/server_b WARNING  ORIGIN - registered function good_func_b 
10-14 23:18 wsRpc-server /ws/server_b WARNING  ORIGIN - registered function good_func_c 
10-14 23:18 wsRpc-server /ws/server_a WARNING  ORIGIN - registered function good_func_c 
INFO:     ('127.0.0.1', 57406) - "WebSocket /ws/server_a" [accepted]
10-14 23:36 uvicorn.error INFO     ('127.0.0.1', 57406) - "WebSocket /ws/server_a" [accepted]
10-14 23:36 wsRpc-server /ws/server_a WARNING  created websocket connection with endpoint 44d87c38-0e65-11eb-8ae3-2f2bf6388831
good_func_a 1 5 7
good_func_c 1 {'keyword': 'value'}
10-14 23:36 wsRpc-server /ws/server_a WARNING  deleted websocket connection with endpoint 44d87c38-0e65-11eb-8ae3-2f2bf6388831

# Client
$ python test_client.py 
{'good_func_a': [1, 5, 7]}
{'good_func_c': [1, {'keyword': 'value'}]}
{'get_registered_functions': <function create_proxy_from_config.<locals>.__proxy__ at 0x7f4e8d238200>, 'good_func_a': <function create_proxy_from_config.<locals>.__proxy__ at 0x7f4e8d24dcb0>, 'good_func_c': <function create_proxy_from_config.<locals>.__proxy__ at 0x7f4e8d24df80>}

A Helpful look at proxy signature

# Client
good_func_c = p.proxy_funcs['good_func_c']
help(good_func_c)

Help on function good_func_c_proxy in module easyrpc.register:

good_func_c_proxy(a, **kw)

Under the hood

easyrpc is made easy via the amazing fastapi framework for handling server side websocket communciation, aiohttp for the client-side websocket communication, makefun along with some standard library 'inspect' magic for translating origin functions into remote-useable functions with parameter validation, and lastly pyjwt for authentication & encryption.

Registered functions are made available as callables which return co-routines and thus 'awaitable' to the remote-endpoints, this is true for both async and non-async registered functions. Due to this, the functions must be awaited within a running event_loop. When called, the input parameters are verified via the origin functions signature.

Supported Functions Features

  • async def & def
  • *args, **kwargs
  • positional & default parmeters
  • TODO - type annotations

Common Use Cases

  • Long running processes where inter-process, intracluster communication occurs frequently.
  • Function Namespaces
  • Function Chaining
  • TODO - TBD

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

easyrpc-0.2-py3-none-any.whl (10.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page