Skip to main content

Create distributed ASGI applications that pull events from a central Redis queue.

Project description

Distributed ASGI

Uses Redis to distribute ASGI messages between worker ASGI apps based on routes. Workers can be on different machines, they just must be able to connect to the central Redis server.

Usage

Routes are made of two parts: a regular expression that matches a path, and a key template string that is used to form the key that ASGI events are pushed to.

Key templates can also use numbered regex backslash replacements. For example, the route {"/api/([a-z-]+)":r"API-\1"} will match and produce the following keys:

PATH                KEY
/api/test           API-test
/api/test/38        API-test
/api/banana         API-banana
/test/api/test2     API-test2

Here's an example:

# distributor.py
from distributed_asgi import create_path_distributor

app = create_path_distributor(
    host="mywebsite.com",       # point to redis server
    port=6379,
    db=0,
    password="abc123",
    routes={
        "/api/([a-z-]+)": r"ASGI-\1",
        "/worker/([0-9]+)": r"worker-queue-\1",
        "/": "ALL-WEBSITE-TRAFFIC"
    }
)

To actually make use of this, we need to make a worker that listens on one or more of the keys

# worker.py
from distributed_asgi import Node


class ASGIApp:
    def __init__(self, scope):
        self.scope = scope

    async def __call__(self, receive, send):
        await send({
            "type": "http.response.start",
            "status": 200
        })

        await send({
            "type": "http.response.body",
            "body": b"Hello World!"
        })


node = Node(
    host="mywebsite.com",
    port="6379",
    password="abc123",
    cls=ASGIApp,
    key='ASGI-testing'
)

print(f"Starting worker")
node.run()

Once you have worker.py and server.py, use some interface server to run distributor.py.

$ uvicorn server:App

and run worker.py as a normal python script:

$ python worker.py

The worker will only respond to http requests with a path that contains /api/testing because that is the only key we told it to listen to.

What happens if there are no workers?

If there is no Node instance listening on a key that the Distributor pushes events to, the Distributor will timeout after 5 seconds, close the connection, and return a 405 error. Workers do not need to respond within 5 seconds, the Node class will automatically let the Distributor know there is a worker at least listening.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

distributed-asgi-0.0.7.tar.gz (4.4 kB view details)

Uploaded Source

File details

Details for the file distributed-asgi-0.0.7.tar.gz.

File metadata

  • Download URL: distributed-asgi-0.0.7.tar.gz
  • Upload date:
  • Size: 4.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.11.0 pkginfo/1.4.2 requests/2.19.1 setuptools/40.2.0 requests-toolbelt/0.8.0 tqdm/4.19.6 CPython/3.6.4

File hashes

Hashes for distributed-asgi-0.0.7.tar.gz
Algorithm Hash digest
SHA256 f6855711b2b8d39ac1315657307377247676548d0ae4155396a8ae365ce3715a
MD5 944a2c30f4f932a77058f43a945a7951
BLAKE2b-256 c5c4b4adc38e5f44d4f4b5a026f11298613e7237d1094b0d8863869d6cf3e423

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page