HTTP Router based on radix trees

These details have been verified by PyPI

Project links

GitHub Statistics

Maintainers

erewok

These details have not been verified by PyPI

Project links

documentation

Project description

Tokamak

Tokamak is a pure-Python router based on Radix trees intended for ASGI Python applications.

[Read the documentation]

Primary Project Goals

There are many HTTP routers based on radix trees available in other language communities, including Go, Javascript, Rust, and others. In Python, however, most open-source Python web frameworks instead utilize lists to store and look up HTTP routes.

The primary goal for this project is to provide a radix-tree-based router for Python web frameworks (or any custom ASGI or WSGI implementation).

This library is experimental. Use at your own risk.

Other Goals

As a secondary goal, a minimal and highly experimental web framework is included an optional install in this library.

The reasons for including this web framework are as follows:

It provides a convenient way to test the AsgiRouter class, and
It allows the authors of this library to explore experimental ASGI-framework features, including request-cancellation, background task time-limits, and background task cancellation.

Developers should consider more fully-featured web frameworks before this one and there are many to choose from:

Installation

You can install tokamak with:

pip install tokamak

By default tokamak has no dependencies.

If you would like to try out the experimental web framework, you can install with optional extras web, which will include trio:

$ pip install "tokamak[web]"
...

Usage

This library provides a radix tree implementation and a basic AsgiRouter router implementation for low-level ASGI applications. You can use the AsgiRouter class as follows.

First, we start with some with some imports and some fallback handlers:

from hypercorn.config import Config
from hypercorn.trio import serve
import trio

from tokamak import AsgiRouter, Route
from tokamak.router import MethodNotAllowedError, UnknownEndpointError


# # Fallback Handlers # #
async def method_not_allowed(scope, receive, send):
    await send(
        {
            "type": "http.response.start",
            "status": 405,
            "headers": [(b"Content-Type", b"text/html; charset=UTF-8")],
        }
    )
    await send(
        {
            "type": "http.response.body",
            "body": b"<html><body><h1>405 Method not allowed</h1></body></html>",
        }
    )


async def unknown_handler(scope, receive, send):
    await send(
        {
            "type": "http.response.start",
            "status": 404,
            "headers": [(b"Content-Type", b"text/html; charset=UTF-8")],
        }
    )
    await send(
        {
            "type": "http.response.body",
            "body": b"<html><body><h1>404 Not Found!</h1></body></html>",
        }
    )

Next we'll build two different application endpoint handlers. These do roughly the same thing, so this is purely for demonstration purposes:

async def index(path_context, scope, receive, send):
    message = await receive()
    if message["type"] == "http.request":
        body = message.get("body", b"")
        # here's our response:
        await send({"type": "http.response.start", "status": 200})
        await send({"type": "http.response.body", "body": body if body else b"OK"})
    elif message["type"] == "http.disconnect":
        print("Disconnected! ")


async def other_handler(path_context, scope, receive, send):
    context = bytes(json.dumps(path_context), encoding="utf-8")
    message = await receive()
    if message["type"] == "http.request":
        body = message.get("body", b"")
        await send({"type": "http.response.start", "status": 200})
        await send({"type": "http.response.body", "body": body if body else context})
    elif message["type"] == "http.disconnect":
        print("Disconnected! ")

Finally, we can build an AsgiRouter and a working ASGI app, like this:

# `AsgiRouter` and `Route` class provided by this library
ROUTER = AsgiRouter(
    routes=[
        Route("/", handler=index, methods=["GET"]),
        # Routes will match on regexes and bind to variables
        # given on the left side of the colon
        Route(
            "/other_handler/{name:[a-z1-9]+}", handler=other_handler, methods=["POST"],
        ),
    ]
)

# This is a basic implementation of the ASGI spec
# See: https://asgi.readthedocs.io/en/latest/specs/main.html
async def asgi_app(scope, receive, send):
    path = scope.get("path", "")
    try:
        # Routers provider a `get_route` method
        # If no route is matched, they throw `UnknownEndpointError`
        # If a route is matched, we'll get path context and a handler
        handler, context = ROUTER.get_route(path)
    except UnknownEndpointError:
        await unknown_handler(scope, receive, send)
        return None

    try:
        # If a matched router doesn't handle this method
        # it will throw `MethodNotAllowedError`
        await handler(context, scope, receive, send, method=scope.get("method"))
    except MethodNotAllowedError:
      await method_not_allowed(scope, receive, send)
      return None


async def app_with_lifespan(scope, receive, send):
    if scope["type"] == "lifespan":
        while True:
            message = await receive()
            if message["type"] == "lifespan.startup":
                await send({"type": "lifespan.startup.complete"})
            elif message["type"] == "lifespan.shutdown":
                await send({"type": "lifespan.shutdown.complete"})
                return
    if scope["type"] == "http":
        return await asgi_app(scope, receive, send)

Finally, to run our ASGI app, we'll add the following:

if __name__ == "__main__":
    config = Config()
    config.bind = ["localhost:8000"]
    trio.run(partial(serve, app_with_lifespan, config))

This example relies on the following dependencies:

hypercorn
trio

If we have these dependencies in our Python environment, we can run this simple script:

$ python examples/asgi_minimal.py
[2022-03-20 16:59:58 -0700] [91988] [INFO] Running on http://127.0.0.1:8000 (CTRL + C to quit)

In a separate terminal, we can try it out like so:

❯ curl http://localhost:8000/
OK

# No capital letters matched
❯ curl -XPOST http://localhost:8000/other_handler/bla1AA
<html><body><h1>404 Not Found!</h1></body></html>

# GET not POST -> 405
❯ curl http://localhost:8000/other_handler/bla
<html><body><h1>405 Method not allowed</h1></body></html>

# Success
❯ curl -XPOST http://localhost:8000/other_handler/bla1
{"name": "bla1"}

Note: that our regex path does not match capital letters, so that request 404s.

For Contributors

This project uses uv for managing dependencies and virtual environments.

In addition, to contribute to this project, we recommend using just: https://github.com/casey/just

You can run various common workflows using the above tools, try the following:

❯ just
just --list
Available recipes:
    benchmark                         # Run the benchmark
    bootstrap default="3.12"          # Install dependencies used by this project
    build *args                       # Build the project as a package (uv build)
    check                             # Run code quality checks
    check-types                       # Run mypy checks
    ci-test coverage_dir='./coverage' # Run the project tests for CI environment (e.g. with code coverage)
    example name                      # Run an example
    format                            # Run the code formatter
    sync                              # Sync dependencies with environment
    test *args                        # Run all tests locally

❯ just check
+ uv run ruff check tokamak tests
All checks passed!

❯ just test
+ uv run pytest
...

Examples

Runnable examples are provided in the examples directory. In addition, this project includes a justfile (see just) for easily running examples.

For instance, you can run the experimental tokamak application with trio and hypercorn like so:

$  just example tokamak_app
uv run --extra examples python examples/tokamak_app.py
Installed 13 packages in 5.55s
========·°·°~> Starting tokamak °°···°°🚀···°°
[2024-11-19 09:01:24 -0800] [32768] [INFO] Running on http://127.0.0.1:8000 (CTRL + C to quit)

In a separate terminal, you can make various requests, such as the following:

❯ curl http://localhost:8000
ok

❯ curl http://localhost:8000/info/erik -d '{"some_data": "something"}'
{"received": {"some_data": "something"}}

Back in the first terminal, where you launched the example tokamak application, you should see the following:

❯ just example tokamak_app
uv run --extra examples python examples/tokamak_app.py
Installed 13 packages in 5.55s
========·°·°~> Starting tokamak °°···°°🚀···°°
[2024-11-19 09:01:24 -0800] [32768] [INFO] Running on http://127.0.0.1:8000 (CTRL + C to quit)
request.app.db={}, request.context={'user': 'erik'}, request.scope={'type': 'http', 'http_version': '1.1', 'asgi': {'spec_version': '2.1', 'version': '3.0'}, 'method': 'POST', 'scheme': 'http', 'path': '/info/erik', 'raw_path': b'/info/erik', 'query_string': b'', 'root_path': '', 'headers': [(b'host', b'localhost:8000'), (b'user-agent', b'curl/8.7.1'), (b'accept', b'*/*'), (b'content-length', b'26'), (b'content-type', b'application/x-www-form-urlencoded')], 'client': ('127.0.0.1', 63965), 'server': ('127.0.0.1', 8000), 'state': {}, 'extensions': {}, 'app': <tokamak.web.app.Tokamak object at 0x11865c4a0>}, headers=[(b'host', b'localhost:8000'), (b'user-agent', b'curl/8.7.1'), (b'accept', b'*/*'), (b'content-length', b'26'), (b'content-type', b'application/x-www-form-urlencoded')], qparams=b'', http_version='1.1', method='POST'
Sleeping 1s for total iterations: 0
Sleeping 1s for total iterations: 1
Sleeping 1s for total iterations: 2

Benchmark

This project was iniatated around the time that the router for Werkzeug (which powers Flask) was rewritten as well. That router was redesigned to use a modified Radix Tree and so we created a benchmark to compare their implementation with this one.

To run the benchmark against Werkzeug main, run the following:

uv run --extra benchmarks python -m benchmark.compare_werkzeug
Path                                                                   | Ratio (percent difference from baseline)
Tokamak Tree is quicker: /users/{username}/following                                            | 0.64
Werkzeug Tree is quicker: /repos/{owner}/{repo}/downloads                                        | 0.80
Werkzeug Tree is quicker: /repos/{owner}/{repo}/hooks/{id}/pings                                 | 0.70
...

****** TIMING STATISTICS TOKAMAK FASTER THAN BASELINE ******
Better Total 5270
Best improvement (min vs baseline) 0.12283152787580384 for path /
Mean Improvement:  0.6338085629349435
Median Improvement:  0.618251951398763
Std Dev Improvements:  0.18516141441278625
Mean Path Length:  19.97020872865275
Mean Dynamic Segment Count:  0.6757115749525616
****** TIMING STATISTICS TOKAMAK END ******

****** TIMING STATISTICS WERKZEUG FASTER THAN BASELINE ******
Better Total 4730
Best improvement (min vs baseline) 0.23255522605196324 for path /repos/{owner}/{repo}/labels/{name}
Mean Improvement:  0.6003032685655771
Median Improvement:  0.5382439859668042
Std Dev Improvements:  0.20348200780749615
Mean Path Length:  36.14545454545455
Mean Dynamic Segment Count:  2.468076109936575
****** TIMING STATISTICS WERKZEUG END ******

Project details

These details have been verified by PyPI

Project links

GitHub Statistics

Maintainers

erewok

These details have not been verified by PyPI

Project links

documentation

Release history Release notifications | RSS feed

This version

0.6.1

Nov 19, 2024

0.5.5

Aug 19, 2022

0.5.4

Aug 15, 2022

0.5.3

Aug 14, 2022

0.5.2

Aug 14, 2022

0.5.1

Aug 13, 2022

0.5.0

Aug 12, 2022

0.4.0

Aug 6, 2022

0.3.1

Mar 21, 2022

0.3.0

Mar 21, 2022

0.2.1

Mar 20, 2022

0.1.1

Nov 21, 2021

0.1.0

Nov 21, 2021

0.0.1

Sep 14, 2020

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tokamak-0.6.1.tar.gz (40.8 kB view details)

Uploaded Nov 19, 2024 Source

Built Distribution

tokamak-0.6.1-py3-none-any.whl (44.6 kB view details)

Uploaded Nov 19, 2024 Python 3

File details

Details for the file tokamak-0.6.1.tar.gz.

File metadata

Download URL: tokamak-0.6.1.tar.gz
Upload date: Nov 19, 2024
Size: 40.8 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokamak-0.6.1.tar.gz
Algorithm	Hash digest
SHA256	`404b15495cf511ebd7733cd80ad708bbfa2e6543f583bb7274b484f3be979bfc`
MD5	`791eae20fb66e5e5607f5adca6481c37`
BLAKE2b-256	`5590f542868eee60e7497f928f3c2f87cfd532e499fab59429827fe70dbe1bd8`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tokamak-0.6.1.tar.gz:

Publisher: release.yaml on erewok/tokamak

Attestations:

Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tokamak-0.6.1.tar.gz
- Subject digest: 404b15495cf511ebd7733cd80ad708bbfa2e6543f583bb7274b484f3be979bfc
- Sigstore transparency entry: 149943878
- Sigstore integration time: Nov 19, 2024

File details

Details for the file tokamak-0.6.1-py3-none-any.whl.

File metadata

Download URL: tokamak-0.6.1-py3-none-any.whl
Upload date: Nov 19, 2024
Size: 44.6 kB
Tags: Python 3
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/5.1.1 CPython/3.12.7

File hashes

Hashes for tokamak-0.6.1-py3-none-any.whl
Algorithm	Hash digest
SHA256	`6e338bb0819c811c4141808e0fc8c2763e03c47648e503224cdd580af7bacb98`
MD5	`d391fe2fa208826d881f17088a2e75d5`
BLAKE2b-256	`792dc0de8106002e01b468e1f20aca1b35d8c8ec05fc8da8f92698e60fe15a23`

See more details on using hashes here.

Provenance

The following attestation bundles were made for tokamak-0.6.1-py3-none-any.whl:

Publisher: release.yaml on erewok/tokamak

Attestations:

Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: tokamak-0.6.1-py3-none-any.whl
- Subject digest: 6e338bb0819c811c4141808e0fc8c2763e03c47648e503224cdd580af7bacb98
- Sigstore transparency entry: 149943879
- Sigstore integration time: Nov 19, 2024