ldcorn 0.1.2

master proxy and process manager for path-based routing across Uvicorn worker pools.

Ldcorn (WIP)

A master proxy and process manager for path-based routing across Uvicorn worker pools.

What it does

Ldcorn allows you to route incoming HTTP requests to specific Uvicorn worker processes based on the URL path. This gives you request-level load balancing across the multiple worker processes without needing separate deployments. For example, you can have one dedicated worker handling fast I/O queries, and another strictly reserved for heavy machine learning compute or long-lived WebSockets, all from the exact same monolithic codebase.

Is this for you?

The majority of the time, the answer is NO. If you have this sort of requirement, the standard answer is almost always to split your app into microservices and use Nginx to route traffic between them.

However, sometimes i get annoyed managing multiple deployments, dealing with code duplication, and handling schema drift is incredibly painful. That's where Ldcorn comes in. I built this because I prefer keeping my fast I/O workers completely insulated from my CPU-bound workers without managing a distributed system. You can even configure your heavy dependencies (like ML models) to lazy-load specifically on the compute workers. It gives you massive fine-grained control over a monolithic Python app.

Fair warning: Avoid using this as a hack for state management. You should generally rely on Redis for that. However, if you really need to pin stateful connections (like WebSockets or in-memory counters) to a single dedicated worker, Ldcorn's path routing will absolutely let you do that. Check out the included examples for proof.

Features

• Path-Based Routing: Route specific endpoints (like /ws or /heavy-compute) to dedicated worker processes. Ldcorn uses Longest Prefix Match for routing. If multiple groups share routes of the exact same length, the tie-breaker is their top-to-bottom definition order in your ldconfig.py. (Note: The upcoming ldcorn-go rewrite will also support cookie/header-based routing, but this pure-Python version intentionally opts out of those to avoid proxy overhead.)
• Max Requests Per Worker: Built-in concurrency queuing. Limit specific workers to exactly X concurrent requests to avoid locking databases or overloading threads. Requests exceeding this limit will queue asynchronously and wait for an available slot in that specific group (they do not fail over to other groups).
• Zero-Downtime Hot Reloads: Send SIGHUP to Ldcorn and it will elegantly spin up new workers, hot-swap the routing tables, and let the old workers gracefully finish their active requests. Zero dropped connections.
  • Dynamic Scaling & Proxy Updates: If you opt a worker out of SIGHUP reloads (reload_on_sighup=False), you can still edit your config to change its routes or max_req_per_worker and Ldcorn will instantly apply them at the proxy level without restarting the physical process!
  • Scale Up/Down Seamlessly: If you change an opted-out worker's instances from 1 to 4, Ldcorn will preserve the 1 running instance and spawn 3 new ones.
• Auto-Restarts: Built-in process monitoring revives crashed workers instantly.

Quickstart

1. Create ldconfig.py

from ldcorn.config import LdConfig, WorkerGroup

config = LdConfig(
    bind="127.0.0.1:8000", # for master process to listen on.
    workers=[
        WorkerGroup(
            name="default",
            app="main:app", 
            instances=2,
            max_req_per_worker=0, # Built-in concurrency queueing! 0 = unlimited
            routes=["*"], # all requests except another workergroup ones
            reload_on_sighup=True,
            max_restarts_on_crash=3,
            restart_backoff_on_crash=2
        ),
        WorkerGroup(
            name="ml_heavy",
            app="main:app", # this does not need to be same app
            instances=1,
            max_req_per_worker=100,
            routes=["/ml-pipeline"],
            reload_on_sighup=False, # Opt-out of hot-reloads so your ML model doesn't constantly reboot!
            max_restarts_on_crash=3,
            restart_backoff_on_crash=2
        )
    ]
)

2. Start Ldcorn

ldcorn -c ldconfig.py

3. Zero-Downtime Hot Swap Updated your code or your config? Reload seamlessly:

kill -HUP <ldcorn_pid>

Note: Changes to the bind parameter in your config will NOT take effect during a SIGHUP reload. That requires a full restart.

If you are deploying with systemd, configure ExecReload to send SIGHUP to the master process. This allows you to use systemctl reload ldcorn for completely seamless, zero-downtime deployments.

⚠️ Production Memory warning

During SIGHUP Ldcorn first creates new workers and once they are healthy then it sends graceful shutdown to old workers. so until all active requests in the old pool are completed , both the old and the new worker pools will be running at same time

As a result, your application will experience a transient 2x memory usage spike during the reload handover phase. If your system is running close to the memory limit (especially when loading large ML models or heavy frameworks), ensure you have enough swap space or free memory headroom to accommodate both pools running simultaneously to prevent the Linux Out-Of-Memory (OOM) killer from shutting down your application processes.

⚠️ Production Deployment & Reverse Proxy

Ldcorn currently does not automatically inject X-Forwarded-For or X-Real-IP headers. To ensure your worker processes receive the correct client IP addresses and protocol schemes, you must run Ldcorn behind a reverse proxy like Nginx, HAProxy, or Traefik that is configured to set these headers.

⚠️ Connection Keep-Alive

Ldcorn proxies traffic transparently but does not currently maintain persistent HTTP Keep-Alive connections between the client and the proxy itself. All connections are closed after the request is fulfilled due to hotreload. Performance overhead is negligible and can be discarded given how much performance boost you get due to request isolation per process.

Benchmarks: The "Perfect" Async Scenario

The following benchmarks compare Ldcorn against vanilla Uvicorn in a perfectly non-blocking scenario.

Setup:

• Load tester running 100 concurrent fast connections, and 10 concurrent connections for heavy/database endpoints. Total duration: 120 seconds.

• Database: MongoDB via motor (fully async connection pooling, no event loop blocking).

• Compute: Heavy ML and Math functions fully offloaded to asyncio.to_thread().

• Hardware: 16-core 13th Gen Intel(R) Core(TM) i7-13620H

• Uvicorn: uvicorn app:app --workers 6 --log-level error

• Ldcorn: Total of 6 workers, strictly isolated by endpoint. (3 default , 1 ML , 1 math , 1 websocket_stateful)

Endpoint	Ldcorn Group (workers)	Ldcorn (Req/s)	Uvicorn (Req/s)	Difference
Fast I/O	default : 3 workers handles all unmatched routes	1,438	1,699	-15.3%
DB Read		152	178	-14.4%
DB Insert		149	183	-18.7%
DB Upsert		148	184	-19.6%
DB Delete		151	181	-16.5%
Math Prime	math : 1 worker dedicated to /math	171	173	-1.3%
Heavy ML	ml : 1 worker dedicated to /ml-pipeline	56	88	-36.9%
Stateful Counter	websocket_stateful : 1 worker dedicated to /ws, /counter	201	197	+1.8%
TCP Edge Cases *	—	143	772	-81.4%

* Raw TCP gap reflects proxy hop overhead on zero-work requests with no application logic. This cost is invisible on real endpoints. fast I/O avg latency is 78ms vs 58ms at 100 concurrent.

Important: This is not ldcorn overhead. Notice that math (1 worker) nearly matches uvicorn's 6-worker throughput. The "loss" on other endpoints is purely because ldcorn intentionally assigns fewer workers to each route,those workers are free and not being utilized here. In a real mixed workload with any blocking code, those isolated workers are what keep your fast endpoints alive while the heavy ones are under load.

Why Uvicorn Won This Round

When an application is coded perfectly using pure async non-blocking drivers (motor) and completely offloading CPU tasks to thread pools (to_thread) on powerful hardware (big BUT here) the Python asyncio event loop is never blocked. In a perfectly non-blocking environment, round-robin distribution is mathematically optimal.

So why ldcorn?

Real-world production apps are rarely perfect in matter of full async. in such scenarios ldcorn helps , but if you have fully async production app then round robin of uvicorn, gunicorn is best for you.

Ldcorn is an architecture firewall against messy legacy code. Additionally, it provides zero-downtime hot reloads, dynamic proxy scaling, and state pinning out-of-the-box. which AGAIN can be done by using microservice as well as kube and sticky request by load balancer.

• its very small places where ldcorn has its use. in most cases round robin wins effectively
• state management part is there and process isolation is also very useful features, also knowing before hand how much concurrency you will get or what will be spike usage , fine grain control and process isolation helps a lot. you could based on load during runtime swap and hot reload , scale up and down , add more workers from python itself.

Benchmarks: The "Messy" Real-World Scenario

while writing this doc i felt like i am underselling ldcorn so here is somewhat blocking code.

i added this in both math and ml routes

    random_time = random.uniform(1,3)
    time.sleep(random_time) # well in real prod app this is not going be there at all, if it is then you have bigger problems , checkout loopsentry to find blocks (yes shameless selfpromotion)
    # common real-world blockers: sync ORMs (SQLAlchemy without async), requests library, legacy DB drivers, CPU-bound ML inference without to_thread, subprocess calls etc... or just some random running loop 

i will be dammed , did not expected this much diff

Here is what happens when that blocking time.sleep() is introduced to the ML and Math endpoints.

Endpoint	Ldcorn Group (workers)	Ldcorn Total	Ldcorn (Req/s)	Uvicorn Total	Uvicorn 6w (Req/s)	Difference
Fast I/O	default: 3 workers completely unaffected	114,164	950	4,077	32	+2,899%
DB Read		12,149	101	423	3	+2,972%
DB Insert		11,932	99	214	2	+4,867%
DB Upsert		11,874	99	263	2	+4,843%
DB Delete		12,160	101	230	2	+5,069%
Math Prime	math : 1 worker blocked: blast radius contained	71	0.52	191	1	—
Heavy ML	ml : 1 worker blocked: blast radius contained	70	0.50	174	1	—
Stateful Counter	websocket_stateful : 1 worker completely unaffected	15,964	133	487	4	+3,362%

Blocking time.sleep(1–3s) added to ML + Math routes starved all 6 uvicorn workers simultaneously, collapsing every endpoint to near-zero. Ldcorn's isolated groups contained the damage, default and websocket_stateful continued serving normally, completely unaware of the blockage in ml and math.

The Catastrophic Uvicorn Failure

Look at what happened to Uvicorn: Fast I/O dropped by 98% (from 1,698 req/s down to 31 req/s). Why? Because Uvicorn routes requests round-robin. Every time a user requested the ML or Math endpoint, that Uvicorn worker's event loop completely froze for 1-3 seconds. so all 6 workers were frozen. Thousands of pending Fast I/O and DB requests piled up in the OS socket queue waiting for the event loops to wake up. The entire API effectively went offline.

The Ldcorn Firewall

Ldcorn worked exactly as designed. The blocking ML and Math endpoints completely choked their isolated quarantine workers (dropping to 0.5 req/s). But the default worker group was completely unaffected. Ldcorn continued to serve Fast I/O at 950 req/s and handled Database operations at ~100 req/s, seamlessly maintaining uptime for 90% of the application while the heavy endpoints choked.

This is the entire point of Ldcorn. It trades a tiny bit of optimal performance in perfect scenarios for architectural resilience in messy, real-world production environments.

NOTE: ALL OF THESE CAN BE REPRODUCED , look at /examples