Skip to main content

generic distributed-automation framework — master/worker, browser pool, anti-bot helpers, and abstract automation contracts (SERP, content, news, place, marketplace, jobs, social)

Project description

Fleet

A framework for running Python automations across a fleet of machines. You write the automation once; Fleet hands it out to your workers, keeps them running, restarts the ones that fall over, and ships their output where you want it to go.

One process, the master, runs on one host. Many workers, one per machine, do the actual work. Output from one workload can feed another across hosts; you don't write the plumbing.

Battle-tested in the wild as the brains behind a Cloudflare Turnstile-solving farm.

What it is

Fleet is two pieces: a master and some workers.

The master runs on one machine. It's where you set what to do and read what got done. Workers run on every other machine in your fleet. They check in with the master, get instructions, and do the work.

The work itself is a small Python class you write. Ship it as a pip package, install it on the machines, and Fleet finds it on its own.

What's in the box

  • One master, many workers. Change config in one place, every worker follows.
  • Workers heal themselves. A crashed task is restarted on the next tick.
  • Workloads hand output to each other across machines, no glue code.
  • Optional Chromium pool for anything that needs a real browser.
  • Auth, a small dashboard, and observability hooks. All wired up.

Getting started

pip install fleet-framework

Then write your automation as a Python class. Roughly 30 lines:

import asyncio
import httpx
from fleet.core import BaseConfig, ContinuousAutomation, register

class PingerConfig(BaseConfig):
    url: str
    interval_seconds: float = 5.0

@register("pinger")
class Pinger(ContinuousAutomation[PingerConfig]):
    Config = PingerConfig

    async def run_slot(self, ctx):
        async with httpx.AsyncClient(proxy=ctx.proxy) as client:
            while not ctx.shutdown.is_set():
                r = await client.get(ctx.config.url)
                await ctx.emit({"status": r.status_code})
                await asyncio.sleep(ctx.config.interval_seconds)

Full walkthrough — master, worker, config, output — at http://fleet.hackmap.win/getting-started/quickstart.

Documentation

Full docs live at http://fleet.hackmap.win/.

License

MIT. See LICENSE.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

fleet_framework-1.1.0.tar.gz (153.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

fleet_framework-1.1.0-py3-none-any.whl (191.9 kB view details)

Uploaded Python 3

File details

Details for the file fleet_framework-1.1.0.tar.gz.

File metadata

  • Download URL: fleet_framework-1.1.0.tar.gz
  • Upload date:
  • Size: 153.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.14.5

File hashes

Hashes for fleet_framework-1.1.0.tar.gz
Algorithm Hash digest
SHA256 1ec3cb64df42f456e849f885b17093fa1404fadb91e0ab10ecb07b92ab72ecd0
MD5 931100ece07881e80662dae9c2edd91e
BLAKE2b-256 50ace2120a45fff741dd41c775c25cc52acb1215c35dcdd591cc02c3bb6d38f6

See more details on using hashes here.

File details

Details for the file fleet_framework-1.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for fleet_framework-1.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 ddd0d8aa726550f2c5a3bc3c8c1bcb3fd733a203058d416fb7390de459b1a023
MD5 543c1c22814c5fff447f3c711ecda37b
BLAKE2b-256 b3421c3914ea555c7268e749cee82e3a543513aa8ca573ccb78f1a01e35b3f06

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page