Skip to main content

This is a Python package for HireFire – The Heroku Process Manager:

HireFire has the ability to automatically scale your web and worker dynos up and down when necessary. When new jobs are queued in to your application’s worker queue [..], HireFire will spin up new worker dynos to process these jobs. When the queue is empty, HireFire will shut down the worker dynos again so you’re not paying for idle workers.

HireFire also has the ability to scale your web dynos. When your web application experiences heavy traffic during certain times of the day, or if you’ve been featured somewhere, chances are your application’s backlog might grow to a point that your web application will run dramatically slow, or even worse, it might result in a timeout. In order to prevent this, HireFire will automatically scale your web dynos up when traffic increases to ensure that your application runs fast at all times. When traffic decreases, HireFire will spin down your web dynos again.

—from the HireFire frontpage

It supports the following Python queuing systems as backends:

Feel free to contribute other backends if you’re using a different queuing system.

Installation

Install the HireFire package with your favorite installer, e.g.:

pip install HireFire

Sign up for HireFire and set the HIREFIRE_TOKEN environment variable with the Heroku CLI as provided on the specific HireFire application page, e.g.:

heroku config:set HIREFIRE_TOKEN=f69f0c0ddebe041248daf187caa6abb3e5d943ca

Now follow the quickstart guide below and don’t forget to tweak the options in the HireFire management system.

For more help see the Hirefire documentation.

Configuration

The hirefire Python package currently supports three frameworks: Django, Tornado, and Flask. Implementations for other frameworks are planned but haven’t been worked on: Pyramid (PasteDeploy), WSGI middleware, ..

Feel free to contribute one if you can’t wait.

The following guides imply you have defined at least one hirefire.procs.Proc subclass defined matching one of the processes in your Procfile. For each process you want to monitor you have to have one subclass.

For example here is a Procfile which uses RQ for the “worker” proccess:

web: python manage.py runserver
worker: DJANGO_SETTINGS_MODULE=mysite.settings rqworker high default low

Define a RQProc subclass somewhere in your project, e.g. mysite/procs.py, with the appropriate attributes (name and queues)

from hirefire.procs.rq import RQProc

class WorkerProc(RQProc):
    name = 'worker'
    queues = ['high', 'default', 'low']

See the procs API documentation if you’re using another backend. Now follow the framework specific guidelines below.

Django

Setting up HireFire support for Django is easy:

  1. Add 'hirefire.contrib.django.middleware.HireFireMiddleware' to your MIDDLEWARE setting

    # Use ``MIDDLEWARE_CLASSES`` prior to Django 1.10
    MIDDLEWARE = [
        'hirefire.contrib.django.middleware.HireFireMiddleware',
        # ...
    ]

    Make sure it’s the first item in the list/tuple.

  2. Set the HIREFIRE_PROCS setting to a list of dotted paths to your procs. For the above example proc

    HIREFIRE_PROCS = ['mysite.procs.WorkerProc']
  3. Set the HIREFIRE_TOKEN setting to the token that HireFire shows on the specific application page (optional)

    HIREFIRE_TOKEN = 'f69f0c0ddebe041248daf187caa6abb3e5d943ca'

    This is only needed if you haven’t set the HIREFIRE_TOKEN environment variable already (see the installation section how to do that on Heroku).

  4. Add 'hirefire.contrib.django.middleware.QueueTimeMiddleware' to your MIDDLEWARE setting to enable HireFire’s support for scaling according to Heroku request queue times (optional).

    # Use ``MIDDLEWARE_CLASSES`` prior to Django 1.10
    MIDDLEWARE = [
        'hirefire.contrib.django.middleware.HireFireMiddleware',
        # ...
    ]

    Make sure to place it before any other item in the list/tuple so that request queue time is calculated as accurately as possible.

  5. Check that the middleware has been correctly setup by opening the following URL in a browser:

    http://localhost:8000/hirefire/test

    You should see an empty page with ‘HireFire Middleware Found!’.

    You can also have a look at the page that HireFire checks to get the number of current tasks:

    http://localhost:8000/hirefire/<HIREFIRE_TOKEN>/info

    where <HIREFIRE_TOKEN> needs to be replaced with your token or – in case you haven’t set the token in your settings or environment – just use development.

Tornado

Setting up HireFire support for Tornado is also easy:

  1. Use hirefire.contrib.tornado.handlers.hirefire_handlers when defining your tornado.web.Application instance

    import os
    from hirefire.contrib.tornado.handlers import hirefire_handlers
    
    application = tornado.web.Application([
        # .. some patterns and handlers
    ] + hirefire_handlers(os.environ['HIREFIRE_TOKEN'],
                          ['mysite.procs.WorkerProc']))

    Make sure to pass a list of dotted paths to the hirefire_handlers function.

  2. Set the HIREFIRE_TOKEN environment variable to the token that HireFire shows on the specific application page (optional)

    export HIREFIRE_TOKEN='f69f0c0ddebe041248daf187caa6abb3e5d943ca'

    See the installation section above for how to do that on Heroku.

  3. Check that the handlers have been correctly setup by opening the following URL in a browser:

    http://localhost:8888/hirefire/test

    You should see an empty page with ‘HireFire Middleware Found!’.

    You can also have a look at the page that HireFire checks to get the number of current tasks:

    http://localhost:8888/hirefire/<HIREFIRE_TOKEN>/info

    where <HIREFIRE_TOKEN> needs to be replaced with your token or – in case you haven’t set the token as an environment variable – just use development.

Flask

Setting up HireFire support for Flask is (again!) also easy:

  1. The module hirefire.contrib.flask.blueprint provides a build_hirefire_blueprint factory function that should be called with HireFire token and procs as arguments. The result is a blueprint providing the hirefire routes and which should be registered inside your app

    import os
    from flask import Flask
    from hirefire.contrib.flask.blueprint import build_hirefire_blueprint
    
    app = Flask(__name__)
    bp = build_hirefire_blueprint(os.environ['HIREFIRE_TOKEN'],
                                  ['mysite.procs.WorkerProc'])
    app.register_blueprint(bp)

    Make sure to pass a list of dotted paths to the build_hirefire_blueprint function.

  2. Set the HIREFIRE_TOKEN environment variable to the token that HireFire shows on the specific application page (optional)

    export HIREFIRE_TOKEN='f69f0c0ddebe041248daf187caa6abb3e5d943ca'

    See the installation section above for how to do that on Heroku.

  3. Check that the handlers have been correctly setup by opening the following URL in a browser:

    http://localhost:8080/hirefire/test

    You should see an empty page with ‘HireFire Middleware Found!’.

    You can also have a look at the page that HireFire checks to get the number of current tasks:

    http://localhost:8080/hirefire/<HIREFIRE_TOKEN>/info

    where <HIREFIRE_TOKEN> needs to be replaced with your token or – in case you haven’t set the token as an environment variable – just use development.

Release files for HireFire 1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for HireFire 1.1
File Size Uploaded
HireFire-1.1.tar.gz 25.4 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for HireFire 1.1
File Interpreter ABI Platform
HireFire-1.1-py2.py3-none-any.whl Python 2, Python 3 none any Details

Total release size: 51.7 kB

Release files / HireFire-1.1.tar.gz

Download URL HireFire-1.1.tar.gz
Size 25.4 kB
Tags Source
SHA-256 checksum
How to use checksums
f18ee47fbfd3848d80c8d1801c83ac0db908da661cd50317dd43b6f7c2860624
BLAKE2b-256 checksum
How to use checksums
c10d1e2288aa13598f7aa134c65e11eef847c358d557abd0a9f1ddb5756998e6
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.4.1 importlib_metadata/4.4.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.7.1

Release files / HireFire-1.1-py2.py3-none-any.whl

Download URL HireFire-1.1-py2.py3-none-any.whl
Size 26.4 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
23642574806c159cb6f6b4dffa704f76d2c869745d5b8d3a1fde68fd6277579f
BLAKE2b-256 checksum
How to use checksums
9fac71c526f8df2d58185ae5f6efccf69e3393c7be9f16607a06a9adfe983d28
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.4.1 importlib_metadata/4.4.0 pkginfo/1.7.0 requests/2.25.1 requests-toolbelt/0.9.1 tqdm/4.61.0 CPython/3.7.1

Release history Release notifications | RSS feed

This release

1.1 This release

2 release files

1.0

2 release files

0.8

2 release files

0.7

2 release files

0.6

2 release files

0.5

2 release files

0.4

2 release files

0.3

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2

2 release files

0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page