Skip to main content

FastQ

A high-performance job queue written in C, backed by Redis.

FastQ aims to be faster, leaner on RAM, and polyglot compared to existing job queue solutions like BullMQ, Sidekiq, or Celery.

Features

Core (free)

  • Fast push/pop via Redis Lists with MULTI/EXEC pipelining
  • Priority queues (high / normal / low)
  • Multi-threaded worker pool (pthreads) with Redis connection pooling
  • Automatic retries with exponential backoff
  • Dead Letter Queue for permanently failed jobs
  • Crash recovery for orphaned jobs
  • Daemon mode with systemd support
  • Python bindings (CPython C extension)
  • Node.js bindings (N-API)
  • CLI tool (fastq push, pop, stats, worker, recover)
  • Colored logging (DEBUG/INFO/WARN/ERROR)
  • Zero external runtime dependencies beyond Redis

Pro

  • Cron scheduler — persistent cron jobs and one-shot delayed jobs, survive restarts
  • Rate limiting — token-bucket limiter attached per worker
  • Batch processing — pop up to N jobs in one shot with a deadline
  • Job chaining — automatically push a child job when a parent completes
  • DAG workflows — multi-step dependency graphs; dependents unlock atomically
  • Metrics — Prometheus /metrics and JSON /health HTTP endpoint
  • HMAC license — key verification with constant-time comparison (OpenSSL)

Building

Dependencies

  • hiredis — Redis C client
  • json-c — JSON parsing
  • OpenSSL — HMAC-SHA256 (license + TLS)
  • Redis server (runtime)

Compile

make          # library + CLI
make test     # run all tests
make bench    # throughput benchmark
make python   # Python C extension
make node     # Node.js N-API addon

Run tests

# Start a dev Redis instance (in a separate terminal)
./run_redis.sh

make test

# Run tests under Valgrind
make valgrind

Benchmark

make bench

Python bindings

make python
python3 -c "import fastq; q = fastq.Queue('myqueue'); q.push('{\"hello\":1}')"

Node.js bindings

make node
node -e "const { Queue } = require('./bindings/node'); const q = new Queue('127.0.0.1', 6379, 'myqueue'); console.log(q.push('{\"hello\":1}'));"

Project Structure

fastq/
├── src/                # Core library source
├── include/            # Public headers
├── tests/              # Unit and integration tests (38 tests, 7 suites)
├── benchmarks/         # Throughput benchmarks
├── bindings/
│   ├── python/         # Python CPython extension
│   └── node/           # Node.js N-API addon
├── docs/               # Architecture & API design docs
├── Makefile
├── fastq.service       # systemd unit file
└── run_redis.sh        # Dev Redis launcher

Roadmap

Done

  • Core engine, priority queues, multi-threaded workers, connection pooling
  • Crash recovery, daemon mode, CLI
  • Python and Node.js bindings
  • Cron scheduler, rate limiter, batch processing
  • Job chaining and DAG workflows
  • Prometheus metrics, health endpoint
  • HMAC license system
  • 38 tests across 7 suites, all passing

Next

  • Go / Rust bindings
  • Web UI
  • Kubernetes operator

Metadata

Release files for fastq-redis 1.1.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for fastq-redis 1.1.1
File Size Uploaded
fastq_redis-1.1.1.tar.gz 6.3 kB Details

Built distributions (wheels)

Table of built distributions (wheels) for fastq-redis 1.1.1
File
fastq_redis-1.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.12 CPython 3.12 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details
fastq_redis-1.1.1-cp312-cp312-macosx_11_0_arm64.whl CPython 3.12 CPython 3.12 macOS 11.0+ ARM64 Details
fastq_redis-1.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.11 CPython 3.11 Linux glibc 2.28+ x86-64, Linux glibc 2.27+ x86-64 Details
fastq_redis-1.1.1-cp311-cp311-macosx_11_0_arm64.whl CPython 3.11 CPython 3.11 macOS 11.0+ ARM64 Details
fastq_redis-1.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details
fastq_redis-1.1.1-cp310-cp310-macosx_11_0_arm64.whl CPython 3.10 CPython 3.10 macOS 11.0+ ARM64 Details
fastq_redis-1.1.1-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.9 CPython 3.9 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details
fastq_redis-1.1.1-cp39-cp39-macosx_11_0_arm64.whl CPython 3.9 CPython 3.9 macOS 11.0+ ARM64 Details
fastq_redis-1.1.1-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl CPython 3.8 CPython 3.8 Linux glibc 2.27+ x86-64, Linux glibc 2.28+ x86-64 Details
fastq_redis-1.1.1-cp38-cp38-macosx_11_0_arm64.whl CPython 3.8 CPython 3.8 macOS 11.0+ ARM64 Details

Total release size: 36.0 MB

Release files / fastq_redis-1.1.1.tar.gz

Download URL fastq_redis-1.1.1.tar.gz
Size 6.3 kB
Tags Source
SHA-256 checksum
How to use checksums
aa23c0101c5ff96d4a92c66e27b1a34ec0a2e1035a47dbd88c6286354bc794a6
BLAKE2b-256 checksum
How to use checksums
6dd059575005951106b47150775bebf2f3c24b274650cac4fee14c88527e3994
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL fastq_redis-1.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 4.5 MB
Tags CPython 3.12 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
24f298c8740e9df7b35259110220daa8fd89eb9107c57a1939c982f0124991d5
BLAKE2b-256 checksum
How to use checksums
148be4975c179097997b959b559b721e24dde30c5709dbc991703f3ea3468f5c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp312-cp312-macosx_11_0_arm64.whl

Download URL fastq_redis-1.1.1-cp312-cp312-macosx_11_0_arm64.whl
Size 2.7 MB
Tags CPython 3.12 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
01c1714df6e07cc440e162d8d161eeb35c729fc253126b017fe9da11d66d7701
BLAKE2b-256 checksum
How to use checksums
a2fac9b2bd7a7905d68bed33dfa3051579f8f4ca1e59e291351f76230dd0bb1e
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL fastq_redis-1.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 4.5 MB
Tags CPython 3.11 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
d6cebec2ba4492dccec3b08601845dc941b7b85c1cf41a4016a9777eaa4e69a3
BLAKE2b-256 checksum
How to use checksums
0eec47d53153447ccf962a7b039e7b366171d934f1c223bf7d6c58eb107eb85c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp311-cp311-macosx_11_0_arm64.whl

Download URL fastq_redis-1.1.1-cp311-cp311-macosx_11_0_arm64.whl
Size 2.7 MB
Tags CPython 3.11 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
8d0207ed444bd0f5b343456825f90d0df87365cb23c681af0b1ceaa4a814ec9f
BLAKE2b-256 checksum
How to use checksums
1b4da52f8685a8e195a7aca4c343cefee71c9de4abed062519f3a3f472910b3b
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL fastq_redis-1.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 4.5 MB
Tags CPython 3.10 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
9c81297c07d4f538b3fbb03bd662f123ee51b54038e977cfd3900000f2f151cc
BLAKE2b-256 checksum
How to use checksums
6537159df0cd3385355703cee6f98ab52fbc24809f5162ffc58d76729b0ca8c3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp310-cp310-macosx_11_0_arm64.whl

Download URL fastq_redis-1.1.1-cp310-cp310-macosx_11_0_arm64.whl
Size 2.7 MB
Tags CPython 3.10 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
07eb1f5cfd2f659f3134b5edbb59415f9c87857950f9b64cf6e7314476412fa0
BLAKE2b-256 checksum
How to use checksums
bcf5e94e2db1b5f2e80896b0a4af077f2e164fa41352db3204ed48848a154c4d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL fastq_redis-1.1.1-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 4.5 MB
Tags CPython 3.9 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
f49378bc56d6d8f142ea939754d52679df2390008a81beea0437de0434676c51
BLAKE2b-256 checksum
How to use checksums
aa58261f8fc2e898aab1b407314b8791055f20ee679dc0ff3bb75f66ea71ac80
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp39-cp39-macosx_11_0_arm64.whl

Download URL fastq_redis-1.1.1-cp39-cp39-macosx_11_0_arm64.whl
Size 2.7 MB
Tags CPython 3.9 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
5adc4207d5993aea258ad5ae5c38ce97d3de2bcea184494cc3a5dbb6b84ca331
BLAKE2b-256 checksum
How to use checksums
e519a6371f3174db84fd01e6aad70a78e7be215644f352f8d0298cbed600cfa0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl

Download URL fastq_redis-1.1.1-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
Size 4.5 MB
Tags CPython 3.8 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64
SHA-256 checksum
How to use checksums
b4a883e135f417a6962aa8456cb6c0c3e99b8373ece9869343d9a00577f2f8ac
BLAKE2b-256 checksum
How to use checksums
1d7ce439e4501cbde9023ef8bdcbb8251af741ef36b0120bcc5772e4a8d86948
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release files / fastq_redis-1.1.1-cp38-cp38-macosx_11_0_arm64.whl

Download URL fastq_redis-1.1.1-cp38-cp38-macosx_11_0_arm64.whl
Size 2.7 MB
Tags CPython 3.8 macOS 11.0+ ARM64
SHA-256 checksum
How to use checksums
79785c2acdb978414445d81648907644573435fcc5ea9a3107895c501a5df163
BLAKE2b-256 checksum
How to use checksums
51d7c4288b883f175226e967772875c3761bbcb27e2fd2ff42232a0ef4f84634
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.7

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.

Transparency log

Release history Release notifications | RSS feed

This release

1.1.1 This release

11 release files

1.1.0

10 release files

1.0.0

1 release file

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page