FastQ
A high-performance job queue written in C, backed by Redis.
FastQ aims to be faster, leaner on RAM, and polyglot compared to existing job queue solutions like BullMQ, Sidekiq, or Celery.
Features
Core (free)
- Fast push/pop via Redis Lists with MULTI/EXEC pipelining
- Priority queues (high / normal / low)
- Multi-threaded worker pool (pthreads) with Redis connection pooling
- Automatic retries with exponential backoff
- Dead Letter Queue for permanently failed jobs
- Crash recovery for orphaned jobs
- Daemon mode with systemd support
- Python bindings (CPython C extension)
- Node.js bindings (N-API)
- CLI tool (
fastq push,pop,stats,worker,recover) - Colored logging (DEBUG/INFO/WARN/ERROR)
- Zero external runtime dependencies beyond Redis
Pro
- Cron scheduler — persistent cron jobs and one-shot delayed jobs, survive restarts
- Rate limiting — token-bucket limiter attached per worker
- Batch processing — pop up to N jobs in one shot with a deadline
- Job chaining — automatically push a child job when a parent completes
- DAG workflows — multi-step dependency graphs; dependents unlock atomically
- Metrics — Prometheus
/metricsand JSON/healthHTTP endpoint - HMAC license — key verification with constant-time comparison (OpenSSL)
Building
Dependencies
- hiredis — Redis C client
- json-c — JSON parsing
- OpenSSL — HMAC-SHA256 (license + TLS)
- Redis server (runtime)
Compile
make # library + CLI
make test # run all tests
make bench # throughput benchmark
make python # Python C extension
make node # Node.js N-API addon
Run tests
# Start a dev Redis instance (in a separate terminal)
./run_redis.sh
make test
# Run tests under Valgrind
make valgrind
Benchmark
make bench
Python bindings
make python
python3 -c "import fastq; q = fastq.Queue('myqueue'); q.push('{\"hello\":1}')"
Node.js bindings
make node
node -e "const { Queue } = require('./bindings/node'); const q = new Queue('127.0.0.1', 6379, 'myqueue'); console.log(q.push('{\"hello\":1}'));"
Project Structure
fastq/
├── src/ # Core library source
├── include/ # Public headers
├── tests/ # Unit and integration tests (38 tests, 7 suites)
├── benchmarks/ # Throughput benchmarks
├── bindings/
│ ├── python/ # Python CPython extension
│ └── node/ # Node.js N-API addon
├── docs/ # Architecture & API design docs
├── Makefile
├── fastq.service # systemd unit file
└── run_redis.sh # Dev Redis launcher
Roadmap
Done
- Core engine, priority queues, multi-threaded workers, connection pooling
- Crash recovery, daemon mode, CLI
- Python and Node.js bindings
- Cron scheduler, rate limiter, batch processing
- Job chaining and DAG workflows
- Prometheus metrics, health endpoint
- HMAC license system
- 38 tests across 7 suites, all passing
Next
- Go / Rust bindings
- Web UI
- Kubernetes operator
Metadata
Release files for fastq-redis 1.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| fastq_redis-1.1.1.tar.gz | 6.3 kB | Details |
Built distributions (wheels)
Total release size: 36.0 MB
Release files / fastq_redis-1.1.1.tar.gz
| Download URL | fastq_redis-1.1.1.tar.gz |
|---|---|
| Size | 6.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
aa23c0101c5ff96d4a92c66e27b1a34ec0a2e1035a47dbd88c6286354bc794a6
|
|
BLAKE2b-256 checksum How to use checksums |
6dd059575005951106b47150775bebf2f3c24b274650cac4fee14c88527e3994
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | fastq_redis-1.1.1-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 4.5 MB |
| Tags | CPython 3.12 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
24f298c8740e9df7b35259110220daa8fd89eb9107c57a1939c982f0124991d5
|
|
BLAKE2b-256 checksum How to use checksums |
148be4975c179097997b959b559b721e24dde30c5709dbc991703f3ea3468f5c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp312-cp312-macosx_11_0_arm64.whl
| Download URL | fastq_redis-1.1.1-cp312-cp312-macosx_11_0_arm64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | CPython 3.12 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
01c1714df6e07cc440e162d8d161eeb35c729fc253126b017fe9da11d66d7701
|
|
BLAKE2b-256 checksum How to use checksums |
a2fac9b2bd7a7905d68bed33dfa3051579f8f4ca1e59e291351f76230dd0bb1e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | fastq_redis-1.1.1-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 4.5 MB |
| Tags | CPython 3.11 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
d6cebec2ba4492dccec3b08601845dc941b7b85c1cf41a4016a9777eaa4e69a3
|
|
BLAKE2b-256 checksum How to use checksums |
0eec47d53153447ccf962a7b039e7b366171d934f1c223bf7d6c58eb107eb85c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp311-cp311-macosx_11_0_arm64.whl
| Download URL | fastq_redis-1.1.1-cp311-cp311-macosx_11_0_arm64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | CPython 3.11 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
8d0207ed444bd0f5b343456825f90d0df87365cb23c681af0b1ceaa4a814ec9f
|
|
BLAKE2b-256 checksum How to use checksums |
1b4da52f8685a8e195a7aca4c343cefee71c9de4abed062519f3a3f472910b3b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | fastq_redis-1.1.1-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 4.5 MB |
| Tags | CPython 3.10 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
9c81297c07d4f538b3fbb03bd662f123ee51b54038e977cfd3900000f2f151cc
|
|
BLAKE2b-256 checksum How to use checksums |
6537159df0cd3385355703cee6f98ab52fbc24809f5162ffc58d76729b0ca8c3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp310-cp310-macosx_11_0_arm64.whl
| Download URL | fastq_redis-1.1.1-cp310-cp310-macosx_11_0_arm64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | CPython 3.10 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
07eb1f5cfd2f659f3134b5edbb59415f9c87857950f9b64cf6e7314476412fa0
|
|
BLAKE2b-256 checksum How to use checksums |
bcf5e94e2db1b5f2e80896b0a4af077f2e164fa41352db3204ed48848a154c4d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | fastq_redis-1.1.1-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 4.5 MB |
| Tags | CPython 3.9 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
f49378bc56d6d8f142ea939754d52679df2390008a81beea0437de0434676c51
|
|
BLAKE2b-256 checksum How to use checksums |
aa58261f8fc2e898aab1b407314b8791055f20ee679dc0ff3bb75f66ea71ac80
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp39-cp39-macosx_11_0_arm64.whl
| Download URL | fastq_redis-1.1.1-cp39-cp39-macosx_11_0_arm64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | CPython 3.9 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
5adc4207d5993aea258ad5ae5c38ce97d3de2bcea184494cc3a5dbb6b84ca331
|
|
BLAKE2b-256 checksum How to use checksums |
e519a6371f3174db84fd01e6aad70a78e7be215644f352f8d0298cbed600cfa0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl
| Download URL | fastq_redis-1.1.1-cp38-cp38-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl |
|---|---|
| Size | 4.5 MB |
| Tags | CPython 3.8 Linux glibc 2.27+ x86-64 Linux glibc 2.28+ x86-64 |
|
SHA-256 checksum How to use checksums |
b4a883e135f417a6962aa8456cb6c0c3e99b8373ece9869343d9a00577f2f8ac
|
|
BLAKE2b-256 checksum How to use checksums |
1d7ce439e4501cbde9023ef8bdcbb8251af741ef36b0120bcc5772e4a8d86948
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency logRelease files / fastq_redis-1.1.1-cp38-cp38-macosx_11_0_arm64.whl
| Download URL | fastq_redis-1.1.1-cp38-cp38-macosx_11_0_arm64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | CPython 3.8 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
79785c2acdb978414445d81648907644573435fcc5ea9a3107895c501a5df163
|
|
BLAKE2b-256 checksum How to use checksums |
51d7c4288b883f175226e967772875c3761bbcb27e2fd2ff42232a0ef4f84634
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Mar 29, 2026.
Transparency log