pitchfork
Rust-powered search engine for Django's haystack ecosystem. Name: pitchfork (the tool you work a haystack with).
Install pitchfork-engine; import pitchfork — the PyPI name pitchfork
belongs to an unrelated, abandoned project:
pip install pitchfork-engine
The Django-facing haystack replacement API ships separately as
django-pitchfork on top
of this engine.
Evolution path: C → B —
- Strategy C (shipped): a standalone Rust search server
(
pitchfork-server, Tantivy-backed, axum HTTP) with a Haystack backend adapter (pitchfork.backends.server.PitchforkEngine). Unmodified django-haystack works over HTTP; switchingHAYSTACK_CONNECTIONS["default"]["ENGINE"]from WhooshEngine to PitchforkEngine requires no other change. - Strategy B (shipped): embed the same Rust core in-process via PyO3
(
pitchfork._core), GIL released during search and indexing.pitchfork.backends.embedded.AutoEnginepicks this mode when the compiled extension is importable, and falls back to server mode otherwise.
Layout (layered bottom-up, no back-edges)
crates/
├── pitchfork-core/ # Layer 1: pure Rust — schema, query AST, wire protocol
├── pitchfork-engine/ # Layer 2: Tantivy-backed SearchEngine impl
└── pitchfork-server/ # Layer 3: axum HTTP service (Strategy C)
python/ # pitchfork package (adapter + wire codec)
pitchfork_tests/ # Backend test suite (starts its own server fixture)
Quick start
just build # cargo build (server binary + crates)
just test-rs # native Rust tests
just test-py # Python backend suite against a live server fixture
just clippy # lint gate (-D warnings)
Migrating an existing haystack project: see docs/MIGRATION.md.
Wire protocol: PitchforkWireV1 — one versioned contract; every
cross-boundary type adds the Python constant (pitchfork/wire.py)
and the Rust static (pitchfork-core/src/wire.rs) in the same commit.
Name map
| Thing | Name |
|---|---|
| Import package | pitchfork |
| PyPI distribution | pitchfork-engine (pitchfork on PyPI is an unrelated abandoned project) |
| Native extension module | pitchfork._core (private) |
| Rust crates / server binary | pitchfork-core, pitchfork-engine, pitchfork-server |
| Haystack-replacement API | django-pitchfork — a separate package, not this repo |
Supported features
Full-text, all haystack filter ops, sort, pagination, facets, highlighted
snippets, more-like-this, FST-based spelling suggestions (Levenshtein
automaton over the term dictionary), and geospatial within (bounding box) /
dwithin (haversine radius) queries. Date facets / query facets and true
spatial indexing (s2/h3) are later-release work.
Filter-op semantics worth knowing (0.2):
__containsis a substring match (*value*), like every haystack backend; previously it was compiled asexact. On analyzed text fields the needle is matched case-insensitively (the default tokenizer lowercases); raw indexed fields are case-sensitive. The substring is regex-escaped — metacharacters never act as wildcards.- Bare lookups (
filter(field=value), no__lookup) match typed terms on typed fields (int/float/bool/date), like whoosh's%sfragment — not silently nothing. - Facet counts are computed over the whole match set, not the current
page; a client
FACET_SCAN_LIMIT(default 10 000) caps the scan and the response flags truncated counts viafacets["truncated"]. order_byworks on text/char/ngram fields (fast fields), and multiple sort keys are honoured if they share one direction — whoosh's own all-or-nothing rule; mixed asc/desc is a loud error.boost(...)anddistance_pointare accepted for API compatibility but warn: V1 has no boost IR op and no distance annotation yet.
Rebuild required when upgrading from 0.1: the on-disk schema now enables text fast fields (needed for sorting), and contains semantics changed — run
manage.py rebuild_indexafter upgrading.
Hardening
just clippygates every merge (-D warnings).- Property tests on both sides of the wire parser:
proptest!next to the Rust parser, Hypothesis inpitchfork_tests/test_fuzz.py. .gitlab-ci.yml(the scheduled fuzz pipeline): weekly extended fuzz runs; on-demand mutation testing (cargo-mutants + mutmut,.cargo/mutants.toml).- Coverage both languages: cargo-llvm-cov + pytest-cov (CI
coveragejob). - Methodology:
docs/PROFILING.md; recorded numbers:BENCHMARK.md.
Metadata
Release files for pitchfork-engine 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| pitchfork_engine-0.1.0.tar.gz | 159.4 kB | Without provenance | Details for pitchfork_engine-0.1.0.tar.gz |
Built distributions (wheels)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| pitchfork_engine-0.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl | CPython 3.14 | CPython 3.14 | Linux glibc 2.17+ x86-64 | Details for pitchfork_engine-0.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
| pitchfork_engine-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl | CPython 3.10 | CPython 3.10 | Linux glibc 2.17+ x86-64 | Details for pitchfork_engine-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
Total release size: 5.6 MB
Release files / pitchfork_engine-0.1.0.tar.gz
| Download URL | pitchfork_engine-0.1.0.tar.gz |
|---|---|
| Size | 159.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
849b89634b2f74a8420f021d68b10c2b4b7455a9600090ee0451a9705ca01213
|
|
BLAKE2b-256 checksum How to use checksums |
7a2f807606a160b64d63dbfdb747f4fe83f174e3425b5bc4079206a635ce677d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.16
|
Release files / pitchfork_engine-0.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
| Download URL | pitchfork_engine-0.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | CPython 3.14 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
b5488d3e0ab0a17bad02e5c85d6d55c0374f4147fda51f7a2e6dbe005cf7a4f5
|
|
BLAKE2b-256 checksum How to use checksums |
b426d5e11bfbf977703f89591a9b9a4f4e336a30ee0a16adbad838bf9480a722
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.16
|
Release files / pitchfork_engine-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
| Download URL | pitchfork_engine-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl |
|---|---|
| Size | 2.7 MB |
| Tags | CPython 3.10 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
ea23a6dc95a48c62d03ea85e5a6621bb12c52bf67a89969dbb45bf2372870bc1
|
|
BLAKE2b-256 checksum How to use checksums |
456fa7e5cffec5972c70dd1cae233db5bab8ef2febc81caa9b4f40de0bf2e589
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.16
|
Security for pitchfork-engine 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.