Skip to main content

pitchfork

Rust-powered search engine for Django's haystack ecosystem. Name: pitchfork (the tool you work a haystack with).

Install pitchfork-engine; import pitchfork — the PyPI name pitchfork belongs to an unrelated, abandoned project:

pip install pitchfork-engine

The Django-facing haystack replacement API ships separately as django-pitchfork on top of this engine.

Evolution path: C → B —

  • Strategy C (shipped): a standalone Rust search server (pitchfork-server, Tantivy-backed, axum HTTP) with a Haystack backend adapter (pitchfork.backends.server.PitchforkEngine). Unmodified django-haystack works over HTTP; switching HAYSTACK_CONNECTIONS["default"]["ENGINE"] from WhooshEngine to PitchforkEngine requires no other change.
  • Strategy B (shipped): embed the same Rust core in-process via PyO3 (pitchfork._core), GIL released during search and indexing. pitchfork.backends.embedded.AutoEngine picks this mode when the compiled extension is importable, and falls back to server mode otherwise.

Layout (layered bottom-up, no back-edges)

crates/
├── pitchfork-core/     # Layer 1: pure Rust — schema, query AST, wire protocol
├── pitchfork-engine/   # Layer 2: Tantivy-backed SearchEngine impl
└── pitchfork-server/   # Layer 3: axum HTTP service (Strategy C)
python/                 # pitchfork package (adapter + wire codec)
pitchfork_tests/        # Backend test suite (starts its own server fixture)

Quick start

just build        # cargo build (server binary + crates)
just test-rs      # native Rust tests
just test-py      # Python backend suite against a live server fixture
just clippy       # lint gate (-D warnings)

Migrating an existing haystack project: see docs/MIGRATION.md.

Wire protocol: PitchforkWireV1 — one versioned contract; every cross-boundary type adds the Python constant (pitchfork/wire.py) and the Rust static (pitchfork-core/src/wire.rs) in the same commit.

Name map

Thing Name
Import package pitchfork
PyPI distribution pitchfork-engine (pitchfork on PyPI is an unrelated abandoned project)
Native extension module pitchfork._core (private)
Rust crates / server binary pitchfork-core, pitchfork-engine, pitchfork-server
Haystack-replacement API django-pitchfork — a separate package, not this repo

Supported features

Full-text, all haystack filter ops, sort, pagination, facets, highlighted snippets, more-like-this, FST-based spelling suggestions (Levenshtein automaton over the term dictionary), and geospatial within (bounding box) / dwithin (haversine radius) queries. Date facets / query facets and true spatial indexing (s2/h3) are later-release work.

Filter-op semantics worth knowing (0.2):

  • __contains is a substring match (*value*), like every haystack backend; previously it was compiled as exact. On analyzed text fields the needle is matched case-insensitively (the default tokenizer lowercases); raw indexed fields are case-sensitive. The substring is regex-escaped — metacharacters never act as wildcards.
  • Bare lookups (filter(field=value), no __lookup) match typed terms on typed fields (int/float/bool/date), like whoosh's %s fragment — not silently nothing.
  • Facet counts are computed over the whole match set, not the current page; a client FACET_SCAN_LIMIT (default 10 000) caps the scan and the response flags truncated counts via facets["truncated"].
  • order_by works on text/char/ngram fields (fast fields), and multiple sort keys are honoured if they share one direction — whoosh's own all-or-nothing rule; mixed asc/desc is a loud error.
  • boost(...) and distance_point are accepted for API compatibility but warn: V1 has no boost IR op and no distance annotation yet.

Rebuild required when upgrading from 0.1: the on-disk schema now enables text fast fields (needed for sorting), and contains semantics changed — run manage.py rebuild_index after upgrading.

Hardening

  • just clippy gates every merge (-D warnings).
  • Property tests on both sides of the wire parser: proptest! next to the Rust parser, Hypothesis in pitchfork_tests/test_fuzz.py.
  • .gitlab-ci.yml (the scheduled fuzz pipeline): weekly extended fuzz runs; on-demand mutation testing (cargo-mutants + mutmut, .cargo/mutants.toml).
  • Coverage both languages: cargo-llvm-cov + pytest-cov (CI coverage job).
  • Methodology: docs/PROFILING.md; recorded numbers: BENCHMARK.md.

Metadata

Release files for pitchfork-engine 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pitchfork-engine 0.1.0
File Size Uploaded Provenance Actions
pitchfork_engine-0.1.0.tar.gz 159.4 kB Without provenance Details for pitchfork_engine-0.1.0.tar.gz

Built distributions (wheels)

Table of built distributions (wheels) for pitchfork-engine 0.1.0
File Interpreter ABI Platform Actions
pitchfork_engine-0.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl CPython 3.14 CPython 3.14 Linux glibc 2.17+ x86-64 Details for pitchfork_engine-0.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
pitchfork_engine-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl CPython 3.10 CPython 3.10 Linux glibc 2.17+ x86-64 Details for pitchfork_engine-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl

Total release size: 5.6 MB

Release files / pitchfork_engine-0.1.0.tar.gz

Download URL pitchfork_engine-0.1.0.tar.gz
Size 159.4 kB
Tags Source
SHA-256 checksum
How to use checksums
849b89634b2f74a8420f021d68b10c2b4b7455a9600090ee0451a9705ca01213
BLAKE2b-256 checksum
How to use checksums
7a2f807606a160b64d63dbfdb747f4fe83f174e3425b5bc4079206a635ce677d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.16

Release files / pitchfork_engine-0.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl

Download URL pitchfork_engine-0.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Size 2.7 MB
Tags CPython 3.14 Linux glibc 2.17+ x86-64
SHA-256 checksum
How to use checksums
b5488d3e0ab0a17bad02e5c85d6d55c0374f4147fda51f7a2e6dbe005cf7a4f5
BLAKE2b-256 checksum
How to use checksums
b426d5e11bfbf977703f89591a9b9a4f4e336a30ee0a16adbad838bf9480a722
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.16

Release files / pitchfork_engine-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl

Download URL pitchfork_engine-0.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl
Size 2.7 MB
Tags CPython 3.10 Linux glibc 2.17+ x86-64
SHA-256 checksum
How to use checksums
ea23a6dc95a48c62d03ea85e5a6621bb12c52bf67a89969dbb45bf2372870bc1
BLAKE2b-256 checksum
How to use checksums
456fa7e5cffec5972c70dd1cae233db5bab8ef2febc81caa9b4f40de0bf2e589
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.16

Security for pitchfork-engine 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

3 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page