Wagtailmeili

A search backend for Wagtail using MeiliSearch.

[!CAUTION] This package is still in development and until version 1.0.0, I will not maintain a DeprecationWarning pattern. I built the integration with meilisearch about 2 years ago for a project and decided to make it a public package to improve it and integrate more features.

[!TIP]
If you need support or require help with a Wagtail project, you can hire me 😊

Introduction

en - https://softquantum.com/resources/wagtailmeili-integrating-a-blazing-fast-search-engine-with-wagtail

fr - https://softquantum.com/fr/ressources/wagtailmeili-integrer-un-moteur-de-recherche-rapide-avec-wagtail/

Requirements

Wagtailmeili requires the following:

• Python >= 3.11
• Wagtail >= 5.2

Installation

In your Wagtail project

Configure your MeiliSearch instance in your settings.py file.

Install Meilisearch python client e.g., using pip

  pip install meilisearch

Add wagtailmeili to your INSTALLED_APPS

INSTALLED_APPS = [
    # ...
    "wagtailmeili",
    # ...
]

add the search backend 'meilisearch' to your WAGTAILSEARCH_BACKENDS

[!CAUTION] Leave the 'default' backend for the admin as you don't want to depend only on what was indexed in meilisearch Different use cases to consider so still work in progress.

import os

WAGTAILSEARCH_BACKENDS = {
    "meilisearch": {
        "BACKEND": "wagtailmeili.backend",
        "HOST":  os.environ.get("MEILISEARCH_HOST", "http://127.0.0.1"),
        "PORT": os.environ.get("MEILISEARCH_PORT", "7700"),
        "MASTER_KEY": os.environ.get("MEILISEARCH_MASTER_KEY", "your-master-key"),
        # "STOP_WORDS": ...
        # "RANKING_RULES: ...
        # "SKIP_MODELS": ...
        # "SKIP_MODELS_BY_FIELD_VALUE": ...
    },
    "default": {
        "BACKEND": "wagtail.search.backends.database",
    }
}

Features

Default search configs

• STOP_WORDS: see defaults in settings.py
• RANKING_RULES: see defaults in settings.py
• SKIP_MODELS: "skip_models" is a list of models that you want to skip from indexing no matter the model setup.

WAGTAILSEARCH_BACKENDS = {
    "meilisearch": {
        "SKIP_MODELS": ["app_label.Model1", "app_label.Model2",],
        # ...
    }
}

• SKIP_MODELS_BY_FIELD_VALUE: A convenient way to skip instances based on attributes

WAGTAILSEARCH_BACKENDS = {
    "meilisearch": {
        # add this to not index pages that are not published for example
        "SKIP_MODELS_BY_FIELD_VALUE": {
            "wagtailmeili_testapp.MoviePage": {
                "field": "live",
                "value": False,
            },
        },
        # ...
    }
}

Model fields

In any model you will be doing a search on with Meilisearch, add the page or model manager.
It will use the correct backend when doing something like MySuperPage.objects.search().

from wagtail.models import Page
from django.db import models
from wagtailmeili.manager import MeilisearchPageManager, MeilisearchModelManager

class MySuperPage(Page):
    """A Super Page to do incredible things indexed in Meilisearch."""
 
    objects = MeilisearchPageManager()

class MySuperModel(models.Model):
    """A Super Model to do incredible things indexed in Meilisearch."""
 
    objects = MeilisearchModelManager()

To declare sortable attributes or add ranking rules for the model, just add, for example:

from wagtail.models import Page


class MySuperPage(Page):
    """A Super Page to do incredible things indexed in Meilisearch."""
    
    sortable_attributes = [
        "published_last_timestamp", 
        # ...
    ]
    ranking_rules = [
        "published_last_timestamp:desc",
        # ...
    ]

Template Tag filter

{% load meilisearch %}

{% get_matches_position result %}

Index Cleanup

Wagtailmeili automatically handles cleanup of stale documents from your MeiliSearch indexes to ensure search results remain accurate and up-to-date.

Automatic Cleanup

Real-time Cleanup: When items are deleted or unpublished, they are automatically removed from the search index via Django signals.

Rebuild Cleanup: When running python manage.py update_index, stale documents are automatically detected and removed.

Smart Detection: The system automatically handles both regular models and Page models with live field detection.

Manual Cleanup

Use the management command for manual cleanup operations:

# Clean all indexes (dry-run mode)
python manage.py cleanup_search_index --dry-run

# Clean all indexes
python manage.py cleanup_search_index

# Clean specific model
python manage.py cleanup_search_index --model wagtailmeili_testapp.MoviePage

# Verbose output
python manage.py cleanup_search_index --verbosity 2

Programmatic Cleanup

You can also perform cleanup operations programmatically:

from wagtail.search.backends import get_search_backend

# Get the MeiliSearch backend
backend = get_search_backend('meilisearch')
index = backend.get_index_for_model(MyModel)

# Remove specific items
index.delete_item(item_id)

# Bulk remove multiple items
index.bulk_delete_items([id1, id2, id3])

# Clean up stale documents
live_ids = MyModel.objects.filter(live=True).values_list('pk', flat=True)
index.cleanup_stale_documents(live_ids)

Configuration

The cleanup system respects your existing configuration:

• SKIP_MODELS: Models in this list won't be cleaned up
• SKIP_MODELS_BY_FIELD_VALUE: Field-based skipping is honored during cleanup
• Page Models: Only live pages (live=True) are considered valid for Page models

Roadmap before 1.0.0 (unsorted)

• -[x] Adding tests (v0.3.3)
• -[x] Cleaning up index if pages are unpublished or models deleted (v0.4.0)
• -[ ] Exploring Meilisearch and bringing more of its features for Wagtail
• -[ ] Getting a leaner implementation (looking at Autocomplete and rebuilder)
• -[ ] Giving more love to the Sample project with a frontend
• -[ ] official documentation

Sample Project: WMDB

The Wagtail Movie Database (WMDB) is a sample project for testing purposes. To run the project, follow these steps:

1. start the local meilisearch instance

meilisearch --master-key=<your masterKey>

2. copy the directory wagtail_moviedb wherever you want
3. create a virtualenv and activate it (instructions on linux/macOS)

python -m venv .venv
source .venv/bin/activate

4. install the dependencies

pip install -r requirements.txt

5. add an .env file

MEILISEARCH_MASTER_KEY="your masterKey"

6. apply migrations

python manage.py migrate

7. Create a superuser (optional)

python manage.py createsuperuser

8. load movies & start local web server

python manage.py load_movies
python manage.py runserver

9. visit your meilisearch local instance: https://127.0.0.1:7700, give it your master-key. You should see some movies loaded.
10. update index (optional):

python manage.py update_index

Development

Development Setup

This package uses flit for both local development and publishing.

1. Install flit (if not already available):

# Via pip
pip install flit

# Via pyenv (if you have a Python version with flit pre-installed)
# flit may already be available in your pyenv Python installation

2. Install the package locally for development:

# Using pip
pip install "pip>=21.3"
pip install -e '.[dev]' -U

# Using flit
flit install -s

For more information on installing and using flit, see the official flit documentation.

Contributions

Welcome to all contributions and reviews!

Prerequisites

• Install Meilisearch locally following their documentation
• Start Meilisearch instance in your favorite terminal

meilisearch --master-key correctmasterkey

Install

To make changes to this project, first clone this repository:

git clone git@github.com:softquantum/wagtailmeili.git
cd wagtailmeili

With your preferred virtualenv activated, install testing dependencies:

Using pip

pip install "pip>=21.3"
pip install -e '.[dev]' -U

How to run tests

You can run tests as shown below:

pytest 

or with tox

tox

Disclaimer

• ✅ This project is to experiment with my dev experience and improve my skills.
• ✅ It is, since v0.3, developed in an augmented development setup (JetBrains Pycharm, Claude Code with custom commands and configs)
• ✅ I commit to have a test suite that makes sense (reviews are welcome)
• ✅ The project is used in production in real projects: no shortcuts for quality standards, so if you find a bug please report it.
• ✅ It is an open source project so you can hire me for support ☕️

License

This project is released under the 3-Clause BSD License.