Wagtail MeiliSearch

This is a Wagtail search backend for the MeiliSearch search engine.

Installation

uv add wagtail_meilisearch or pip install wagtail_meilisearch

Upgrading

If you're upgrading MeiliSearch from 0.9.x to anything higher, you will need to destroy and re-create MeiliSearch's data.ms directory.

Requirements

• Python >=3.10
• wagtail >=6.0
• meilisearch-python >= 0.36.0

Tested against Meilisearch server v1.15.2 - latest at the time of writing.

Hybrid search is tested against Meilisearch server v1.18.0 and meilisearch-python 0.42.0.

Configuration

See the MeiliSearch docs for info on the values you want to add here.

WAGTAILSEARCH_BACKENDS = {
    'default': {
        'BACKEND': 'wagtail_meilisearch.backend',
        'HOST': os.environ.get('MEILISEARCH_HOST', 'http://127.0.0.1'),
        'PORT': os.environ.get('MEILISEARCH_PORT', '7700'),
        'MASTER_KEY': os.environ.get('MEILI_MASTER_KEY', '')
    },
}

Update strategies

Indexing a very large site with python manage.py update_index can be pretty taxing on the CPU, take quite a long time, and reduce the responsiveness of the MeiliSearch server. Wagtail-MeiliSearch offers two update strategies, soft and hard. The default, soft strategy will do an "add or update" call for each document sent to it, while the hard strategy will delete every document in the index and then replace them.

There are tradeoffs with either strategy - hard will guarantee that your search data matches your model data, but be hard work on the CPU for longer. soft will be faster and less CPU intensive, but if a field is removed from your model between indexings, that field data will remain in the search index.

Delta strategy

The delta strategy is useful if you habitually add created_at and updated_at timestamps to your models. This strategy will check the fields...

• first_published_at
• last_published_at
• created_at
• updated_at

And only update the records for objects where one or more of these fields has a date more recent than the time delta specified in the settings.

WAGTAILSEARCH_BACKENDS = {
    'default': {
        'BACKEND': 'wagtail_meilisearch.backend',
        'HOST': os.environ.get('MEILISEARCH_HOST', 'http://127.0.0.1'),
        'PORT': os.environ.get('MEILISEARCH_PORT', '7700'),
        'MASTER_KEY': os.environ.get('MEILI_MASTER_KEY', '')
        'UPDATE_STRATEGY': delta,
        'UPDATE_DELTA': {
            'weeks': -1
        }
    }
}

If the delta is set to {'weeks': -1}, wagtail-meilisearch will only update indexes for documents where one of the timestamp fields has a date within the last week. Your time delta must be a negative.

Under the hood we use Arrow, so you can use any keyword args supported by Arrow's shift().

If you set UPDATE_STRATEGY to delta but don't provide a value for UPDATE_DELTA wagtail-meilisearch will default to {'weeks': -1}.

Skip models

Sometimes you might have a site where a certain page model is guaranteed not to change, for instance an archive section. After creating your initial search index, you can add a SKIP_MODELS key to the config to tell wagtail-meilisearch to ignore specific models when running update_index. Behind the scenes wagtail-meilisearch returns a dummy model index to the update_index management command for every model listed in your SKIP_MODELS - this ensures that this setting only affects update_index, so if you manually edit one of the models listed it should get re-indexed with the update signal.

WAGTAILSEARCH_BACKENDS = {
    'default': {
        'BACKEND': 'wagtail_meilisearch.backend',
        'HOST': os.environ.get('MEILISEARCH_HOST', 'http://127.0.0.1'),
        'PORT': os.environ.get('MEILISEARCH_PORT', '7700'),
        'MASTER_KEY': os.environ.get('MEILI_MASTER_KEY', ''),
        'UPDATE_STRATEGY': 'delta',
        'SKIP_MODELS': [
            'core.ArchivePage',
        ]
    }
}

Stop Words

Stop words are words for which we don't want to place significance on their frequency. For instance, the search query tom and jerry would return far less relevant results if the word and was given the same importance as tom and jerry. There's a fairly sane list of English language stop words supplied, but you can also supply your own. This is particularly useful if you have a lot of content in any other language.

MY_STOP_WORDS = ['a', 'list', 'of', 'words']

WAGTAILSEARCH_BACKENDS = {
    'default': {
        'BACKEND': 'wagtail_meilisearch.backend',
        [...]
        'STOP_WORDS': MY_STOP_WORDS
    },
}

Or alternatively, you can extend the built in list.

from wagtail_meilisearch.settings import STOP_WORDS

MY_STOP_WORDS = STOP_WORDS + WELSH_STOP_WORDS + FRENCH_STOP_WORDS

WAGTAILSEARCH_BACKENDS = {
    'default': {
        'BACKEND': 'wagtail_meilisearch.backend',
        [...]
        'STOP_WORDS': MY_STOP_WORDS
    },
}

Ranking

We now support Meilisearch's native ranking system which is considerably faster than the rather hacky way we were having to do it before. Meilisearch takes a list of fields ordered by precedence to affect the attribute ranking so we build that list by inspecting the index.SearchFields and index.AutocompleteFields on each model and ordering by boost. As an example, if you want the page title to be the most important field to rank on...

search_fields = Page.search_fields + [
    index.AutocompleteField("title", boost=10),
    index.SearchField("body"),
    index.SearchField("search_description", boost=5),
]

Any field that doesn't have a boost value will be given a default of 0 but will still be sent to Meilisearch's settings as part of the ordered list, so the above settings send an attribute ranking order to Meilisearch of...

['title', 'search_description', 'body']

In the backend, we automatically annotate the search results with their ranking, with a float between 0 and 1 as search_rank so in your search view you can sort by that value.

def search_view(request):
    search_query = request.GET.get('query', '')
    search_results = Page.objects.search(search_query)

    # Results are already sorted by search_rank
    # You can access the rank for each result
    for result in search_results:
        print(f"Result: {result.title}, Rank: {result.search_rank}")

    return render(request, 'search_results.html.j2', {
        'search_query': search_query,
        'search_results': search_results,
    })

And you might even fancy using the search rank in your template...

{% for result in search_results %}
    <div class="result {% if result.search_rank > 0.8 %}high-relevance{% endif %}">
        <h3>{{ result.title }}</h3>
        <p>Relevance: {{ result.search_rank }}</p>
    </div>
{% endfor %}

Faceting

We now support faceting. In order to use it, you need to add FilterFields to your model on any field that you might want to facet on...

search_fields = Page.search_fields + [
    index.AutocompleteField("title", boost=10),
    index.SearchField("body"),
    index.SearchField("search_description", boost=5),
    index.FilterField("category"),
]

With that in place, you can call facet on a search to get an OrderedDict of the facet values and their counts. By default, Wagtail adds several FilterFields to the Page model too, so for instance you can get the facet results of content_type_id with...

Page.objects.search("query").facet("content_type")

# OrderedDict([('58', 197), ('75', 2), ('52', 1), ('54', 1), ('61', 1)])

The ordered dict contains tuples of the form (value, count) where value: str is the value of the field (typically its pk) and count is the number of documents that have that value.

Filtering

Armed with your facet counts, you can filter your search results by passing filters to the filter method. For example, to filter by content_type_id...

Page.objects.search("query").filter(filters=[("content_type", "58")])

# <PageQuerySet [<Page: Page 1>, <Page: Page 2>, ...]

The filters param should be a list of tuples, where each tuple is of the form (field, value). Being a list, you can pass multiple tuples to filter by multiple fields. For example, to filter by content_type and category...

Page.objects.search("query").filter(filters=[("content_type", "58"), ("category", "1")])

# <PageQuerySet [<Page: Page 1>, <Page: Page 2>, ...]

And finally, you can choose the operator for the filter. By default, the operator is AND, but you can also use OR...

Page.objects.search("query").filter(filters=[("content_type", "58"), ("category", "1")], operator="OR")

# <PageQuerySet [<Page: Page 1>, <Page: Page 2>, ...]

Query limits

If you have a lot of DB documents, the final query to the database can be quite a heavy load. Meilisearch's relevance means that it's usually pretty safe to restrict the number of documents Meilisearch returns, and therefore the number of documents your app needs to get from the database. The limit is per model, so if your project has 10 page types and you set a limit of 1000, there's a possible 10000 results.

WAGTAILSEARCH_BACKENDS = {
    'default': {
        'BACKEND': 'wagtail_meilisearch.backend',
        [...]
        'QUERY_LIMIT': 1000
    },
}

Hybrid search (semantic search)

Keyword search only finds pages that contain the words in the query. Hybrid search also finds pages with a similar meaning, so "laptop for travelling" can find a page about "lightweight ultrabooks". Meilisearch does this with an embedding model, which turns text into a list of numbers (a vector). Meilisearch calls the embedding model itself, both when it indexes a document and when it runs a search, so wagtail-meilisearch never handles vectors.

Hybrid search is off unless you add EMBEDDERS. Without it, nothing changes.

Configuration

EMBEDDERS uses Meilisearch's own embedder settings, keyed by embedder name. This example uses a local Ollama server:

WAGTAILSEARCH_BACKENDS = {
    'default': {
        'BACKEND': 'wagtail_meilisearch.backend',
        [...]
        'EMBEDDERS': {
            'site': {
                'source': 'ollama',
                'url': 'http://127.0.0.1:11434/api/embed',
                'model': 'qwen3-embedding:0.6b',
                'dimensions': 1024,
            },
        },
        'SEMANTIC_RATIO': 0.4,
        'SEMANTIC_SCORE_THRESHOLD': 0.78,
    },
}

The url is called by the Meilisearch server, not by Django, so it must work from wherever Meilisearch runs.

After you add or change EMBEDDERS, run python manage.py update_index. Meilisearch then embeds every document again, which takes time on a large site. When the embedder settings have not changed, update_index does not cause any embedding.

Set a semantic score threshold

Every document has some similarity to every query, so a search with a semantic part returns every document in the index unless you set SEMANTIC_SCORE_THRESHOLD. The right value depends on the embedding model and your content. Search for a few unrelated words, look at the search_rank of the results, and set the threshold above those scores. As an example, with qwen3-embedding:0.6b unrelated pages scored about 0.70 to 0.78 and good matches about 0.8 and up.

The threshold only applies to results found by meaning. A page that contains the search words is always in the results, even when its score is low. A word that only appears in a low priority field, such as the body, can score as little as 0.5. To do this, a hybrid search with a threshold sends a second, keyword only query to each index.

Choose a semantic ratio

Hybrid results are ordered the way Meilisearch orders them: keyword scores are weighted by 1 - SEMANTIC_RATIO and scores by meaning by SEMANTIC_RATIO. Some embedding models give even unrelated text a fairly high score. With qwen3-embedding:0.6b, unrelated pages scored about 0.75, so at a ratio of 0.5 a page with the search word in its body ranked below pages that were only loosely related. A ratio of 0.4 put those pages first and still found pages by meaning. Test with words that only appear in page bodies, and with queries that share no words with the page you expect.

search_rank holds the weighted score, divided by the larger of the two weights. At a ratio of 0.5 or lower, a keyword match keeps its usual 0 to 1 score.

Change the options for one search

hybrid() changes the options for one search. Options you leave out use the settings.

Page.objects.search("laptop for travelling").hybrid(semantic_ratio=0.8)
Page.objects.search("SKU-12345").hybrid(semantic_ratio=0)  # keyword only
Page.objects.search("garden furniture").hybrid(semantic_score_threshold=0.85)

Autocomplete searches are always keyword only. facet() counts are from a keyword search, so they can be lower than the number of hybrid results.

What text gets embedded

Meilisearch uses a document template to decide which text to send to the embedding model. If an embedder has no documentTemplate, wagtail-meilisearch builds one for each model from its SearchFields, AutocompleteFields and the SearchFields inside RelatedFields, highest boost first. For a page with a title (boost 10), search description (boost 5) and body, the embedded text looks like this:

title: Lightweight ultrabooks
search description: Thin and light notebooks for working away from the office
body: ...

Only the first 400 characters of this text are embedded by default. To use more of each page, set documentTemplateMaxBytes in the embedder config. More text gives better results for long pages but makes indexing much slower. The generated template cuts the text itself, because Meilisearch v1.54 ignored documentTemplateMaxBytes and embedded whole pages.

To use your own template for one model, set meilisearch_document_template on the model. It uses Meilisearch's Liquid template syntax:

class BlogPage(Page):
    meilisearch_document_template = "A blog post titled {{ doc.title }}: {{ doc.body }}"

A template that reads doc.<field> makes Meilisearch reject the settings update if any document in the index does not have that field. The generated templates do not have this problem. Your own template is not cut to documentTemplateMaxBytes; use Liquid's truncate filter if you need a limit.

Running the embedding model

• Use the query cache. A search on Page searches one index for each page type, and Meilisearch embeds the query once for each index. Start Meilisearch with MEILI_EXPERIMENTAL_EMBEDDING_CACHE_ENTRIES=1000 so that it embeds each query only once.

• Allow internal addresses. Newer Meilisearch versions (v1.54 does, v1.18 does not) refuse to call internal IP addresses, such as 127.0.0.1 or a Docker network. The log then shows bad uri: Rejected URI. Allow the embedder's address with MEILI_EXPERIMENTAL_ALLOWED_IP_NETWORKS, for example 127.0.0.1/32.

• Keep the embedder running. If Meilisearch cannot reach the embedder, it keeps retrying the indexing task for several minutes. Meilisearch runs one task at a time, so all indexing on that Meilisearch server waits until the embedder is back or the task fails. Searches still work. If the embedder is too slow when Meilisearch embeds a query, Meilisearch leaves out the semantic results and does not show an error.

• Ollama and threads. By default Ollama can use one thread per CPU core, and each request can then take seconds. Limit it with PARAMETER num_thread 8 in a Modelfile.

• llama.cpp instead of Ollama. llama-server --embeddings works through Meilisearch's rest source. With the same model file it gives the same vectors as Ollama. Qwen3-Embedding needs --pooling last; a wrong pooling setting gives poor vectors without any error.

  'EMBEDDERS': {
      'site': {
          'source': 'rest',
          'url': 'http://127.0.0.1:8080/v1/embeddings',
          'dimensions': 1024,
          'request': {'input': ['{{text}}', '{{..}}']},
          'response': {'data': [{'embedding': '{{embedding}}'}, '{{..}}']},
      },
  },
  

Contributing

If you want to help with the development I'd be more than happy. The vast majority of the heavy lifting is done by MeiliSearch itself, but there is a TODO list...

Run the tests with pytest from the repository root. They need Wagtail, pytest and pytest-django installed, but no Meilisearch server.

TODO

• Write tests
• Performance improvements
• Make use of the async in meilisearch-python
• Faceting
• Implement boosting in the sort algorithm
• Implement stop words
• Search results
• Add support for the autocomplete api
• Ensure we're getting results by relevance

Change Log

1.2.0rc2

• The generated document template cuts its text to documentTemplateMaxBytes (default 400) itself, as Meilisearch v1.54 ignores that setting and embeds whole pages
• Documents MEILI_EXPERIMENTAL_ALLOWED_IP_NETWORKS for newer Meilisearch versions

1.2.0rc1

• Adds hybrid (semantic) search with the EMBEDDERS, SEMANTIC_EMBEDDER, SEMANTIC_RATIO and SEMANTIC_SCORE_THRESHOLD settings, and hybrid() on search results
• Fixes searches also searching the indexes of unrelated models, which could mix up results from different models that have the same id
• Fixes a result found in more than one index getting its lowest ranking score instead of its highest

1.1.0

• adds a get_key method for Wagtail 7 support, thanks @aznszn!

1.0.0

• Big speed improvements thanks to using Meilisearch's native ranking system
• Adds faceting
• Adds filtering
• Adds typing throughout

0.17.3

• Fixes a bug where the meilisearch indexes could end up with a wrong maxTotalHits

0.17.2

• Fixes a bug where the backend could report the wrong counts for results. This turned out to be down to the fact that _do_count can sometimes get called before _do_search, possibly due to Django's paginator. This finally explains why sometimes search queries ran twice.

0.17.1

• Fixes a bug where multi_search can fail when a model index doesn't exist. For models have no documents meilisearch doesn't create the empty index, so we need to check active indexes before calling multi_search otherwise the entire call fails.

0.17.0

• A few small performance and reliability improvements, and a lot of refactoring of the code into multiple files to make future development a bit simpler.

0.16.0

• Thanks to @BertrandBordage, a massive speed improvement through using the /multi-search endpoint introduced in Meilisearch 1.1.0

0.14.0

• Adds Django 4 support and compatibility with the latest meilisearch server (0.30.2) and meilisearch python (0.23.0)

0.14.0

• Updates to work with the latest versions of Meilisearch (v0.28.1) and meilisearch-python (^0.19.1)

0.13.0

• Yanked, sorry

0.12.0

• Adds QUERY_LIMIT option to settings

0.11.0

• Compatibility changes to keep up with MeiliSearch and meilisearch-python
• we've also switched to more closely tracking the major and minor version numbers of meilisearch-python so that it's easier to see compatibility at a glance.
• Note: if you're upgrading from an old version of MeiliSearch you may need to destroy MeiliSearch's data directory and start with a clean index.

0.1.5

• Adds the delta update strategy
• Adds the SKIP_MODELS setting
• Adds support for using boost on your search fields

Thanks

Thank you to the devs of Wagtail-Whoosh. Reading the code over there was the only way I could work out how Wagtail Search backends are supposed to work.