🔍 elasticsearch-pydantic

Use the Elasticsearch DSL with Pydantic models.

This minimal library is for those who...

• ... want to benefit from Pydantic's extensive validation and type checking ecosystem,
• ... want the convenient and idiomatic persistence layer and query DSL of Elasticsearch DSL,
• ... and do not want to reimplement everything from scratch.

To interconnect the Elasticsearch DSL and Pydantic, we override a limited set of methods from the Elasticsearch Document and InnerDoc base classes with Pydantic's BaseModel functionality. Elasticsearch field types are inferred from the model's type annotations and can be overridden by using Annotated type hints.

Installation

Install the package from PyPI:

pip install elasticsearch-pydantic

Usage

To migrate from Elasticsearch DSL to elasticsearch-pydantic, just change your ORM classes to inherit from elasticsearch_pydantic.BaseDocument instead of elasticsearch_dsl.Document. Then, gradually replace your field definitions with Pydantic type annotations.

For example, in Elasticsearch DSL, you would typically define a document like this:

from elasticsearch_dsl import Document, Text, Date

class BlogPost(Document):
    title = Text()
    content = Text()
    published_at = Date()

With elasticsearch-pydantic, you can define the same document using Pydantic models:

from elasticsearch_pydantic import BaseDocument

class BlogPost(BaseDocument):
    title: str
    content: str
    published_at: datetime

And that's about it! You now get all the type-safety and validation benefits of Pydantic, while still being able to use the powerful features of Elasticsearch DSL.

Most Pydantic types are naturally mapped to Elasticsearch field types. To learn more about the field type mappings, see the mapping code.

Annotated types

You can use Annotated type hints to customize the Elasticsearch field types:

from typing import Annotated
from elasticsearch_dsl import Text, Keyword
from elasticsearch_pydantic import BaseDocument

class BlogPost(BaseDocument):
    title: Annotated[str, Text(analyzer="standard")]
    tags: Annotated[list[str], Keyword]

Field type aliases

For convenience, elasticsearch-pydantic provides type aliases for all standard Elasticsearch field types:

from elasticsearch_pydantic import BaseDocument, TextField, KeywordField

class BlogPost(BaseDocument):
    title: TextField
    tags: list[KeywordField]

Accessing meta fields

For convenience, the Elasticsearch meta fields (like _id, _index, _score, etc.) are directly accessible as attributes on your document models:

post = BlogPost(id="1", title="My first post")
print(post.id)        # Access the document ID
print(post.meta.index)  # Access the index name

The meta attribute is kept for compatibility with Elasticsearch DSL and contains all meta fields.

Compatibility

This library works fine with any of the following Pip packages installed:

• elasticsearch-dsl<8.12.0
• elasticsearch6-dsl
• elasticsearch7-dsl
• elasticsearch8-dsl<8.12.0

The elasticsearch-pydantic library will automatically detect which Elasticsearch DSL is installed.

Development

To build this package and contribute to its development you need to install the build, setuptools and wheel packages:

pip install build setuptools wheel

(On most systems, these packages are already pre-installed.)

Development installation

Install package and test dependencies:

pip install -e .[tests,tests-es6]       # For elasticsearch-dsl~=6.0
pip install -e .[tests,tests-es6-major] # For elasticsearch6-dsl
pip install -e .[tests,tests-es7]       # For elasticsearch-dsl~=7.0
pip install -e .[tests,tests-es7-major] # For elasticsearch7-dsl
pip install -e .[tests,tests-es8]       # For elasticsearch-dsl~=8.0
pip install -e .[tests,tests-es8-major] # For elasticsearch8-dsl

Testing

Verify your changes against the test suite to verify.

ruff check .  # Code format and LINT
mypy .        # Static typing
pytest .      # Unit tests

Please also add tests for your newly developed code.

Build wheels

Wheels for this package can be built with:

python -m build

Support

If you have any problems using this package, please file an issue. We're happy to help!

License

This repository is released under the MIT license.