Skip to main content

Graphene Elasticsearch (DSL) integration

Project description

Elasticsearch (DSL) integration for Graphene.

PyPI Version Supported Python versions Build Status GPL-2.0-only OR LGPL-2.1-or-later Coverage

Note

Project status is alpha.

Prerequisites

  • Graphene 2.x. Support for Graphene 1.x is not planned, but might be considered.
  • Python 3.6, 3.7. Support for Python 2 is not intended.
  • Elasticsearch 6.x, 7.x. Support for Elasticsearch 5.x is not intended.

Main features and highlights

  • Implemented ElasticsearchConnectionField and ElasticsearchObjectType are the core classes to work with graphene.
  • Pluggable backends for searching, filtering, ordering, etc. Don’t like existing ones? Override, extend or write your own.
  • Implemented search backend.
  • Implemented filter backend.
  • Implemented ordering backend.
  • Implemented pagination.

See the Road-map for what’s yet planned to implemented.

Documentation

Documentation is available on Read the Docs.

Installation

Install latest stable version from PyPI:

pip install graphene-elastic

Or latest development version from GitHub:

pip install https://github.com/barseghyanartur/graphene-elastic/archive/master.zip

Examples

Install requirements

pip install -r requirements.txt

Populate sample data

The following command will create indexes for User and Post documents and populate them with sample data:

./scripts/populate_elasticsearch_data.sh

Sample document definition

search_index/documents/post.py

See examples/search_index/documents/post.py for full example.

import datetime
from elasticsearch_dsl import (
    Boolean,
    Date,
    Document,
    InnerDoc,
    Keyword,
    Nested,
    Text,
    Integer,
)

class Comment(InnerDoc):

    author = Text(fields={'raw': Keyword()})
    content = Text(analyzer='snowball')
    created_at = Date()

    def age(self):
        return datetime.datetime.now() - self.created_at


class Post(Document):

    title = Text(
        fields={'raw': Keyword()}
    )
    content = Text()
    created_at = Date()
    published = Boolean()
    category = Text(
        fields={'raw': Keyword()}
    )
    comments = Nested(Comment)
    tags = Text(
        analyzer=html_strip,
        fields={'raw': Keyword(multi=True)},
        multi=True
    )
    num_views = Integer()

    class Index:
        name = 'blog_post'
        settings = {
            'number_of_shards': 1,
            'number_of_replicas': 1,
            'blocks': {'read_only_allow_delete': None},
        }

Sample apps

Sample Flask app

Run the sample Flask app:

./scripts/run_flask.sh

Open Flask graphiql client

http://127.0.0.1:8001/graphql

Sample Django app

Run the sample Django app:

./scripts/run_django.sh runserver

Open Flask graphiql client

http://127.0.0.1:8000/graphql

ConnectionField example

ConnectionField is the most flexible and feature rich solution you have. It uses filter backends which you can tie to your needs the way you want in a declarative manner.

Sample schema definition

import graphene
from graphene_elastic import (
    ElasticsearchObjectType,
    ElasticsearchConnectionField,
)
from graphene_elastic.filter_backends import (
    FilteringFilterBackend,
    SearchFilterBackend,
    OrderingFilterBackend,
    DefaultOrderingFilterBackend,
)
from graphene_elastic.constants import (
    LOOKUP_FILTER_PREFIX,
    LOOKUP_FILTER_TERM,
    LOOKUP_FILTER_TERMS,
    LOOKUP_FILTER_WILDCARD,
    LOOKUP_QUERY_EXCLUDE,
    LOOKUP_QUERY_IN,
)

# Object type definition
class Post(ElasticsearchObjectType):

    class Meta(object):
        document = PostDocument
        interfaces = (Node,)
        filter_backends = [
            FilteringFilterBackend,
            SearchFilterBackend,
            OrderingFilterBackend,
            DefaultOrderingFilterBackend,
        ]

        # For `FilteringFilterBackend` backend
        filter_fields = {
            # The dictionary key (in this case `title`) is the name of
            # the corresponding GraphQL query argument. Since lookups are
            # provided, number of lookups is limited, while term is the
            # default lookup (as specified). The dictionary value
            # (in this case `title.raw`) is the field name in the
            # Elasticsearch document (`PostDocument`).
            'title': {
                'field': 'title.raw',
                # Available lookups
                'lookups': [
                    LOOKUP_FILTER_TERM,
                    LOOKUP_FILTER_TERMS,
                    LOOKUP_FILTER_PREFIX,
                    LOOKUP_FILTER_WILDCARD,
                    LOOKUP_QUERY_IN,
                    LOOKUP_QUERY_EXCLUDE,
                ],
                # Default lookup
                'default_lookup': LOOKUP_FILTER_TERM,
            },

            # The dictionary key (in this case `category`) is the name of
            # the corresponding GraphQL query argument. Since no lookups
            # or default_lookup is provided, defaults are used (all lookups
            # available, term is the default lookup). The dictionary value
            # (in this case `category.raw`) is the field name in the
            # Elasticsearch document (`PostDocument`).
            'category': 'category.raw',

            # The dictionary key (in this case `tags`) is the name of
            # the corresponding GraphQL query argument. Since no lookups
            # or default_lookup is provided, defaults are used (all lookups
            # available, term is the default lookup). The dictionary value
            # (in this case `tags.raw`) is the field name in the
            # Elasticsearch document (`PostDocument`).
            'tags': 'tags.raw',

            # The dictionary key (in this case `num_views`) is the name of
            # the corresponding GraphQL query argument. Since no lookups
            # or default_lookup is provided, defaults are used (all lookups
            # available, term is the default lookup). The dictionary value
            # (in this case `num_views`) is the field name in the
            # Elasticsearch document (`PostDocument`).
            'num_views': 'num_views',
        }

        # For `SearchFilterBackend` backend
        search_fields = {
            'title': {'boost': 4},
            'content': {'boost': 2},
            'category': None,
        }

        # For `OrderingFilterBackend` backend
        ordering_fields = {
            # The dictionary key (in this case `tags`) is the name of
            # the corresponding GraphQL query argument. Since no lookups
            # or default_lookup is provided, defaults are used (all lookups
            # available, term is the default lookup). The dictionary value
            # (in this case `tags.raw`) is the field name in the
            # Elasticsearch document (`PostDocument`).
            'title': 'title.raw',

            # The dictionary key (in this case `created_at`) is the name of
            # the corresponding GraphQL query argument. Since no lookups
            # or default_lookup is provided, defaults are used (all lookups
            # available, term is the default lookup). The dictionary value
            # (in this case `created_at`) is the field name in the
            # Elasticsearch document (`PostDocument`).
            'created_at': 'created_at',

            # The dictionary key (in this case `num_views`) is the name of
            # the corresponding GraphQL query argument. Since no lookups
            # or default_lookup is provided, defaults are used (all lookups
            # available, term is the default lookup). The dictionary value
            # (in this case `num_views`) is the field name in the
            # Elasticsearch document (`PostDocument`).
            'num_views': 'num_views',
        }

        # For `DefaultOrderingFilterBackend` backend
        ordering_defaults = (
            '-num_views',  # Field name in the Elasticsearch document
            'title.raw',  # Field name in the Elasticsearch document
        )

# Query definition
class Query(graphene.ObjectType):
    all_post_documents = ElasticsearchConnectionField(Post)

# Schema definition
schema = graphene.Schema(query=Query)
Filter
Sample queries

Since we didn’t specify any lookups on category, by default all lookups are available and the default lookup would be term. Note, that in the {value:"Elastic"} part, the value stands for default lookup, whatever it has been set to.

query PostsQuery {
  allPostDocuments(filter:{category:{value:"Elastic"}}) {
    edges {
      node {
        id
        title
        category
        content
        createdAt
        comments
      }
    }
  }
}

But, we could use another lookup (in example below - terms). Note, that in the {terms:["Elastic", "Python"]} part, the terms is the lookup name.

query PostsQuery {
  allPostDocuments(
        filter:{category:{terms:["Elastic", "Python"]}}
    ) {
    edges {
      node {
        id
        title
        category
        content
        createdAt
        comments
      }
    }
  }
}

Or apply a gt (range) query in addition to filtering:

{
  allPostDocuments(filter:{
        category:{term:"Python"},
        numViews:{gt:"700"}
    }) {
    edges {
      node {
        category
        title
        comments
        numViews
      }
    }
  }
}
Implemented filter lookups

The following lookups are available:

  • contains
  • ends_with (or endsWith for camelCase)
  • exclude
  • exists
  • gt
  • gte
  • in
  • is_null (or isNull for camelCase)
  • lt
  • lte
  • prefix
  • range
  • starts_with (or startsWith for camelCase)
  • term
  • terms
  • wildcard

See dedicated documentation on filter lookups for more information.

Ordering

Possible choices are ASC and DESC.

query {
  allPostDocuments(
        filter:{category:{term:"Photography"}},
        ordering:{title:ASC}
    ) {
    edges {
      node {
        category
        title
        content
        numViews
        tags
      }
    }
  }
}

Road-map

Road-map and development plans.

Lots of features are planned to be released in the upcoming Beta releases:

  • Geo-spatial backend
  • Filter lookup geo_bounding_box (or geoBoundingBox for camelCase)
  • Filter lookup geo_distance (or geoDistance for camelCase)
  • Filter lookup geo_polygon (or geoPolygon for camelCase)
  • Aggregations (faceted search) backend
  • Post-filter backend
  • Nested backend
  • Highlight backend
  • Suggester backend
  • Global aggregations backend
  • More-like-this backend
  • Complex search backends, such as Simple query search
  • Source filter backend

Stay tuned or reach out if you want to help.

Testing

Project is covered with tests.

By defaults tests are executed against the Elasticsearch 7.x.

Running Elasticsearch

Run Elasticsearch 7.x with Docker

docker-compose up elasticsearch

Running tests

Make sure you have the test requirements installed:

pip install -r requirements/test.txt

To test with all supported Python versions type:

tox

To test against specific environment, type:

tox -e py37

To test just your working environment type:

./runtests.py

To run a single test module in your working environment type:

./runtests.py src/graphene_elastic/tests/test_filter_backend.py

To run a single test class in a given test module in your working environment type:

./runtests.py src/graphene_elastic/tests/test_filter_backend.py::FilterBackendElasticTestCase

Debugging

For development purposes, you could use the flask app (easy to debug). Standard pdb works (import pdb; pdb.set_trace()). If ipdb does not work well for you, use ptpdb.

Writing documentation

Keep the following hierarchy.

=====
title
=====

header
======

sub-header
----------

sub-sub-header
~~~~~~~~~~~~~~

sub-sub-sub-header
^^^^^^^^^^^^^^^^^^

sub-sub-sub-sub-header
++++++++++++++++++++++

sub-sub-sub-sub-sub-header
**************************

License

GPL-2.0-only OR LGPL-2.1-or-later

Support

For any issues contact me at the e-mail given in the Author section.

Author

Artur Barseghyan <artur.barseghyan@gmail.com>

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for graphene-elastic, version 0.0.13
Filename, size File type Python version Upload date Hashes
Filename, size graphene_elastic-0.0.13-py2.py3-none-any.whl (105.5 kB) File type Wheel Python version py2.py3 Upload date Hashes View hashes
Filename, size graphene-elastic-0.0.13.tar.gz (75.8 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page