Skip to main content

Python Elastic Search driver

Project description

Keywords:search, elastisearch, distribute search

pyes is a connector to use elasticsearch from python.

This version requires elasticsearch 0.12 or above.


  • Thrift/HTTP protocols
  • Bulk insert
  • Index management
  • Every search query types
  • Facet Support
  • Geolocalization support
  • Highlighting
  • Initial River support


These function are taken from pycassa.

Import the module:

>>> import pyes

pyes is able to use standard http or thrift protocol. If your port starts with “92” http protocol is used, otherwise thrift.

For a single connection (which is _not_ thread-safe), pass a list of servers.

For thrift:

>>> client = pyes.connect() # Defaults to connecting to the server at ''
>>> client = pyes.connect([''])

For http:

>>> client = pyes.connect([''])

With thrift protocol, framed transport is the default. You may disable it by passing framed_transport=False.

>>> client = pyes.connect(framed_transport=False)

Thread-local connections opens a connection for every thread that calls an ElasticSearch function. It also automatically balances the number of connections between servers, unless round_robin=False.

>>> client = pyes.connect_thread_local() # Defaults to connecting to the server at ''
>>> client = pyes.connect_thread_local(['', 'other_server:9500']) # Round robin connections
>>> client = pyes.connect_thread_local(['', 'other_server:9500'], round_robin=False) # Connect in list order

With http protocol transport:

>>> client = pyes.connect_thread_local(['', 'other_server:9200']) # Round robin connections
>>> client = pyes.connect_thread_local(['', 'other_server:9200'], round_robin=False) # Connect in list order

Connections are robust to server failures. Upon a disconnection, it will attempt to connect to each server in the list in turn. If no server is available, it will raise a NoServerAvailable exception.

Timeouts are also supported and should be used in production to prevent a thread from freezing while waiting for the server to return.

>>> client = pyes.connect(timeout=3.5) # 3.5 second timeout
(Make some pyes calls and the connection to the server suddenly becomes unresponsive.)

Traceback (most recent call last): … pyes.connection.NoServerAvailable

Note that this only handles socket timeouts.


Creating a connection:

>>> from pyes import *
>>> conn = ES('')

Deleting an index:

>>> try:
>>>     conn.delete_index("test-index")
>>> except:
>>>     pass

(an exception is fored if the index is not present)

Create an index:

>>> conn.create_index("test-index")

Creating a mapping:

>>> mapping = { u'parsedtext': {'boost': 1.0,
>>>                  'index': 'analyzed',
>>>                  'store': 'yes',
>>>                  'type': u'string',
>>>                  "term_vector" : "with_positions_offsets"},
>>>          u'name': {'boost': 1.0,
>>>                     'index': 'analyzed',
>>>                     'store': 'yes',
>>>                     'type': u'string',
>>>                     "term_vector" : "with_positions_offsets"},
>>>          u'title': {'boost': 1.0,
>>>                     'index': 'analyzed',
>>>                     'store': 'yes',
>>>                     'type': u'string',
>>>                     "term_vector" : "with_positions_offsets"},
>>>          u'pos': {'store': 'yes',
>>>                     'type': u'integer'},
>>>          u'uuid': {'boost': 1.0,
>>>                    'index': 'not_analyzed',
>>>                    'store': 'yes',
>>>                    'type': u'string'}}
>>> conn.put_mapping("test-type", {'properties':mapping}, ["test-index"])

Index some documents:

>>> conn.index({"name":"Joe Tester", "parsedtext":"Joe Testere nice guy", "uuid":"11111", "position":1}, "test-index", "test-type", 1)
>>> conn.index({"name":"Bill Baloney", "parsedtext":"Joe Testere nice guy", "uuid":"22222", "position":2}, "test-index", "test-type", 2)

Refresh an index:

>>> conn.refresh(["test-index"])

Execute a query

>>> q = TermQuery("name", "joe")
>>> result = = q)

For more examples looks at the tests.


  1. 0.14.0: Added delete of mapping type.

    Embedded urllib3 to be buildout safe and for users sake.

  1. 0.13.1: Added jython support (HTTP only for now).
  1. 0.13.0: API Changes: errors -> exceptions.

    Splitting of query/filters.

    Added open/close of index.

    Added the number of retries if server is down.

    Refactory Range query. (Andrei)

    Improved HTTP connection timeout/retries. (Sandymahalo)

    Cleanup some imports. (Sandymahalo)

  1. 0.12.1: Added collecting server info.

    Version 0.12 or above requirement.

    Fixed attachment plugin.

    Updated bulk insert to use new api.

    Added facet support (except geotypes).

    Added river support.

    Cleanup some method.

    Added default_indexes variable.

    Added datetime deserialization.

    Improved performance and memory usage in bulk insert replacing list with StringIO.

    Initial propagation of elasticsearch exception to python.

  1. 0.12.0: added http transport, added autodetect of transport, updated thrift interface.
  1. 0.10.3: added bulk insert, explain and facet.
  1. 0.10.2: added new geo query type.
  1. 0.10.1: added new connection pool system based on pycassa one.
  1. 0.10.0: initial working version.


  • API rewriting for mapping
  • more docs
  • more tests
  • cleanup
  • add coverage
  • add jython native client protocol


This software is licensed under the New BSD License. See the LICENSE file in the top distribution directory for the full license text.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pyes, version 0.14.0
Filename, size File type Python version Upload date Hashes
Filename, size pyes-0.14.0.tar.gz (177.1 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page