Skip to main content

Run queries against Kibana's Elasticsearch that gets logs from Logstash.

Project description

elasticsearch-query

PyPI Build Status

Run queries against Kibana's Elasticsearch that gets logs from Logstash. Forked from Wikia's kibana.py.

pip install elasticsearch-query

Basic Usage

from elasticsearch_query import ElasticsearchQuery
es_query = ElasticsearchQuery(es_host='es.prod', since=12345, period=900, index_prefix='logstash-my-app')

es_host needs to be specified with a host of Elasticsearch instance to connect.

Provide either since (absolute timestamp) or period (last N seconds):

  • since: UNIX timestamp data should be fetched since (if None, then period specifies the last n seconds).
  • period: period (in seconds) before now() to be used when since is empty (defaults to last 15 minutes).

index_prefix argument will be used to build indices names to query in. They should follow the index-name-YYYY.MM.DD naming convention, e.g. logstash-my-app-2014.08.19.

get_rows

Returns data matching the given query (provided as a dict).

es_query.get_rows(match={"tags": 'edge-cache-requestmessage'}, limit=2000)
  • match: query to be run against log messages (ex. {"@message": "Foo Bar DB queries"}).
  • limit: the number of results (defaults to 10).

query_by_string

Returns data matching the given query string (provided as a Lucene query).

es_query.query_by_string(query='@message:"^PHP Fatal"', limit=2000)
es_query.query_by_string(query='@message:"^PHP Fatal"', fields=['@message', '@es_query_host'], limit=2000)
  • query: query string to be run against log messages (ex. @message:"^PHP Fatal").
  • fields: optional list of fields to fetch
  • limit: the number of results (defaults to 10).

query_by_sql

Returns data matching the given SQL query.

This feature requires non-OSS version of Elasticsearch ("To be clear, while the X-Pack source code is now available in the public repositories, it isn’t under an Open Source license").

es_query.query_by_sql(sql='SELECT host FROM "app-requests" WHERE host = \'app2.prod\'')

count

Returns number of matching entries

es_query.count(query='@message:"^PHP Fatal"')

Integration tests

elasticsearch-query comes with integration tests suite. .travis.yml will install elasticsearch OSS version and run them.

But you can also run it locally. Simply pass ES_TEST_HOST=<elasticsearch IP> env variable when running tests (make test).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

elasticsearch-query-2.4.0.tar.gz (6.4 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page