Skip to main content
Donate to the Python Software Foundation or Purchase a PyCharm License to Benefit the PSF! Donate Now

Run queries against Kibana's Elasticsearch that gets logs from Logstash.

Project description

elasticsearch-query

PyPI Build Status

Run queries against Kibana's Elasticsearch that gets logs from Logstash. Forked from Wikia's kibana.py.

pip install elasticsearch-query

Basic Usage

from elasticsearch_query import ElasticsearchQuery
es_query = ElasticsearchQuery(es_host='es.prod', since=12345, period=900, index_prefix='logstash-my-app')

es_host needs to be specified with a host of Elasticsearch instance to connect.

Provide either since (absolute timestamp) or period (last N seconds):

  • since: UNIX timestamp data should be fetched since (if None, then period specifies the last n seconds).
  • period: period (in seconds) before now() to be used when since is empty (defaults to last 15 minutes).

index_prefix argument will be used to build indices names to query in. They should follow the index-name-YYYY.MM.DD naming convention, e.g. logstash-my-app-2014.08.19.

get_rows

Returns data matching the given query (provided as a dict).

es_query.get_rows(match={"tags": 'edge-cache-requestmessage'}, limit=2000)
  • match: query to be run against log messages (ex. {"@message": "Foo Bar DB queries"}).
  • limit: the number of results (defaults to 10).

query_by_string

Returns data matching the given query string (provided as a Lucene query).

es_query.query_by_string(query='@message:"^PHP Fatal"', limit=2000)
es_query.query_by_string(query='@message:"^PHP Fatal"', fields=['@message', '@es_query_host'], limit=2000)
  • query: query string to be run against log messages (ex. @message:"^PHP Fatal").
  • fields: optional list of fields to fetch
  • limit: the number of results (defaults to 10).

query_by_sql

Returns data matching the given SQL query.

This feature requires non-OSS version of Elasticsearch ("To be clear, while the X-Pack source code is now available in the public repositories, it isn’t under an Open Source license").

es_query.query_by_sql(sql='SELECT host FROM "app-requests" WHERE host = \'app2.prod\'')

count

Returns number of matching entries

es_query.count(query='@message:"^PHP Fatal"')

Integration tests

elasticsearch-query comes with integration tests suite. .travis.yml will install elasticsearch OSS version and run them.

But you can also run it locally. Simply pass ES_TEST_HOST=<elasticsearch IP> env variable when running tests (make test).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
elasticsearch-query-2.4.0.tar.gz (6.4 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page