es2csv 5.5.2

A CLI tool for exporting data from Elasticsearch into a CSV file

Command line utility, written in Python, for querying Elasticsearch in Lucene query syntax or Query DSL syntax and exporting result as documents into a CSV file. This tool can query bulk docs in multiple indices and get only selected fields, this reduces query execution time.

Quick Look Demo

Requirements

Installation

>From source:

$ pip install git+https://github.com/taraslayshchuk/es2csv.git

>From pip:

$ pip install es2csv

Usage

$ es2csv [-h] -q QUERY [-u URL] [-a AUTH] [-i INDEX [INDEX ...]]
         [-D DOC_TYPE [DOC_TYPE ...]] [-t TAGS [TAGS ...]] -o FILE
         [-f FIELDS [FIELDS ...]] [-S FIELDS [FIELDS ...]] [-d DELIMITER]
         [-m INTEGER] [-s INTEGER] [-k] [-r] [-e] [--verify-certs]
         [--ca-certs CA_CERTS] [--client-cert CLIENT_CERT]
         [--client-key CLIENT_KEY] [-v] [--debug]

Arguments:
 -q, --query QUERY                        Query string in Lucene syntax.               [required]
 -o, --output-file FILE                   CSV file location.                           [required]
 -u, --url URL                            Elasticsearch host URL. Default is http://localhost:9200.
 -a, --auth                               Elasticsearch basic authentication in the form of username:password.
 -i, --index-prefixes INDEX [INDEX ...]   Index name prefix(es). Default is ['logstash-*'].
 -D, --doc-types DOC_TYPE [DOC_TYPE ...]  Document type(s).
 -t, --tags TAGS [TAGS ...]               Query tags.
 -f, --fields FIELDS [FIELDS ...]         List of selected fields in output. Default is ['_all'].
 -S, --sort FIELDS [FIELDS ...]           List of <field>:<direction> pairs to sort on. Default is [].
 -d, --delimiter DELIMITER                Delimiter to use in CSV file. Default is ",".
 -m, --max INTEGER                        Maximum number of results to return. Default is 0.
 -s, --scroll-size INTEGER                Scroll size for each batch of results. Default is 100.
 -k, --kibana-nested                      Format nested fields in Kibana style.
 -r, --raw-query                          Switch query format in the Query DSL.
 -e, --meta-fields                        Add meta-fields in output.
 --verify-certs                           Verify SSL certificates. Default is False.
 --ca-certs CA_CERTS                      Location of CA bundle.
 --client-cert CLIENT_CERT                Location of Client Auth cert.
 --client-key CLIENT_KEY                  Location of Client Cert Key.
 -v, --version                            Show version and exit.
 --debug                                  Debug mode on.
 -h, --help                               show this help message and exit

[ Usage Examples | Release Changelog ]

5.5.2 (2018-03-21)

• Fixed encoding in field name to UTF-8. (Issue #35)

• Added –sort(-S) argument for sorting data by selected field. (Issue #41)

• Added requirement for version of python 2.7.*. (Issue #8, #12, #20, #29, #33 and #38)

• Update documentation with examples.

• Updating version elasticsearch-py to 5.5.*.

5.2.1 (2017-04-02)

• Added –verify-certs, –ca-certs, –client-cert, –client-key arguments for SSL configuration. (Issue #11 and #24, Pull #22)

• Added –scroll_size(-s) argument to specify the scroll size of requests. (Pull #27)

5.2.0 (2017-02-16)

• Updating version elasticsearch-py to 5.2.* and added support of Elasticsearch 5. (Issue #19)

2.4.3 (2017-02-15)

• Update doc according to wildcard support in fields naming.

• Added support of old version pip. (Issue #16)

2.4.2 (2017-02-14)

• Added wildcard support in fields naming.

• Removed column sorting. (Issue #21)

2.4.1 (2016-11-10)

• Added –auth(-a) argument for Elasticsearch basic authentication. (Pull #17)

• Added –doc_types(-D) argument for specifying document type. (Pull #13)

2.4.0 (2016-10-26)

• Added JSON validation for raw query. (Issue #7)

• Added checks to exclude hangs during connection issues. (Issue #9)

• Updating version elasticsearch-py to 2.4.0 and freeze this dependence according to mask 2.4.*. (Issue #14)

• Updating version progressbar2 to fix issue with visibility.

1.0.3 (2016-06-12)

• Added option to read query string from file –query(-q) @’~/filename.json’. (Issue #5)

• Added –meta_fields(-e) argument for selecting meta-fields: _id, _index, _score, _type. (Issue #6)

• Updating version elasticsearch-py to 2.3.0.

1.0.2 (2016-04-12)

• Added –raw_query(-r) argument for using the native Query DSL format.

1.0.1 (2016-01-22)

• Fixed support elasticsearch-1.4.0.

• Added –version argument.

• Added history changelog.

1.0.0.dev1 (2016-01-04)

• Fixed encoding in CSV to UTF-8. (Issue #3, Pull #1)

• Added better progressbar unit names. (Pull #2)

• Added pip installation instruction.

1.0.0.dev0 (2015-12-25)

• Initial registration.

• Added first dev-release on github.

• Added first release on PyPI.