EsXport

A Python-based CLI utility and module designed for querying Elasticsearch and exporting results as a CSV file.

Requirements

1. This tool should be used with Elasticsearch 8.x version.
2. You also need >= Python 3.10.x.

Installation

From source:

pip install esxport

For development purpose

pip install "esxport[dev]"

Usage

CLI Usage

Run esxport --help for detailed information on available options:

OPTIONS

Usage: esxport [OPTIONS]

Options:
  -q, --query JSON           Query string in Query DSL syntax. [required]
  -o, --output-file PATH     CSV file location. [required]
  -i, --index-prefixes TEXT  Index name prefix(es). [required]
  -u, --url URL              Elasticsearch host URL. [default: https://localhost:9200]
  -U, --user TEXT            Elasticsearch basic authentication user. [default: elastic]
  -p, --password TEXT        Elasticsearch basic authentication password. [required]
  -f, --fields TEXT          List of _source fields to present in the output. [default: _all]
  -S, --sort ELASTIC SORT    List of fields to sort in the format `<field>:<direction>`.
  -d, --delimiter TEXT       Delimiter to use in the CSV file. [default: ,]
  -m, --max-results INTEGER  Maximum number of results to return. [default: 10]
  -s, --scroll-size INTEGER  Scroll size for each batch of results. [default: 100]
  -e, --meta-fields [_id|_index|_score]
                             Add meta-fields to the output.
  --verify-certs             Verify SSL certificates.
  --ca-certs PATH            Location of CA bundle.
  --client-cert PATH         Location of Client Auth cert.
  --client-key PATH          Location of Client Cert Key.
  -v, --version              Show version and exit.
  --debug                    Enable debug mode.
  --help                     Show this message and exit.

Module Usage

In addition to the CLI, EsXport can now be used as a Python module. Below is an example of how to integrate it into your Python application:

from esxport import CliOptions, EsXport

kwargs = {
    "query": {
        "query": {"match_all": {}},
        "size": 1000
    },
    "output_file": "output.csv",
    "index_prefixes": ["my-index-prefix"],
    "url": "https://localhost:9200",
    "user": "elastic",
    "password": "password",
    "verify_certs": False,
    "debug": True,
    "max_results": 1000,
    "scroll_size": 100,
    "sort": ["field_name:asc"],
    "ca_certs": "path/to/ca.crt"
}

# Create CLI options and initialize EsXport
cli_options = CliOptions(kwargs)
es = EsXport(cli_options)

# Export data
es.export()

Class Descriptions

CliOptions

A configuration class to manage CLI arguments programmatically when using the module.

Attributes

Example Initialization

from esxport import CliOptions

cli_options = CliOptions({
    "query": {"query": {"match_all": {}}},
    "output_file": "data.csv",
    "url": "https://localhost:9200",
    "user": "elastic",
    "password": "password",
    "index_prefixes": ["my-index-prefix"],
    "fields": ["field1", "field2"],
    "sort": ["field1:asc"],
    "max_results": 1000,
    "scroll_size": 100
})

EsXport

The main class for executing the export operation.

Methods

Example Initialization and Usage

from esxport import CliOptions, EsXport

# Define CLI options
cli_options = CliOptions({
    "query": {"query": {"match_all": {}}},
    "output_file": "output.csv",
    "url": "https://localhost:9200",
    "user": "elastic",
    "password": "password",
    "index_prefixes": ["my-index-prefix"]
})

# Initialize EsXport
esxport = EsXport(cli_options)

# Export data
esxport.export()

Development

This project uses Hatch for development environment management and packaging.

Quick Start

# Install hatch
pip install hatch

# Run tests
hatch run test

# Format code
hatch run lint:fmt

# Type checking
hatch run lint:typing

# Serve documentation locally
hatch run docs:serve

Available Environments

• default - Development and testing environment
• lint - Code formatting, linting, and type checking
• docs - Documentation building and serving
• release - Version management and publishing
• all - Matrix testing across Python versions (3.10-3.14)

For comprehensive documentation on the development workflow, see: docs/HATCH_DEVELOPMENT.md