Skip to main content

Python - Elasticsearch Integration (for Aasaanjobs Internal Usage)

Project description

Aasaanjobs Elasticsearch SDK

Library to handle elasticsearch related transactions in Python (for internal usage in Aasaanjobs)

Requirements

  • Python 3.5 and greater
  • Elasticsearch Python

Installation

Install using pip

pip install ajelastic

Configuration

The AJ Elastic library reads configurations from module path specified in the environment variable AJELASTIC_SETTINGS_MODULE if its a standalone python project, or the DJANGO_SETTINGS_MODULE environment variable if its a Django project.

export AJELASTIC_SETTINGS_MODULE=tests.settings

Example

Below is an example settings module that can be consumed by the library

# Mandatory Settings

# Elasticsearch host and port
ES_HOST = "http://localhost:9200"
# The environment suffix that will be added in the Elasticsearch index names
ES_ENV = "development"              

# Optional Settings

# List of Elasticsearch indices
ES_INDICES = {
    "User": {
        "name": "users",                                # The index name prefix
        "doc_type": "user",                             # Elastic document type
        "data_functions": {                             
            "single": "tests.data:get_user",          # Function to fetch a user; should accept a single ID argument
            "multi": "tests.data:list_users"          # Function to fetch list of users; should accept two arguments; limit and offset
        },
        "mapping_path": "tests/user_mapping.json"    # Path to the JSON mapping of the document structure
    }
}

Commands to Sync Data with Elasticsearch

This library provides the following command line scripts to synchronize/initialize data

aj-es-reindex

This command will reindex the documents from scratch fetching data from the function defined in the data_functions.multi settings of the entity (i.e., withing the ES_INDICES settings).

For eg, the below command will reindex the User entity, fetching data from the list_users function (as specified in the settings) with a batch size of 100, i.e., in a single request 100 users will be index to elasticsearch.

aj-es-reindex User 100

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ajelastic, version 1.0.2
Filename, size File type Python version Upload date Hashes
Filename, size ajelastic-1.0.2.tar.gz (7.4 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page