Elasctic load CSV utility
Project description
Elastic CSV Loader
This command line utility loads csv file into a elasticsearch index, using a provided yaml config file.
Considerations:
- CSV files MUST include a header with field names
- Header field names will be used as elastic index fields
- A
@timestamp
anddate
fields will be added to all indexed docs- A
date
logic date could be forced through command parameter.
- A
- Depending on
elastic_index.data_format.parent_data_object
value, all original csv header fields will be arranged under adate
parent object.
Indexed data will use the same field names that
Install
pip install --upgrade elasticcsv
Run
Elastic Connection Config
Connection configuration is based in a YAML text file (connection.yaml
) that must be present in
command directory.
Sample connection.yaml
elastic_connection:
proxies:
http: "http://user:pass@proxy.url:8080"
https: "http://user:pass@proxy.url:8080"
user: myuser
password: mypassword
node: my.elastic.node
port: 9200
elastic_index:
data_format:
parent_data_object: true
Run command
> csv2es load-csv --help
Usage: csv2es load-csv [OPTIONS]
Loads csv to elastic index
Options:
--csv PATH CSV File [required]
--sep TEXT CSV field sepator [required]
--index TEXT Elastic Index [required]
--logic_date [%Y-%m-%d] Date reference for interfaces
--help Show this message and exit.
Example:
csv2es load-csv --csv ./pathtomyfile/file.csv --index myindex --sep ";"
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
elasticcsv-0.1.5.tar.gz
(6.4 kB
view hashes)
Built Distribution
Close
Hashes for elasticcsv-0.1.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 300604cb0ec4d3f544941bdc3b429cba06681494d661ff9c2e5d0ccd3f9715f0 |
|
MD5 | 68dbb9a47e492d0f46fb2910952d0eb1 |
|
BLAKE2b-256 | 7578367c5608922b17db0788f8dfcd5e0d993eb3ca329c0c8f2f43ebf60aa201 |