Custom insert for script in vertica/ch
Project description
Библиотека для вставки в таблицы, используя промежуточную
Типы СУБД для вставки:
- clickhouse
- vertica
Аргументы :
- script - код для вставки, тип: string
- sql_credentials - sql-креды для вставки, тип: json
- insert_table - имя таблицы, в которую идет вставка, тип: string
- add_args - аргументы для замены переменных в скрипте, тип: json
Usage
pip3 install swap-inserter-talenttech-oss
SQL CREDENTIALS:
sql_credentials = {
"vertica": {
"database": <VERTICA_DATABASE>,
"schema": <VERTICA_SCHEMA>,
"staging_schema": <VERTICA_STAGING_SCHEMA>,
"user": <VERTICA_WRITE_USER>,
"host": <VERTICA_HOST>,
"port": <VERTICA_PORT>,
"password": <VERTICA_WRITE_PASSWORD>,
"vertica_configs": {"host": <VERTICA_HOST>,
"port": <VERTICA_PORT>,
"backup_server_node": [<SERVER_NODE_1>, <SERVER_NODE_2>, <SERVER_NODE_3>},
},
"ch": {
"database": <CH_DATABASE>,
"user": <CH_WRITE_USER>,
"host": <CH_HOST_A>,
"port": <CH_PORT_TCP>,
"password": <CH_WRITE_PASSWORD>,
"alt_hosts": "{},{}:{}".format(
<CH_HOST_B>,
<CH_HOST_C>,
<CH_PORT_TCP>,
),
},
}
ПРИМЕР ИСПОЛЬЗОВАНИЯ VERTICA:
from inserter.vertica_swap_inserter import VerticaSwapInserter
script = "SELECT <FIELDS> FROM {schema}.{from_table}"
inserter = VerticaSwapInserter(
script=script,
sql_credentials=sql_credentials["vertica"],
insert_table=<TABLE_INSERT>,
add_args={
"schema": sql_credentials["vertica"]["schema"],
"from_table": <FROM_TABLE>
}
)
inserter.run()
ПРИМЕР ИСПОЛЬЗОВАНИЯ CH:
from inserter.ch_swap_inserter import CHSwapInserter
script = "SELECT <FIELDS> FROM {schema}.{from_table}"
inserter = CHSwapInserter(
script=script,
sql_credentials=sql_credentials["ch"],
insert_table=<TABLE_INSERT>,
add_args={
"database": sql_credential["ch"]["database"],
"from_table": <FROM_TABLE>
},
)
inserter.run()
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for swap-inserter-talenttech-oss-0.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 13f979bf315963cc636b6c1723f685ba7c27d3eca3067f3fb0f5c4c5ace41721 |
|
MD5 | 9340269758e337d782392013c9526bdc |
|
BLAKE2b-256 | d7580231792167fd6a11f8592a24d347e64c6181c50075db2eb5750b30323b72 |
Close
Hashes for swap_inserter_talenttech_oss-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d732d5b7b1f42dc47a5316e34dae6c8d03392e2239a2281d4eb997effd99fce5 |
|
MD5 | 7b1de27ff9bc60ee6e70c8c5246a86a8 |
|
BLAKE2b-256 | e5969db8475472731518c9d8e5f9ad159703e89d1bda81cc9213bced267573d9 |