Custom insert for script in vertica/ch
Project description
Библиотека для вставки в таблицы, используя промежуточную
Типы СУБД для вставки:
- clickhouse
- vertica
Аргументы :
- script - код для вставки, тип: string
- sql_credentials - sql-креды для вставки, тип: json
- insert_table - имя таблицы, в которую идет вставка, тип: string
- add_args - аргументы для замены переменных в скрипте, тип: json
Usage
pip3 install swap-inserter-talenttech-oss
SQL CREDENTIALS:
sql_credentials = {
"vertica": {
"database": <VERTICA_DATABASE>,
"schema": <VERTICA_SCHEMA>,
"staging_schema": <VERTICA_STAGING_SCHEMA>,
"user": <VERTICA_WRITE_USER>,
"host": <VERTICA_HOST>,
"port": <VERTICA_PORT>,
"password": <VERTICA_WRITE_PASSWORD>,
"vertica_configs": {"host": <VERTICA_HOST>,
"port": <VERTICA_PORT>,
"backup_server_node": [<SERVER_NODE_1>, <SERVER_NODE_2>, <SERVER_NODE_3>},
},
"ch": {
"database": <CH_DATABASE>,
"user": <CH_WRITE_USER>,
"host": <CH_HOST_A>,
"port": <CH_PORT_TCP>,
"password": <CH_WRITE_PASSWORD>,
"alt_hosts": "{},{}:{}".format(
<CH_HOST_B>,
<CH_HOST_C>,
<CH_PORT_TCP>,
),
},
}
ПРИМЕР ИСПОЛЬЗОВАНИЯ VERTICA:
from inserter.vertica_swap_inserter import VerticaSwapInserter
script = "SELECT <FIELDS> FROM {schema}.{from_table}"
inserter = VerticaSwapInserter(
script=script,
sql_credentials=sql_credentials["vertica"],
insert_table=<TABLE_INSERT>,
add_args={
"schema": sql_credentials["vertica"]["schema"],
"from_table": <FROM_TABLE>
}
)
inserter.run()
ПРИМЕР ИСПОЛЬЗОВАНИЯ CH:
from inserter.ch_swap_inserter import CHSwapInserter
script = "SELECT <FIELDS> FROM {schema}.{from_table}"
inserter = CHSwapInserter(
script=script,
sql_credentials=sql_credentials["ch"],
insert_table=<TABLE_INSERT>,
add_args={
"database": sql_credential["ch"]["database"],
"from_table": <FROM_TABLE>
},
)
inserter.run()
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for swap-inserter-talenttech-oss-0.0.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2ec73deaddc1f8c8ed9e922f9e74cf1aea2668abfef04c1617b4159698c0712b |
|
MD5 | 2611a06e9de92cd52454b34681ddf859 |
|
BLAKE2b-256 | faed04c379b4d2a97d27b2cae351ca70253bba6a06aef19fc55aafe291e61cbb |
Close
Hashes for swap_inserter_talenttech_oss-0.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c6febc468cdf8b1e2dfed2d375457c405c4813c0aa4fae04ac1d67cd936fea64 |
|
MD5 | abcb52c5abc1c6b8ea984882026366f4 |
|
BLAKE2b-256 | ffcd308a69c52d789cdda5545ef7bbb0fd83c2b150cbaab3cde28bde185c4bf8 |