Custom insert for script in vertica/ch
Project description
Библиотека для вставки в таблицы, используя промежуточную
Типы СУБД для вставки:
- clickhouse
- vertica
Аргументы :
- script - код для вставки, тип: string
- sql_credentials - sql-креды для вставки, тип: json
- insert_table - имя таблицы, в которую идет вставка, тип: string
- add_args - аргументы для замены переменных в скрипте, тип: json
Usage
pip3 install swap-inserter-talenttech-oss
SQL CREDENTIALS:
sql_credentials = {
"vertica": {
"database": <VERTICA_DATABASE>,
"schema": <VERTICA_SCHEMA>,
"staging_schema": <VERTICA_STAGING_SCHEMA>,
"user": <VERTICA_WRITE_USER>,
"host": <VERTICA_HOST>,
"port": <VERTICA_PORT>,
"password": <VERTICA_WRITE_PASSWORD>,
"vertica_configs": {"host": <VERTICA_HOST>,
"port": <VERTICA_PORT>,
"backup_server_node": [<SERVER_NODE_1>, <SERVER_NODE_2>, <SERVER_NODE_3>},
},
"ch": {
"database": <CH_DATABASE>,
"user": <CH_WRITE_USER>,
"host": <CH_HOST_A>,
"port": <CH_PORT_TCP>,
"password": <CH_WRITE_PASSWORD>,
"alt_hosts": "{},{}:{}".format(
<CH_HOST_B>,
<CH_HOST_C>,
<CH_PORT_TCP>,
),
},
}
ПРИМЕР ИСПОЛЬЗОВАНИЯ VERTICA:
from inserter.vertica_swap_inserter import VerticaSwapInserter
script = "SELECT <FIELDS> FROM {schema}.{from_table}"
inserter = VerticaSwapInserter(
script=script,
sql_credentials=sql_credentials["vertica"],
insert_table=<TABLE_INSERT>,
add_args={
"schema": sql_credentials["vertica"]["schema"],
"from_table": <FROM_TABLE>
}
)
inserter.run()
ПРИМЕР ИСПОЛЬЗОВАНИЯ CH:
from inserter.ch_swap_inserter import CHSwapInserter
script = "SELECT <FIELDS> FROM {schema}.{from_table}"
inserter = CHSwapInserter(
script=script,
sql_credentials=sql_credentials["ch"],
insert_table=<TABLE_INSERT>,
add_args={
"database": sql_credential["ch"]["database"],
"from_table": <FROM_TABLE>
},
)
inserter.run()
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for swap-inserter-talenttech-oss-0.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 72dc4fc6e0a610e041407dadd5ca10e090be51211e482d4f61088f96bcfadf04 |
|
MD5 | 81c1790d671c4c3f5399e82af005db11 |
|
BLAKE2b-256 | ae991ff4a60731aeb03f74d11b1278e829ddaa44bddb0024c5a612524a66791f |
Close
Hashes for swap_inserter_talenttech_oss-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6bcb28aa2adc5cab6c91934393a49f633c8d80ab8e3290caf6c50ac11d13014a |
|
MD5 | 9fd3d3f380edc31042ad2e46cd16a574 |
|
BLAKE2b-256 | 1ad650a8136d7444d3e968a8e9775a457a3fb9dc196aabf192ea4ce0cf1b1b81 |