`Tap-DB2` is a Singer tap for IBM DB2 data sources.
Project description
Tap-DB2 👑
Tap-DB2
is a Singer tap for IBM DB2 data sources. Built with the Meltano Tap SDK for Singer Taps.
Installation ⚙️
Install from PyPi:
pipx install tap-ibm-db2
Install from GitHub:
pipx install git+https://github.com/danielptv/tap-db2.git@main
Configuration 📝
Setting | Required | Default | Description |
---|---|---|---|
host | True | localhost | The DB2 hostname. |
port | True | 50000 | The DB2 port. |
database | True | None | The DB2 database. |
schema | False | None | The DB2 schema. |
user | True | None | The DB2 username. |
password | True | None | The DB2 password. |
encryption | True | None | Encryption settings for the DB2 connection. Disabled if omitted. |
connection_parameters | False | None | Additional parameters to be appended to the connection string. This is an objects containing key-value pairs. |
sqlalchemy_execution_options | False | None | Additional execution options to be passed to SQLAlchemy. This is an objects containing key-value pairs. |
query_partitioning | False | None | Partition query into smaller subsets. |
filter | False | None | Apply a custom WHERE condition per stream. Unlike the filter available in stream_maps, this will be evaluated BEFORE extracting the data. |
ignore_supplied_tables | False | True | Ignore DB2-supplied user tables. For more info check out Db2-supplied user tables. |
ignore_views | False | False | Ignore views. |
stream_maps | False | None | Config object for stream maps capability. For more information check out Stream Maps. |
stream_map_config | False | None | User-defined config values to be used within map expressions. |
A full list of supported settings and capabilities for this tap is available by running:
tap-db2 --about --format json
Configure using environment variables ✏️
This Singer tap will automatically import any environment variables within the working directory's
.env
if the --config=ENV
is provided, such that config values will be considered if a matching
environment variable is set either in the terminal context or in the .env
file.
Configure encryption settings 🔒
This Singer tap supports encrypted connection settings to DB2 according to the python-ibmdb driver.
SSL without additional options:
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
encryption: {}
This will append SECURITY=SSL;
to the connection string.
SSL using SSLServerCertificate keyword:
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
encryption:
ssl_server_certificate: <Full path to the server certificate>
This will append SECURITY=SSL;SSLServerCertificate=<Full path to the server certificate>;
to the connection string.
SSL using SSLClientKeyStoreDB and SSLClientKeyStoreDBPassword keywords:
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
encryption:
ssl_client_key_store_db:
database: <Full path to the client keystore database>
password: <Keystore password>
This will append SECURITY=SSL;SSLClientKeyStoreDB=<Full path to the client keystore database>;SSLClientKeyStoreDBPassword=<Keystore password>;
to the connection string.
SSL using SSLClientKeyStoreDB and SSLClientKeyStash keywords:
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
encryption:
ssl_client_key_store_db:
database: <Full path to the client keystore database>
key_stash: <Full path to the client keystore stash>
This will append SECURITY=SSL;SSLClientKeyStoreDB=<Full path to the client keystore database>;SSLClientKeyStash=<Full path to the client keystore stash>;
to the connection string.
Configure query partitioning 🧩
This Singer tap supports the partitioning of SQL queries into smaller sub-queries to reduce the CPU load on the database. This is particularly useful when working with large amounts of data and a DB2 that has set strict resource limits per query. Note: This only works for streams with a numeric primary key.
The configuration for query partitioning should look as follows:
...
plugins:
extractors:
- name: tap-db2
variant: danielptv
pip_url: tap-ibm-db2
config:
...
query_partitioning:
<stream>:
primary_key: <primary key>
partition_size: 1000
Replace <stream>
with the stream name and <primary key>
with the stream's primary key. Use *
to apply a query partitioning setting to all streams not explicitly declared.
Usage 👷♀️
You can easily run tap-db2
by itself or in a pipeline using Meltano.
Executing the Tap Directly 🔨
tap-db2 --version
tap-db2 --help
tap-db2 --config CONFIG --discover > ./catalog.json
Developer Resources 👩🏼💻
Follow these instructions to contribute to this project.
Initialize your Development Environment
pipx install poetry
poetry install
Create and Run Tests 🧪
Create tests within the tests
subfolder and
then run:
poetry run pytest
You can also test the tap-db2
CLI interface directly using poetry run
:
poetry run tap-db2 --help
Testing with Meltano
Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.
Next, install Meltano (if you haven't already) and any needed plugins:
# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-db2
meltano install
Now you can test and orchestrate using Meltano:
# Test invocation:
meltano invoke tap-db2 --version
# OR run a test `elt` pipeline:
meltano elt tap-db2 target-jsonl
SDK Dev Guide
See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tap_ibm_db2-0.10.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f230e3b85bc8c4c4dc0aba267521812718c4a67bcb501b080f8f5f38efef869f |
|
MD5 | 76a82415a34d598901db19528c51d3a1 |
|
BLAKE2b-256 | e0f301e165a58bb75fa00c8db94de3bf3cdd417f1609370d1dad33ad55b6ecaa |