Tap-DB2 👑

Tap-DB2 is a Singer tap for IBM DB2 data sources. Built with the Meltano Tap SDK for Singer Taps.

Installation ⚙️

Install from PyPi:

pipx install tap-ibm-db2

Install from GitHub:

pipx install git+https://github.com/danielptv/tap-db2.git@main

Configuration 📝

A full list of supported settings and capabilities for this tap is available by running:

tap-db2 --about --format json

Configure using environment variables ✏️

This Singer tap will automatically import any environment variables within the working directory's .env if the --config=ENV is provided, such that config values will be considered if a matching environment variable is set either in the terminal context or in the .env file.

Configure encryption settings 🔒

This Singer tap supports encrypted connection settings to DB2 according to the python-ibmdb driver.

SSL without additional options:

...
plugins:
  extractors:
  - name: tap-db2
    variant: danielptv
    pip_url: tap-ibm-db2
    config:
      ...
      encryption: {}

This will append SECURITY=SSL; to the connection string.

SSL using SSLServerCertificate keyword:

...
plugins:
  extractors:
  - name: tap-db2
    variant: danielptv
    pip_url: tap-ibm-db2
    config:
      ...
      encryption:
        ssl_server_certificate: <Full path to the server certificate>

This will append SECURITY=SSL;SSLServerCertificate=<Full path to the server certificate>; to the connection string.

SSL using SSLClientKeyStoreDB and SSLClientKeyStoreDBPassword keywords:

...
plugins:
  extractors:
  - name: tap-db2
    variant: danielptv
    pip_url: tap-ibm-db2
    config:
      ...
      encryption:
        ssl_client_key_store_db:
          database: <Full path to the client keystore database>
          password: <Keystore password>

This will append SECURITY=SSL;SSLClientKeyStoreDB=<Full path to the client keystore database>;SSLClientKeyStoreDBPassword=<Keystore password>; to the connection string.

SSL using SSLClientKeyStoreDB and SSLClientKeyStash keywords:

...
plugins:
  extractors:
  - name: tap-db2
    variant: danielptv
    pip_url: tap-ibm-db2
    config:
      ...
      encryption:
        ssl_client_key_store_db:
          database: <Full path to the client keystore database>
          key_stash: <Full path to the client keystore stash>

This will append SECURITY=SSL;SSLClientKeyStoreDB=<Full path to the client keystore database>;SSLClientKeyStash=<Full path to the client keystore stash>; to the connection string.

Configure query partitioning 🧩

This Singer tap supports the partitioning of SQL queries into smaller sub-queries to reduce the CPU load on the database. This is particularly useful when working with large amounts of data and a DB2 that has set strict resource limits per query. Note: This only works for streams with a numeric primary key.

The configuration for query partitioning should look as follows:

...
plugins:
  extractors:
  - name: tap-db2
    variant: danielptv
    pip_url: tap-ibm-db2
    config:
      ...
      query_partition:
      <stream>:
        partition_key: <partition_key>
        partition_size: 1000

Replace <stream> with the stream name and <partition_key> with the stream's partition key. Use * to apply a query partitioning setting to all streams not explicitly declared.

Usage 👷‍♀️

You can easily run tap-db2 by itself or in a pipeline using Meltano.

Executing the Tap Directly 🔨

tap-db2 --version
tap-db2 --help
tap-db2 --config CONFIG --discover > ./catalog.json

Developer Resources 👩🏼‍💻

Follow these instructions to contribute to this project.

Initialize your Development Environment

pipx install poetry
poetry install

Create and Run Tests 🧪

Create tests within the tests subfolder and then run:

poetry run pytest

You can also test the tap-db2 CLI interface directly using poetry run:

poetry run tap-db2 --help

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Next, install Meltano (if you haven't already) and any needed plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-db2
meltano install

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-db2 --version
# OR run a test `elt` pipeline:
meltano elt tap-db2 target-jsonl

SDK Dev Guide

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.