tap-yandex-cloud 0.9.0

Singer tap for YandexCloud, built with the Meltano Singer SDK.

tap-yandex-cloud

tap-yandex-cloud is a Singer tap for YandexCloud.

Built with the Meltano Tap SDK for Singer Taps.

Configuration

Accepted Config Options

A full list of supported settings and capabilities for this tap is available by running:

tap-yandex-cloud --about

Configure using environment variables

This Singer tap will automatically import any environment variables within the working directory's .env if the --config=ENV is provided, such that config values will be considered if a matching environment variable is set either in the terminal context or in the .env file.

Source Authentication and Authorization

tap-yandex-cloud supports two authentication methods:

iam_token — a short-lived IAM token. service_account_key_json — a Yandex Cloud service account authorized key JSON.

For local development and quick checks, iam_token is the simplest option:

export TAP_YANDEX_CLOUD_IAM_TOKEN="$(yc iam create-token)"

IAM tokens are short-lived and are valid for up to 12 hours, so this method is not recommended for unattended production runs.

For production, prefer service_account_key_json. The tap uses the service account authorized key to generate IAM tokens automatically during execution. This avoids manually rotating iam_token before every scheduled run.

Example:

export TAP_YANDEX_CLOUD_SERVICE_ACCOUNT_KEY_JSON='{"id":"...","service_account_id":"...","created_at":"...","key_algorithm":"RSA_2048","public_key":"...","private_key":"..."}'

The service account must have enough permissions to read billing usage data for the configured billing account. In Yandex Cloud, IAM permissions on the service account itself are not enough. You also need to grant the service account access to the billing account in the Yandex Cloud Billing section.

Minimal required tap settings:

export TAP_YANDEX_CLOUD_BILLING_ACCOUNT_ID="your-billing-account-id"
export TAP_YANDEX_CLOUD_START_DATE="2025-01-01T00:00:00Z"
export TAP_YANDEX_CLOUD_AGGREGATION_PERIOD="DAY"
export TAP_YANDEX_CLOUD_API_ENDPOINT="billing.api.cloud.yandex.net:443"

Then provide either:

export TAP_YANDEX_CLOUD_IAM_TOKEN="$(yc iam create-token)"

or:

export TAP_YANDEX_CLOUD_SERVICE_ACCOUNT_KEY_JSON='...'

Usage

You can easily run tap-yandex-cloud by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-yandex-cloud --version
tap-yandex-cloud --help
tap-yandex-cloud --config CONFIG --discover > ./catalog.json

Developer Resources

Follow these instructions to contribute to this project.

Initialize your Development Environment

Prerequisites:

• Python 3.12+
• uv

uv sync

Create and Run Tests

Create tests within the tests subfolder and then run:

uv run pytest

You can also test the tap-yandex-cloud CLI interface directly using uv run:

uv run tap-yandex-cloud --help

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Use Meltano to run an EL pipeline:

# Install meltano
uv tool install meltano

# Test invocation
meltano invoke tap-yandex-cloud --version

# Run a test EL pipeline
meltano run tap-yandex-cloud target-jsonl

SDK Dev Guide

See the dev guide for more instructions on how to use the SDK to develop your own taps and targets.