Skip to main content

meltano-map-transformer

pre-commit.ci status

A map transformer to duplicate, filter, hash, mask and do many more small transformations in your extract & load pipelines. Implements the Stream Maps capability.

This mapper plugin is fully compliant with the Singer Spec and can be placed in between any Singer tap and target.

Requirements

  • Python 3.10+

meltano-map-transformer in action

For a collection of examples, take a look at examples/README.md.

Settings

Setting Required Default Description
stream_maps True None Stream maps
flattening_enabled False None 'True' to enable schema flattening and automatically expand nested properties.
flattening_max_depth False None The max depth to flatten schemas.
stream_map_config False None User-defined config values to be used within map expressions.

A full list of supported settings and capabilities is available by running: meltano-map-transformer --about

Installation

We recommend using release tags when installing from GitHub. We also recommend using uv or or meltano instead of installing with pip directly.

You can see a full list of published releases here.

For example:

# Use `uv` so that virtual environments are managed automatically:
uv tool install meltano-map-transform

# Alternatively, use `pipx`:
pipx install meltano-map-transform

Meltano installation instructions

  1. Add this plugin to your Meltano project

    meltano add mapper meltano-map-transformer
    
  2. Install the plugin

    meltano install
    
  3. Configure the plugin

    # meltano.yml
    plugins:
      mappers:
      - name: meltano-map-transformer
        pip_url: meltano-map-transform
        executable: meltano-map-transform
        mappings:
        - name: hash_email
          config:
            stream_maps:
              customers:
                email: null # drop the PII field from RECORD and SCHEMA messages
                email_domain: email.split('@')[-1]
                email_hash: md5(config['hash_seed'] + email)
                __else__: null # drop all other fields
            stream_map_config:
              hash_seed: 01AWZh7A6DzGm6iJZZ2T
    
  4. Use the plugin in your pipelines

    meltano run tap-csv hash_email target-sqlite
    

Capabilities

  • stream-maps

Settings

Setting Required Default Description
stream_maps True None Stream maps

A full list of supported settings and capabilities is available by running: meltano-map-transformer --about

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

meltano_map_transform-0.12.3.tar.gz (990.8 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

meltano_map_transform-0.12.3-py3-none-any.whl (8.9 kB view details)

Uploaded Python 3

File details

Details for the file meltano_map_transform-0.12.3.tar.gz.

File metadata

  • Download URL: meltano_map_transform-0.12.3.tar.gz
  • Upload date:
  • Size: 990.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for meltano_map_transform-0.12.3.tar.gz
Algorithm Hash digest
SHA256 ddbee598e2a911670900271d5402730794a6e72b963a76594ae5479df204155d
MD5 16b9014af5dbb9bda7d493c7d00ca389
BLAKE2b-256 d93416eea1f93ee492202ec4bccb7f32a37c8939b1cae390603d7ea4479884cd

See more details on using hashes here.

Provenance

The following attestation bundles were made for meltano_map_transform-0.12.3.tar.gz:

Publisher: build.yaml on MeltanoLabs/meltano-map-transform

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file meltano_map_transform-0.12.3-py3-none-any.whl.

File metadata

File hashes

Hashes for meltano_map_transform-0.12.3-py3-none-any.whl
Algorithm Hash digest
SHA256 55512c13edde744042974a265c04c93d78a0e671932d6ff5f1188a0c90d9c7aa
MD5 737f0dd383ad969be9a6d43f0ee6764d
BLAKE2b-256 d287bf0329eb0c5f57914e9f7dfb527256f6459288ebc309b5478a51fd8f1672

See more details on using hashes here.

Provenance

The following attestation bundles were made for meltano_map_transform-0.12.3-py3-none-any.whl:

Publisher: build.yaml on MeltanoLabs/meltano-map-transform

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.12.3 This release

2 files

0.12.2

2 files

0.12.1

2 files

0.12.0

2 files

0.11.2

2 files

0.11.1

2 files

0.11.0

2 files

0.10.4

2 files

0.10.3

2 files

0.10.2

2 files

0.10.1

2 files

0.10.0

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page