Skip to main content

A collection of Python-based 'connectors' that extract metadata from various sources to ingest into the Metaphor app.

Project description

Metaphor Connectors

Coverage CodeQL PyPI Version Python version 3.8+ PyPI Downloads License

This repository contains a collection of Python-based "connectors" that extract metadata from various sources to ingest into the Metaphor platform.

Installation

This package requires Python 3.8+ installed. You can verify the version on your system by running the following command,

python -V  # or python3 on some systems

Once verified, you can install the package using pip,

pip install "metaphor-connectors[all]"  # or pip3 on some systems

This will install all the connectors and required dependencies. You can also choose to install only a subset of the dependencies by installing the specific extra, e.g.

pip install "metaphor-connectors[snowflake]"

Similarly, you can also install the package using requirements.txt or pyproject.toml.

Connectors

Each connector is placed under its own directory under metaphor and extends the metaphor.common.BaseExtractor class.

Connector Name Metadata
airflow_plugin Lineage
bigquery Schema, description, statistics, queries
bigquery.lineage Lineage
bigquery.profile Data profile
dbt dbt model, test, lineage
dbt.cloud dbt model, test, lineage
fivetran lineage
glue Schema, description
looker Looker view, explore, dashboard, lineage
custom.data_quality Data quality
custom.governance Ownership, tags, description
custom.lineage Lineage
custom.metadata Custom metadata
metabase Dashboard, lineage
mssql Schema
mysql Schema, description
postgresql Schema, description, statistics
postgresql.profile Data profile
postgresql.usage Usage
power_bi Dashboard, lineage
redshift Schema, description, statistics, queries
redshift.lineage Lineage
redshift.profile Data profile
snowflake Schema, description, statistics, queries
snowflake.lineage Lineage
snowflake.profile Data profile
synapse Schema, queries
tableau Dashboard, lineage
thought_spot Dashboard, lineage
unity_catalog Schema, description

Development

See Development Environment for more instructions on how to setup your local development environment.

Custom Connectors

See Adding a Custom Connector for instructions and a full example on how to create your custom connectors.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

metaphor_connectors-0.12.5.tar.gz (162.8 kB view hashes)

Uploaded Source

Built Distribution

metaphor_connectors-0.12.5-py3-none-any.whl (236.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page