metaphor-connectors 0.11.50

A collection of Python-based 'connectors' that extract metadata from various sources to ingest into the Metaphor app.

Metaphor Connectors

This repository contains a collection of Python-based "connectors" that extract metadata from various sources to ingest into the Metaphor platform.

Installation

This package requires Python 3.7+ installed. You can verify the version on your system by running the following command,

python -V  # or python3 on some systems

Once verified, you can install the package using pip,

pip install "metaphor-connectors[all]"  # or pip3 on some systems

This will install all the connectors and required dependencies. You can also choose to install only a subset of the dependencies by installing the specific extra, e.g.

pip install "metaphor-connectors[snowflake]"

Similarly, you can also install the package using requirements.txt or pyproject.toml.

Connectors

Each connector is placed under its own directory under metaphor and extends the metaphor.common.BaseExtractor class.

Connector Name	Metadata
airflow_plugin	Lineage
bigquery	Schema, description, statistics
bigquery.lineage	Lineage
bigquery.profile	Data profile
bigquery.query	Queries
bigquery.usage	Data usage
dbt	dbt model, test, lineage
dbt.cloud	dbt model, test, lineage
glue	Schema, description
looker	Looker view, explore, dashboard, lineage
manual.metadata	Custom metadata
manual.governance	Ownership, tags
manual.lineage	Lineage
metabase	Dashboard, lineage
postgresql	Schema, description, statistics
postgresql.profile	Data profile
postgresql.usage	Usage
power_bi	Dashboard, lineage
redshift	Schema, description, statistics
redshift.lineage	Lineage
redshift.profile	Data profile
redshift.query	Queries
redshift.usage	Usage
snowflake	Schema, description, statistics
snowflake.lineage	Lineage
snowflake.profile	Data profile
snowflake.query	Queries
snowflake.usage	Data usage
tableau	Dashboard, lineage
unity_catalog	Schema, description

Development

See Development Environment for more instructions on how to setup your local development environment.