ODDRN generator package
Project description
OpenDataDiscovery ODDRN
Requirements
Python >= 3.7
Installation
pip install oddrn
Usage and configuration
from oddrn import Generator
oddrn_gen = Generator(data_source="postgresql", cloud={"type":"aws", "region":"reg_id", "account": "acc_id"})
oddrn_gen.get_column("db_name","schema_name","table_name", "column_name")
Generator parameters:
- "data_source" - required. Can be one of:
["postgresql", "mysql", "kafka", "glue", "snowflake", "airflow", "tableau", "hive", "dynamodb", "kuberflow", "odbc", "mssql", "oracle", "redshift"]
- cloud: dict - optional. At now support only AWS. Mutually exclusive with "prefix" or "prefixes" params)
- host: str or hosts: list[str] - optional.
- prefix: str or prefixes: list[str] - optional. Mutually exclusive with "cloud" param
Methods list:
- postgresql:
- get_database(database_name)
- get_schema(database_name, schema_name)
- get_table(database_name, schema_name, table_name)
- get_column(database_name, schema_name, table_name, column_name)
- mysql
- get_database(database_name)
- get_table(database_name, table_name)
- get_column(database_name, table_name, column_name)
- kafka
- Work in progress
- glue
- get_owner(owner_name)
- get_database(database_name)
- get_table(database_name, table_name)
- get_column(database_name, table_name, column_name)
- get_job(job_name)
- snowflake
- get_owner(owner_name)
- get_warehouse(warehouse_name)
- get_database(warehouse_name, database_name)
- get_schema(warehouse_name, database_name, schema_name)
- get_table(warehouse_name, database_name, schema_name, table_name)
- get_view(warehouse_name, database_name, schema_name, view_name)
- get_column(warehouse_name, database_name, schema_name, table_name, column_name)
- airflow
- Work in progress
- tableau
- get_database(database_name)
- get_schema(database_name, schema_name)
- get_table(database_name, schema_name, table_name)
- get_column(database_name, schema_name, table_name, column_name)
- get_workbook(workbook_name)
- get_worksheet(workbook_name, worksheet_name)
- hive
- get_owner(owner_name)
- get_database(database_name)
- get_table(database_name, table_name)
- get_column(database_name, table_name, column_name)
- dynamodb
- get_database(database_name)
- get_schema(database_name, schema_name)
- get_table(database_name, schema_name, table_name)
- get_column(database_name, schema_name, table_name, column_name)
- kuberflow
- get_pipeline(pipeline_id)
- get_experiment(experiment_id)
- get_experiment_run(experiment_id, run_id)
- odbc
- get_database(database_name)
- get_schema(database_name, schema_name)
- get_table(database_name, schema_name, table_name)
- get_column(database_name, schema_name, table_name, column_name)
- mssql
- get_database(database_name)
- get_schema(database_name, schema_name)
- get_table(database_name, schema_name, table_name)
- get_column(database_name, schema_name, table_name, column_name)
- oracle
- get_database(database_name)
- get_schema(database_name, schema_name)
- get_table(database_name, schema_name, table_name)
- get_column(database_name, schema_name, table_name, column_name)
- redshift
- get_database(database_name)
- get_schema(database_name, schema_name)
- get_table(database_name, schema_name, table_name)
- get_column(database_name, schema_name, table_name, column_name)
If you need to generate full custom oddrn, use method create_full_oddrn(data: OrderedDict) Example:
from oddrn import Generator
from collections import OrderedDict
data = OrderedDict({
"sources": "CustomSource",
"hosts": "localhost:3333",
"databases": "test_db",
"tables": "test_table"
})
Generator.create_full_oddrn(data)
'//sources/CustomSource/hosts/localhost:3333/databases/test_db/tables/test_table'
Cloud support:
To add new cloud, you need to add new dataclass with method get_oddrn to clouds.py and add it to cloud_map variable
Adapter support:
To add new adapter, simply add new class to oddrn.py. Parameter "source" is required.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
oddrn-0.0.5.tar.gz
(4.9 kB
view hashes)