A static site generator for data catalogs
Project description
Carte
Carte is a Python library for extracting metadata from your data sources and generating structured Markdown files from it. It's a companion library to the Carte front end, a lightweight data catalog for fast moving data teams. Carte currently supports the AWS Glue catalog and PostgreSQL but more sources are coming.
The files generated by Carte can be fed into the front end as-is to get a statically generated data catalog.
Installation
pip install carte-cli
If you plan to use PostgreSQL as a data source, you should also install the related optional dependencies using the following command instead of the first one:
pip install carte-cli[postgres]
Usage
AWS Glue
Carte uses the Boto3 library to fetch the Glue catalog. This means you need an AWS CLI profile defined that can access the required databases and tables.
PostgreSQL
For PostgreSQL, you need to construct a connection string with your username, password, the DB's host, and the database you'd like to extract on the host.
Carte CLI works from a YAML config that you have to specify when running an extraction. An example config with both Glue and Postgres:
connections:
- type: glue
name: aws_glue
- type: postgresql
config:
connection_string: 'postgresql://<user>:<password>@<host>:<port>/<database>'
If you name this config extract.config.yml
, then you can run an extraction with:
carte extract extract.config.yml -o data/datasets
where the -o
flag specifies the output folder.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for carte_cli-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9383ec0fc53b2afce7fb906a594f23841b435913f208933a956d57832c7cd877 |
|
MD5 | c0081990519a47cad9a203d1fc4000c8 |
|
BLAKE2b-256 | 18cbcae694f9fecac04cc0de53daee82f1d9440996d3de45fc284df8da127107 |