A static site generator for data catalogs
Project description
Carte
Carte is a Python library for extracting metadata from your data sources and generating structured Markdown files from it. It's a companion library to the Carte front end, a lightweight data catalog for fast moving data teams. Carte currently supports the AWS Glue catalog and PostgreSQL but more sources are coming.
The files generated by Carte can be fed into the front end as-is to get a statically generated data catalog.
Installation
pip install carte-cli
Usage
Carte uses the Boto3 library to fetch the Glue catalog. This means you need an AWS CLI profile defined that can access the required databases and tables.
For PostgreSQL, you need to construct a connection string with your username, password, the DB's host, and the database you'd like to extract on the host.
Carte CLI works from a YAML config that you have to specify when running an extraction. An example config with both Glue and Postgres:
connections:
- type: glue
name: aws_glue
- type: postgresql
config:
extractor.sqlalchemy.conn_string: 'postgresql://<user>:<password>@<host>:<port>/<database>'
If you name this config extract.config.yml
, then you can run an extraction with:
carte -c extract.config.yml -o data/datasets
where the -o
flag specifies the output folder.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for carte_cli-0.2.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | e0c185da086f21318d72aa09f10e255ca4482d61fe491a78a4bb766a64da666e |
|
MD5 | 1077d1840ed640d781a1dad1d0a1ca28 |
|
BLAKE2b-256 | 6533289571ac52d8b7c44ed148d3f5541dbe9fdb7e6001cab327869c8504f9bf |