The DAMN (Data Assets Metric Navigation) tool extracts and reports metrics about your data assets
Project description
████████▄ ▄████████ ▄▄▄▄███▄▄▄▄ ███▄▄▄▄
███ ▀███ ███ ███ ▄██▀▀▀███▀▀▀██▄ ███▀▀▀██▄
███ ███ ███ ███ ███ ███ ███ ███ ███
███ ███ ███ ███ ███ ███ ███ ███ ███
███ ███ ▀███████████ ███ ███ ███ ███ ███
███ ███ ███ ███ ███ ███ ███ ███ ███
███ ▄███ ███ ███ ███ ███ ███ ███ ███
████████▀ ███ █▀ ▀█ ███ █▀ ▀█ █▀
Data Asset Metrics Navigator
The DAMN tool extracts and reports metrics about your data assets.
It allows you to inspect your assets, lineage, and all sorts of metrics around materialization, usage, physical space usage and query performance. The objective of the DAMN tool is to give you a convenient command-line tool to track and report on the data assets you're working on.
Installation
To install the DAMN tool, run the following command:
pip install damn-tool
Connector Configuration
The DAMN tool leverages various connectors to interact with different data systems. Configuring these connectors is done via a YAML file located at ~/.damn/connectors.yml
.
YAML Configuration Structure
The configuration file uses the following structure:
connector_name:
profile_name:
param1: value1
param2: value2
- connector_name: The name of the connector (e.g., dagster, dbt, s3, etc.).
- profile_name: The name of the profile for the connector. You can have multiple profiles per connector (e.g., prod, dev, test, etc.).
- param1, param2, etc.: The parameters needed for each connector. The required parameters will depend on the specific connector. For example, a Dagster connector might require endpoint and api_token.
Switching Between Profiles
The active profile for each connector can be changed by specifying the profile when running damn commands. By default, damn will use the prod profile if no profile is specified.
Example usage:
damn ls --profile dev
Connectors
Dagster
This is for the moment the default and only connector supported by the DAMN tool. Here's an example configuration for a dagster connector with prod and dev profiles:
dagster:
prod:
endpoint: https://your-dagster-instance.com/prod/graphql
api_token: your-api-token
dev:
endpoint: https://your-dagster-instance.com/dev/graphql
api_token: your-dev-api-token
Usage
Here are some examples of how to use this CLI tool:
List assets
foo@bar:~$ damn ls
List all assets for a specifc key group
foo@bar:~$ damn ls --prefix gdelt
Show details for a specific asset
foo@bar:~$ damn show gdelt/gdelt_gkg_articles
Contribution
Contributions to the DAMN tool are always welcome. Whether it's feature requests, bug fixes, or new features, your contribution is appreciated.
License
The DAMN tool is open-source software, licensed under MIT.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for damn_tool-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8105b4d17b8de9d3ee3a7a5688f70f8ea737b07d2e4a65e4f22c025edf833860 |
|
MD5 | b92c95fa3d8dc4ff59df955d9b7864ff |
|
BLAKE2b-256 | 6b04db899e51420c3c4b03a765ccd567e2a250a40514e285008316017ecee268 |