data-grimorium 0.1.1

Data Grimorium is a collection of utilities for Data Scientists and Machine Learning Engineers, designed to streamline workflows and accelerate day-to-day coding tasks.

Data Grimorium

Data Grimorium is a collection of utilities for Data Scientists and Machine Learning Engineers, designed to streamline workflows and accelerate day-to-day coding tasks.

Resources

For the developers, check the wiki Package & Modules Section.

Please refer to this Contributing Guidelines in order to contribute to the repository.

Setup

Environment Variables

Add the project root directory as DATA_GRIMORIUM_ROOT_PATH environment variable.

export DATA_GRIMORIUM_ROOT_PATH="/<absolute_path>/DataGrimorium"

Create a .env file in the root folder like

# Set the Root Path
DATA_GRIMORIUM_ROOT_PATH="/<absolute_path>/DataGrimorium"

Setup gcloud CLI

Install gcloud on the local machine (Guide).

Authenticate locally to GCP:

gcloud auth login

Set the project ID.

# List all the projects
gcloud projects list

# Set the project
gcloud config set project <project_id>

Create authentication keys.

gcloud auth application-default login

Justfile

just is a handy way to save and run project-specific commands

The main benefit it to keep all configuration and scripts in one place.

It uses the .env file for ingesting variables.

You can install it by following the Documentation. Afterward, you can execute existing commands located in the justfile.

Type just to list all available commands.

Pre-commit

# Install
pre-commit install

# Check locally
pre-commit run --all-files