A minimal modern data stack with working data pipelines in a single Docker container.
Project description
mimosa
A minimal modern data stack with working data pipelines in a single Docker container.
Explore the docs »
Report Bug
·
Request Feature
Table of Contents
About the Project
THIS IS A CONCEPT AND WORK IN PROGRESS
A minimal modern data stack with working data pipelines in a single Docker container. This is similar in concept to mimodast using alternative software options.
Useful for an exploration of the tools involved:
- dltHub for data loading
Getting Started
Prerequisites
Installation
Usage
Roadmap
- Get source data (Using REST API)
- Transform data, possibly SQL Mesh or dbt.
- Create data vault transformations (https://automate-dv.readthedocs.io/en/latest/).
- dlt update/error messages using Slack
- [-] Storage (currently local DuckDB, maybe consider some cloud alternative. Though that would stray from the data stack in a Docker concept.)
- Scheduling Tool
- Reporting tool (Metabase?)
Contributing
Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also open a feature request or bug report. Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
Contact
Project Link: mimosa
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ternyxmimosa-0.0.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d89d175e1e2ee23fe6aed517fb7a3bb607ff7e85fa9031512f2bc6064db326a5 |
|
MD5 | ae6badda5e19df23192cd7d7fe65c7c3 |
|
BLAKE2b-256 | 9aada6a6d5b5bd7e3a773058457854ff244a25cca4a86be4a75fb9b9dd886bbd |