A minimal modern data stack with working data pipelines in a single Docker container.
Project description
mimosa
A minimal modern data stack with working data pipelines in a single Docker container.
Explore the docs »
Report Bug
·
Request Feature
Table of Contents
About the Project
THIS IS A CONCEPT AND WORK IN PROGRESS
A minimal modern data stack with working data pipelines in a single Docker container. This is similar in concept to mimodast using alternative software options.
Useful for an exploration of the tools involved:
- dltHub for data loading
Getting Started
Prerequisites
Installation
Usage
Roadmap
- Get source data (Using REST API)
- Transform data, possibly SQL Mesh or dbt.
- Create data vault transformations (https://automate-dv.readthedocs.io/en/latest/).
- dlt update/error messages using Slack
- [-] Storage (currently local DuckDB, maybe consider some cloud alternative. Though that would stray from the data stack in a Docker concept.)
- Scheduling Tool
- Reporting tool (Metabase?)
Contributing
Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also open a feature request or bug report. Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
Contact
Project Link: mimosa
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ternyxmimosa-0.0.11-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3893a57a0f7c77502b659a6271600878dacfb840304ad513d0684fdff7f2e0d6 |
|
MD5 | 0b30e472474f9a5f1b9433d771cb7521 |
|
BLAKE2b-256 | bde2adc3fb4c66db3371de30748d436fb92e311d14ce04eadefcd397d60bf0b6 |