A minimal modern data stack with working data pipelines in a single Docker container.
Project description
mimosa
A minimal modern data stack with working data pipelines in a single Docker container.
Explore the docs »
Report Bug
·
Request Feature
Table of Contents
About the Project
THIS IS A CONCEPT AND WORK IN PROGRESS
A minimal modern data stack with working data pipelines in a single Docker container. This is similar in concept to mimodast using alternative software options.
Useful for an exploration of the tools involved:
- dltHub for data loading
Getting Started
Prerequisites
Installation
Usage
Roadmap
- Get source data (Using REST API)
- Transform data, possibly SQL Mesh or dbt.
- Create data vault transformations (https://automate-dv.readthedocs.io/en/latest/).
- dlt update/error messages using Slack
- [-] Storage (currently local DuckDB, maybe consider some cloud alternative. Though that would stray from the data stack in a Docker concept.)
- Scheduling Tool
- Reporting tool (Metabase?)
Contributing
Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also open a feature request or bug report. Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
Contact
Project Link: mimosa
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ternyxmimosa-0.0.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8511f616a5b48da019af8f80816b8722071cb6beaa7b16c94ab792e8b2bb6522 |
|
MD5 | d04e1bcdb9f18e5ba0bb4db41287b7d1 |
|
BLAKE2b-256 | 74efb1dd47b0123039fd8af99798e211717c65e7b133bcab679bafeb92ab60b7 |