A minimal modern data stack with working data pipelines in a single Docker container.
Project description
mimosa
A minimal modern data stack with working data pipelines in a single Docker container.
Explore the docs »
Report Bug
·
Request Feature
Table of Contents
About the Project
THIS IS A CONCEPT AND WORK IN PROGRESS
A minimal modern data stack with working data pipelines in a single Docker container. This is similar in concept to mimodast using alternative software options.
Useful for an exploration of the tools involved:
- dltHub for data loading
Getting Started
Prerequisites
Installation
Usage
Roadmap
- Get source data (Using REST API)
- Transform data, possibly SQL Mesh or dbt.
- Create data vault transformations (https://automate-dv.readthedocs.io/en/latest/).
- dlt update/error messages using Slack
- [-] Storage (currently local DuckDB, maybe consider some cloud alternative. Though that would stray from the data stack in a Docker concept.)
- Scheduling Tool
- Reporting tool (Metabase?)
Contributing
Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also open a feature request or bug report. Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
Contact
Project Link: mimosa
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ternyxmimosa-0.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 06f03238229a3f3a57190eb4632be938fe1e6f5acad3a1ce79e731635398276a |
|
MD5 | b186933c8b225d0d51f1daed12a8803f |
|
BLAKE2b-256 | d7ad75589f6388225892893f86e28ffa03d59d5d4cb798f4dad53d10ab713cca |