A minimal modern data stack with working data pipelines in a single Docker container.
Project description
mimosa
A minimal modern data stack with working data pipelines in a single Docker container.
Explore the docs »
Report Bug
·
Request Feature
Table of Contents
About the Project
THIS IS A CONCEPT AND WORK IN PROGRESS
A minimal modern data stack with working data pipelines in a single Docker container. This is similar in concept to mimodast using alternative software options.
Useful for an exploration of the tools involved:
- dltHub for data loading
Getting Started
Prerequisites
Installation
Usage
Roadmap
- Get source data (Using REST API)
- Transform data, possibly SQL Mesh or dbt.
- Create data vault transformations (https://automate-dv.readthedocs.io/en/latest/).
- dlt update/error messages using Slack
- [-] Storage (currently local DuckDB, maybe consider some cloud alternative. Though that would stray from the data stack in a Docker concept.)
- Scheduling Tool
- Reporting tool (Metabase?)
Contributing
Any contributions you make are greatly appreciated.
If you have a suggestion that would make this better, please fork the repo and create a pull request. You can also open a feature request or bug report. Don't forget to give the project a star! Thanks again!
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature
) - Commit your Changes (
git commit -m 'Add some AmazingFeature'
) - Push to the Branch (
git push origin feature/AmazingFeature
) - Open a Pull Request
Contact
Project Link: mimosa
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ternyxmimosa-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 978c9b8b16679107e87cc6a8a350e1605ef96a901bb9f1dfd325b270b2c0c3ae |
|
MD5 | f9152c0efc7e575d8d149cfe95c78ab0 |
|
BLAKE2b-256 | b0ba95ab8726d749b07c974e5b4be6616d194fab7560b1d2fc8689e6d674c4ff |