CDF (Continuous Data Framework)
Craft end-to-end data pipelines and manage them continuously
📖 Table of Contents
- 📖 Table of Contents
- 📍 Overview
- 📦 Features
- 🚀 Getting Started
- 📚 Documentation
- 🤝 Contributing
- 🛣 Roadmap
- 📄 License
- 👏 Acknowledgments
📍 Overview
CDF (Continuous Data Framework) is an integrated framework designed to manage data across the entire lifecycle, from ingestion through transformation to publishing. It is built on top of two open-source projects, sqlmesh and dlt, providing a unified interface for complex data operations. CDF simplifies data engineering workflows, offering scalable solutions from small to large projects through an opinionated project structure that supports both multi-workspace and single-workspace layouts.
Features
...
Getting Started
-
Installation:
CDF requires Python 3.9 or newer. Install CDF using pip:
pip install python-cdf
Documentation
For detailed documentation, including API references and tutorials, visit CDF Documentation.
Contributing
Contributions to CDF are welcome! Please refer to the contributing guidelines for more information on how to submit pull requests, report issues, or suggest enhancements.
License
CDF is licensed under Apache 2.0 License.
This README provides an overview of the CDF tool, highlighting its primary features, installation steps, basic usage examples, and contribution guidelines. It serves as a starting point for users to understand the capabilities of CDF and how it can be integrated into their data engineering workflows.
🧪 Tests
Run the tests with pytest:
pytest tests
🛣 Project Roadmap
TODO: Add a roadmap for the project.
🤝 Contributing
Contributions are welcome! Here are several ways you can contribute:
- Submit Pull Requests: Review open PRs, and submit your own PRs.
- Join the Discussions: Share your insights, provide feedback, or ask questions.
- Report Issues: Submit bugs found or log feature requests for z3z1ma.
Contributing Guidelines
Click to expand
- Fork the Repository: Start by forking the project repository to your GitHub account.
- Clone Locally: Clone the forked repository to your local machine using a Git client.
git clone <your-forked-repo-url>
- Create a New Branch: Always work on a new branch, giving it a descriptive name.
git checkout -b new-feature-x
- Make Your Changes: Develop and test your changes locally.
- Commit Your Changes: Commit with a clear and concise message describing your updates.
git commit -m 'Implemented new feature x.'
- Push to GitHub: Push the changes to your forked repository.
git push origin new-feature-x
7a. Submit a Pull Request: Create a PR against the original project repository. Clearly describe the changes and their motivations.
Once your PR is reviewed and approved, it will be merged into the main branch.
📄 License
This project is distributed under the Apache 2.0 License. For more details, refer to the LICENSE file.
👏 Acknowledgments
- Harness (https://harness.io/) for being the proving grounds in which the initial concept of this project was born.
- SQLMesh (https://sqlmesh.com) for being a foundational pillar of this project as well as the team for their support, advice, and guidance.
- DLT (https://dlthub.com) for being the other foundational pillar of this project as well as the team for their support, advice, and guidance.
1 maintainer
Metadata
1 maintainer
Release files for python-cdf 0.9.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| python_cdf-0.9.4.tar.gz | 49.3 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| python_cdf-0.9.4-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 106.2 kB
Release files / python_cdf-0.9.4.tar.gz
| Download URL | python_cdf-0.9.4.tar.gz |
|---|---|
| Size | 49.3 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7b5d9c23aaf5aba86a9076e789dac47d02c70a3b92ec36704a82faeacd2de227
|
|
BLAKE2b-256 checksum How to use checksums |
d346f47c114ea3ee2fd6471956c12e290c7f8eb76ac008e12bb824efd408a9b8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.9.0.dev0 CPython/3.11.9 Darwin/23.1.0
|
Release files / python_cdf-0.9.4-py3-none-any.whl
| Download URL | python_cdf-0.9.4-py3-none-any.whl |
|---|---|
| Size | 56.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fbfe5fad6772148d6ed9376706590934890c5e264f886ea68e12fbc6965c9a2e
|
|
BLAKE2b-256 checksum How to use checksums |
4244dd753e109bc903b22ebc2b4f060162ee3837a1dc1c7f4046a3fcedb92e85
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.9.0.dev0 CPython/3.11.9 Darwin/23.1.0
|
Security for python-cdf 0.9.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
