DATAGROWTH
Data Growth is a lightweight Extract, Transform and Load (ETL) library, that helps to connect your software to external data sources.
These data sources can be API's, terminal commands or LLM's. Datagrowth is the bridge between all that data and your application logic. Because data sources and their outputs can be described uniformly in Python, using Datagrowth prevents vendor lock-in to specific data sources. It can also save costs, because it will re-use data sources like LLM responses whenever possible.
Datagrowth is mainly used within Django projects, where the data can be stored inside Django models. Use the different processors to extract and transform data, before loading it into your models exactly how your project needs. When using Django Restframework you can also share the loaded data easily over a REST API to other devices or services. Datagrowth also provides classes to set this up quickly.
Installation
You can install Datagrowth by running
pip install datagrowth
And then add datagrowth.django.apps.DatagrowthConfig to INSTALLED_APPS.
Getting started
Currently there are two major use cases. The Resources provide a way to uniformly gather data from very different sources. Configurations are a way to store and transfer key-value pairs, which your code can use to act on different contexts.
Follow these guides to get an idea how you can use Datagrowth:
Running the tests
There is a Django mock project inside the tests directory of the repository. In it are a lot of tests to demonstrate how Datagrowth and Django can work together, as well as to assure that everything functions as designed.
Apart from the Django tests there is also a pytest suite, which checks lower level Datagrowth functionality. In the future it will be feasible to use Datagrowth without Django in smaller prototype projects.
You can run all tests by running this inside the repo root:
invoke test.run
Alternatively you can execute the tests against multiple Python/Django/database versions by running:
tox
1 maintainer
Metadata
1 maintainer
Release files for datagrowth 0.22.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| datagrowth-0.22.2.tar.gz | 118.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| datagrowth-0.22.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 264.8 kB
Release files / datagrowth-0.22.2.tar.gz
| Download URL | datagrowth-0.22.2.tar.gz |
|---|---|
| Size | 118.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
70691c4d6cb6fdfa8e323960c2de86ae3228536754c80e197c8c32348191dc82
|
|
BLAKE2b-256 checksum How to use checksums |
9c3515d7251391b993c878e89c3b20e6e5d5cf2cac251a4588042f19d48db6ea
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.3
|
Release files / datagrowth-0.22.2-py3-none-any.whl
| Download URL | datagrowth-0.22.2-py3-none-any.whl |
|---|---|
| Size | 146.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a82a26b984ee095d711ff56e535106647e20f2cfaa91eb904de8a57149068cf5
|
|
BLAKE2b-256 checksum How to use checksums |
093ab24f8eea6540dcaed18b3753cf21bd8aa64ca0d914bafd13c334f01b3b04
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.12.3
|
