Example Pipeline with brickblock [bb-core]
Overview
This project demonstrates how to build a flexible data processing pipeline using the bb-core library. The pipeline is designed to process data through a series of functions or modules, supporting both synchronous and asynchronous execution, with real-time updates via server-sent events (SSE).
Installation
-
Install the repository:
pip install bb-core
-
Set up a virtual environment (optional but recommended):
python -m venv venv
Activate the virtual environment:
- Windows:
. env\Scriptsctivate
- macOS/Linux:
source venv/bin/activate
- Windows:
Project Structure
- Pipeline Class: Manages the flow of data through a series of functions.
- BaseModule Class: Abstract base class that all modules should inherit from. Modules must implement the following methods:
run(): The main processing function.onProgressStartMessage(): Sends a progress start message.onProgressEndMessage(): Sends a progress end message.
- SSE Generator: Asynchronous generator that sends real-time updates to clients.
Example Usage
from your_module import Pipeline, BModule, InputModel2
# Initialize the pipeline
pipeline = Pipeline.init(name="example_pipeline", sse=True)
# Add a module to the pipeline
bmodule = BModule()
pipeline.modules([bmodule])
# Input data for the pipeline
input_data = {"c": 5.0}
# Asynchronously run the pipeline and get SSE events
async def test_sse():
async for event in pipeline.sse_generator(InputModel2(**input_data)):
print(event)
# Run the asynchronous SSE generator
asyncio.run(test_sse())
This project is licensed under the MIT License.
1 maintainer
Metadata
1 maintainer
Release files for bb-core 0.3.12
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| bb_core-0.3.12.tar.gz | 19.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| bb_core-0.3.12-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 47.1 kB
Release files / bb_core-0.3.12.tar.gz
| Download URL | bb_core-0.3.12.tar.gz |
|---|---|
| Size | 19.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
1ccd54c9f166e69dcfa13ef3adc7062fc03a7ae5ceab4d321ad55588b4628d18
|
|
BLAKE2b-256 checksum How to use checksums |
be2fa81ff3642accd1ef008d8a8d02888c688fd065c8ce2e24ef4c1d6a58e853
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.2
|
Release files / bb_core-0.3.12-py3-none-any.whl
| Download URL | bb_core-0.3.12-py3-none-any.whl |
|---|---|
| Size | 27.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
7fe2761c53cc73f6d17ff285ddb3d3cb5e63ea5b04a02c22831d5008089733d6
|
|
BLAKE2b-256 checksum How to use checksums |
78c665f84109b9c7ffd3e299527744857813472190707c0fa00c0d6accdcb4c3
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.13.2
|
