chainlet
The chainlet library offers a lightweight model to create processing pipelines from generators, coroutines, functions and custom objects. Instead of requiring you to nest code or insert hooks, chainlet offers a concise, intuitive binding syntax:
# regular nested generators
csv_writer(flatten(xml_reader(path='data.xml'), join='.'.join), path='data.csv')
# chainlet pipeline
xml_reader(path='data.xml') >> flatten(join='.'.join) >> csv_writer(path='data.csv')Creating new chainlets is simple, requiring you only to define the processing of data. It is usually sufficient to use regular functions, generators or coroutines, and let chainlet handle the rest:
@chainlet.genlet
def moving_average(window_size=8):
buffer = collections.deque([(yield)], maxlen=window_size)
while True:
new_value = yield(sum(buffer)/len(buffer))
buffer.append(new_value)Features
We have designed chainlet to be a simple, intuitive library:
Modularize your code with small, independent processing blocks.
Intuitively compose processing pipelines from individual elements.
Automatically integrate functions, generators and coroutines in your pipelines.
Extend your processing with complex pipelines that fork and join as needed.
Under the hood, chainlet merges iterator and functional paradigms in a minimal fashion to stay lightweight.
Fully compliant with the Generator interface to integrate with existing code.
Implicit tail recursion elimination for linear pipelines, and premature end of chain traversal.
Push and pull chains iteratively, continuously, or even asynchronously.
Simple interface to extend or supersede pipeline traversal and processing.
At its heart chainlet strives to be as Pythonic as possible: You write python, and you get python. No trampolines, callbacks, stacks, handlers, …
We take care of the ugly bits so you do not have to.
Looking to get started?
Check out our docs:
Found an issue or have suggestions?
Head straight to our issue tracker:
Status
We use the chainlet library in a production environment. It serves to configure and drive stream based data extraction and translation for monitoring. Both the grammar and general interfaces for processing chains, trees and graphs are stable.
Ongoing work is mainly focused on the iteration interface. We plan to add automatic concurrency, asynchronicity and parallelism. Our target is an opt-in approach to features from functional programming and static optimisations.
1 maintainer
Metadata
1 maintainer
Release files for chainlet 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| chainlet-1.0.0.tar.gz | 5.0 MB | Without provenance | Details for chainlet-1.0.0.tar.gz |
Release files / chainlet-1.0.0.tar.gz
| Download URL | chainlet-1.0.0.tar.gz |
|---|---|
| Size | 5.0 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ab75ec98e69f336a1f0d228ef1d91401837bbdba5d71fee570350f0acd8d1a97
|
|
BLAKE2b-256 checksum How to use checksums |
52807a492d56db282f691d18db8f1fe5e7f89be9cbfc6e7157ff64fbcdfe39c8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
Security for chainlet 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
