Beavers
Documentation / Installation / Repository / PyPI
Beavers is a python library for stream processing, optimized for analytics.
It is used at Tradewell Technologies, to calculate analytics and serve model predictions, for both realtime and batch jobs.
Key Features
- Works in real time (eg: reading from Kafka) and replay mode (eg: reading from Parquet files).
- Optimized for analytics, using micro-batches (instead of processing records one by one).
- Similar to incremental, it updates nodes in a dag incrementally.
- Taking inspiration from kafka streams, there are two types of nodes in the dag:
- Stream: ephemeral micro-batches of events (cleared after every cycle).
- State: durable state derived from streams.
- Clear separation between the business logic and the IO. So the same dag can be used in real time mode, replay mode or can be easily tested.
- Functional interface: no inheritance or decorator required.
- Support for complicated joins, not just "linear" data flow.
Limitations
- No concurrency support. To speed up calculation use libraries like pandas, pyarrow or polars.
- No async code. To speed up IO use kafka driver native thread or parquet IO thread pool.
- No support for persistent state. Instead of saving state, replay historic data from kafka to prime stateful nodes.
Talks
Release files for beavers 0.14.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| beavers-0.14.0.tar.gz | 158.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| beavers-0.14.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 191.6 kB
Release files / beavers-0.14.0.tar.gz
| Download URL | beavers-0.14.0.tar.gz |
|---|---|
| Size | 158.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
d84abc57cad0311ddc2bc6a43550572c787015454bc10cc0a99f0e871791fa52
|
|
BLAKE2b-256 checksum How to use checksums |
0d0dcd6f3584ab7295dc45e9965c8e0f42d9ed4cb7c8b4202d61ee8950c41e9c
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.1 {"installer":{"name":"uv","version":"0.11.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / beavers-0.14.0-py3-none-any.whl
| Download URL | beavers-0.14.0-py3-none-any.whl |
|---|---|
| Size | 32.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fba3f886fb0f15179256328f0e110220dd0f95a8efb854cf4d8b92beb4bd7473
|
|
BLAKE2b-256 checksum How to use checksums |
b7acdf49a6c741ee7ee679d3f10d6c77a619bcc29622b7bcec9c771aefa7995a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.11.1 {"installer":{"name":"uv","version":"0.11.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
