buffr 0.0.3

Collect objects and apply a function once the buffer is full or expired

   ___  __  _____________ 
  / _ )/ / / / __/ __/ _ \
 / _  / /_/ / _// _// , _/
/____/\____/_/ /_/ /_/|_| 

buffr: simple buffering mechanism for Python

Testing
Package
Meta
Social

buffr provides a simple mechanism that allows you to collect values in a buffer. Once a certain time has passed or the buffer's capacity is reached, all values will pe processed with the provided function. As an analogy: don't process each drop, wait till the bucket is full and process that.

pip install buffr

Table of Contents

• Main Features
• Usage Example
• Installation
• Dependencies
• License
• Documentation
• Development
• Contributing to Buffr

Main Features

• 🐍 Pure Python
• 🖌 Eadily configurable
• 👨‍🎨 User-friendly

Use case

You listen to a queue for messages that you'll process and insert into the database. If we receive a message each second, this means that we have to perform an insert 60 times a second.

In this case we could use Buffr to collect messages for 30 seconds before batch inserting them. This reduces the number of costly database-operations from 60 per minute to just 2.

Usage Example

Creating a Buffr is easy.

from buffr import Buffr

# Define the processor
def processing_fn(items:List):
    for item in items:
        print(item * 2)

# Create a buffr
buffr = Buffr(max_capacity=100, buffer_ttl=30, flush_func=processing_fn)

# Add some values to the Buffr
for i in range(4):
    buffr.add(i)

The Buffr defined above will flush in three cases:

1. the capacity of 100 items is exceeded
2. 30 seconds pass (buffr_ttl)
3. buffr.flush() is called

Installation

pip install buffr

The source code is currently hosted on GitHub at: https://github.com/mike-huls/buffr

Binary installers for the latest released version are available at the Python Package Index (PyPI).

Dependencies

Buffr has no dependencies

License

MIT

Documentation

🔨 Under construction

Development

Find the changelog and list of upcoming features here.
Contributions are always welcome; feel free to submit bug reports, bug fixes, feature requests, documentation improvements or enhancements!

---

Go to Top