CircularDict
CircularDict is a Python
dict that acts as a Circular Buffer. This dictionary maintains a controlled size, limited either by a specified number of items (maxlen) or total size in bytes (maxsize_bytes). Upon reaching the defined limit, CircularDict automatically removes the oldest entries, maintaining memory usage under control.
Built upon Python's OrderedDict, CircularDict inherits all standard dictionary usage and operations, augmented by the capability of memory management. It's particularly useful in scenarios such as caching, where limiting memory consumption is crucial. The class combines dictionary and circular-queue behaviors, providing an efficient and scalable solution for various use cases.
Installation
To install CircularDict simply run:
pip install circular-dict
Usage
Working with CircularDict is as simple as using a standard Python dict, with additional parameters maxlen or maxsize_bytes on the initialization to control the buffer size. You can use one of them or both.
from circular_dict import CircularDict
# Initialize a CircularDict with a maximum length of 3 items and a storage limit of 4Mb
my_dict = CircularDict(maxlen=3, maxsize_bytes=4*1024*1024)
Example with maxlen
You can use maxlen to define the maximum amount of items that the dictionary can store. It is useful for defining fixed size buffers.
from circular_dict import CircularDict
# Initialize a CircularDict with a maximum length of 3
my_buffer = CircularDict(maxlen=3)
# Fill it with 3 items
my_buffer['item1'] = 'value1'
my_buffer['item2'] = 'value2'
my_buffer['item3'] = 'value3'
print(f"When filling it: {circ_dict}")
# Add another item
my_buffer['item4'] = 'value4'
print(f"After adding an element beyond maxlen: {circ_dict}")
Output:
When filling it: {'item1': 'value1', 'item2': 'value2', 'item3': 'value3'}
After adding an element beyond maxlen: {'item2': 'value2', 'item3': 'value3', 'item4': 'value4'}
Example with maxsize_bytes
You can use maxsize_bytes to define the maximum amount of memory that the dict can store. It is particularly beneficial when defining caches, to prevent memory overflows.
from circular_dict import CircularDict
import numpy as np
import sys
# Initialize a CircularDict with a maximum length of 100KB
my_buffer = CircularDict(maxsize_bytes=100*1024)
# Add two arrays of ~40Kb (10*1024*4 bytes (int32) + 5 bytes (chars) + 100 bytes (numpy structure) + 50 bytes (str structure))
my_buffer['item1'] = np.zeros((10, 1024), dtype=np.int32)
my_buffer['item2'] = np.ones((10, 1024), dtype=np.int32)
print(f"{len(my_buffer)} Elements {tuple(my_buffer.keys())}. Dict size: {my_buffer.current_size/1024} Kb")
# Add a new element of ~32Kb will delete oldest elements ('item1') until fitting in the `dict`.
my_buffer['item3'] = np.ones((8, 1024), dtype=np.int32)
print(f"{len(my_buffer)} Elements {tuple(my_buffer.keys())}. Dict size: {my_buffer.current_size/1024} Kb")
# Create an element of ~160Kb (bigger than the defined maximum storage) to trigger a MemoryError
too_big_array = np.ones((40, 1024), dtype=np.int32)
try:
# Try to add it to the dict
my_buffer['item4'] = too_big_array
except MemoryError:
print(f"Cannot add an element with {sys.getsizeof(too_big_array)/1024}Kb in a dict with"\
f"maxsize_bytes of {my_buffer.maxsize_bytes/1024} Kb. Current elements {tuple(my_buffer.keys())}")
Output
2 Elements ('item1', 'item2'). Dict size: 80.35 Kb
2 Elements ('item2', 'item3'). Dict size: 72.35 Kb
Cannot add an element with 160.12Kb in a dict with maxsize_bytes of 100.0 Kb. Current elements ('item2', 'item3')
Please remember that the maxsize_bytes parameter considers the total memory footprint, including the sizes of keys and values. If you try to add an item that exceeds the maxsize_bytes, a MemoryError will be raised.
Metadata
Release files for circular-dict 1.9
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| circular_dict-1.9.tar.gz | 6.2 kB | Without provenance | Details for circular_dict-1.9.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| circular_dict-1.9-py3-none-any.whl | Python 3 | none | any | Details for circular_dict-1.9-py3-none-any.whl |
Total release size: 12.8 kB
Release files / circular_dict-1.9.tar.gz
| Download URL | circular_dict-1.9.tar.gz |
|---|---|
| Size | 6.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
dc93ea0b5a79de6235a63b734446f1c3fa57498ff9791e7763a232558baa2f7a
|
|
BLAKE2b-256 checksum How to use checksums |
5cdd0c23c95fc6cc02e8566957d15063bea03d2422b4a095a544d4279b5f2373
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.12.3
|
Release files / circular_dict-1.9-py3-none-any.whl
| Download URL | circular_dict-1.9-py3-none-any.whl |
|---|---|
| Size | 6.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f7936bb7487c93fe478119e47164777af9bf95c8fa4788f20ea41d4c00cdcea9
|
|
BLAKE2b-256 checksum How to use checksums |
d87b9da30fa823f020f3dc39bdee7a381dd3cae877933100a75700edb630fa8b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/5.0.0 CPython/3.12.3
|
Security for circular-dict 1.9
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.