Chunk large QuerySets into small chunks, and iterate over them without killing your RAM.
Tested with all the combinations of:
Python: 3.5, 3.6, 3.7, 3.8
Django: 2, 2.1, 2.2, 3.0, master
Usage
from chunkator import chunkator
for item in chunkator(LargeModel.objects.all(), 200):
do_something(item)This tool is intended to work on Django querysets.
Your model must define a pk field (this is done by default, but sometimes it can be overridden) and this pk has to be unique. django- chunkator has been tested with PostgreSQL and SQLite, using regular PKs and UUIDs as primary keys.
You can also use values():
from chunkator import chunkator
for item in chunkator(LargeModel.objects.values('pk', 'name'), 200):
do_something(item)If you want to manipulate the pages directly, you can use chunkator_page:
from chunkator import chunkator_page
queryset = LargeModel.objects.all().values('pk')
for page in chunkator_page(queryset, 200):
launch_some_task([item['pk'] for item in page])License
MIT License.
5 maintainers
Metadata
5 maintainers
Release files for django-chunkator 2.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| django-chunkator-2.0.0.tar.gz | 3.6 kB | Without provenance | Details for django-chunkator-2.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| django_chunkator-2.0.0-py3-none-any.whl | Python 3 | none | any | Details for django_chunkator-2.0.0-py3-none-any.whl |
Total release size: 6.9 kB
Release files / django-chunkator-2.0.0.tar.gz
| Download URL | django-chunkator-2.0.0.tar.gz |
|---|---|
| Size | 3.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
57d705966762e6ba6879c2344a1b9288045f64c638fa47cbe00467f0eeeadfaf
|
|
BLAKE2b-256 checksum How to use checksums |
acc3b3dd9b543068294cb6cae2b965aba927ac71b8adf17e5255ec2f0632cf78
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/39.2.0 requests-toolbelt/0.8.0 tqdm/4.31.1 CPython/3.6.5
|
Release files / django_chunkator-2.0.0-py3-none-any.whl
| Download URL | django_chunkator-2.0.0-py3-none-any.whl |
|---|---|
| Size | 3.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
210713faf05d68035d067de9b015868b3d902d5405865401384a3c134f64c43a
|
|
BLAKE2b-256 checksum How to use checksums |
1e20c3b02663d901ddbdbc89a7b98de4cdae768214b0931921a14953ef058eb0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/39.2.0 requests-toolbelt/0.8.0 tqdm/4.31.1 CPython/3.6.5
|
Security for django-chunkator 2.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
