zimscraperlib
Collection of python code to re-use across python-based scrapers
Usage
- This library is meant to be installed via PyPI (
zimscraperlib). - Make sure to reference it using a version code as the API is subject to frequent changes.
- API should remain the same only within the same minor version.
Example usage:
zimscraperlib>=1.1,<1.2
See documentation at Read the Docs for details.
Dependencies
Most dependencies are installed automatically by pip (from PyPI by default). The following system packages may be required depending on which features you use:
- libmagic — required for file type detection (used in most scrapers)
- wget — required only for
zimscraperlib.downloadfunctions - FFmpeg — required only for video processing functions
- gifsicle (>=1.92) — required only for GIF optimization
- libcairo — required only for SVG-to-PNG conversion
- libzim — auto-installed via PyPI, not available on Windows
- Pillow — auto-installed via PyPI; pre-built wheels are used by default and no system image libraries are needed. Only if you need to build Pillow from source should you install additional system libraries — see Pillow's build documentation for details.
Note: To run the full test suite, all system dependencies listed above must be installed.
macOS
brew install libmagic wget ffmpeg gifsicle cairo
Linux
sudo apt install libmagic1 wget ffmpeg gifsicle libcairo2
Alpine
apk add ffmpeg gifsicle libmagic wget cairo
Contribution
This project adheres to openZIM's Contribution Guidelines.
This project has implemented openZIM's Python bootstrap, conventions and policies v2.0.0.
All instructions below must be run from the root of your local clone of this repository.
If you do not already have it on your system, install hatch:
pip install hatch
Start a hatch shell — this will install all dependencies including dev in an isolated virtual environment:
hatch shell
Set up the pre-commit Git hook (runs linters automatically before each commit):
pre-commit install
Run tests with coverage:
invoke coverage
Users
Non-exhaustive list of scrapers using it (check status when updating API):
1 maintainer
Metadata
1 maintainer
Release files for zimscraperlib 5.5.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| zimscraperlib-5.5.1.tar.gz | 6.7 MB | With provenance | Details for zimscraperlib-5.5.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| zimscraperlib-5.5.1-py3-none-any.whl | Python 3 | none | any | Details for zimscraperlib-5.5.1-py3-none-any.whl |
Total release size: 6.8 MB
Release files / zimscraperlib-5.5.1.tar.gz
| Download URL | zimscraperlib-5.5.1.tar.gz |
|---|---|
| Size | 6.7 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
e5fd27d3c913ad659a9759c7a975144b607a538bf317f3b2e20f5d8699d11855
|
|
BLAKE2b-256 checksum How to use checksums |
902317255849d1574a8df0b960af9a1cb576c04d1a7535d211e2724f561b5423
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.
Transparency logRelease files / zimscraperlib-5.5.1-py3-none-any.whl
| Download URL | zimscraperlib-5.5.1-py3-none-any.whl |
|---|---|
| Size | 139.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
be75bd66f5a8ee39cca0f66226f3f2d47628bdd783a409e71df29636fb9ca174
|
|
BLAKE2b-256 checksum How to use checksums |
c54b5d55f507207c8ab0257129241c04b2b4364a9e9f3f5127fcbf1255419d8f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.
Transparency logSecurity for zimscraperlib 5.5.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
All 2 files in this release have been uploaded with provenance metadata.
Attestation for zimscraperlib-5.5.1.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Oct 30, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
