Collection of python tools to re-use common code across scrapers
Project description
zimscraperlib
Collection of python code to re-use across python-based scrapers
Usage
- This library is meant to be installed via PyPI (
zimscraperlib
). - Make sure to reference it using a version code as the API is subject to frequent changes.
- API should remain the same only within the same minor version.
Example usage:
zimscraperlib>=1.1,<1.2
Dependencies
- libmagic
- wget
- libzim (auto-installed, not available on Windows)
- Pillow
- FFmpeg
macOS
brew install libmagic wget libtiff libjpeg webp little-cms2 ffmpeg
Linux
sudo apt install libmagic1 wget ffmpeg \
libtiff5-dev libjpeg8-dev libopenjp2-7-dev zlib1g-dev \
libfreetype6-dev liblcms2-dev libwebp-dev tcl8.6-dev tk8.6-dev python3-tk \
libharfbuzz-dev libfribidi-dev libxcb1-dev
Users
Non-exhaustive list of scrapers using it (check status when updating API):
releasing
- Update your dependencies:
pip install -U setuptools wheel twine
- Make sure CHANGELOG.md is up-to-date
- Bump version on
src/zimscraperlib/VERSION
- Build packages
python ./setup.py sdist bdist_wheel
- Upload to PyPI
twine upload dist/zimscraperlib-2.0.0*
. - Commit your Changelog + version bump changes
- Tag version on git
git tag -a v2.0.0
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
zimscraperlib-1.3.1.tar.gz
(66.0 kB
view hashes)
Built Distribution
Close
Hashes for zimscraperlib-1.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f5273b309c1db15e084cc67ff3728a150f07cbd8584cf3425439b211abeef4a5 |
|
MD5 | 8092e29a1a1d2e75be000cf516eda642 |
|
BLAKE2b-256 | 73a298fed3989c7e5bf076ee09994ef5213367b8da5807a6a6bda03b9c98d639 |