Skip to main content

yEnc decoding of usenet data using SIMD routines

Project description

SABYenc 3 - yEnc decoding of usenet data using SIMD routines

Modification of the original yenc module for use within SABnzbd. The module was extended to do header parsing and full yEnc decoding from a Python list of chunks, the way in which data is retrieved from usenet. This is particularly beneficial when SSL is enabled, which limits the size of each chunk to 16K. Parsing these chunks in python is much more costly. Additionally, this module releases Python's GIL during decoding, greatly increasing performance of the overall download process.

Further improved by using yencode from animetosho, which utilizes x86/ARM SIMD optimised routines if such CPU features are available.

Installing

As simple as running:

pip install sabyenc3 --upgrade

When you want to compile from sources, you can run in the sabyenc directory:

pip install .

SIMD detection

To see which SIMD set was detected on your system, run:

python -c "import sabyenc3; print(sabyenc3.simd);"

Testing

For testing we use pytest (install via pip install -r tests/requirements.txt) and test can simply be executed by browsing to the sabyenc directory and running:

pytest

Note that tests can fail if git modified the line endings of data files when checking out the repository!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sabyenc3-5.1.3.tar.gz (224.6 kB view hashes)

Uploaded Source

Built Distributions

sabyenc3-5.1.3-cp310-cp310-win_amd64.whl (37.6 kB view hashes)

Uploaded CPython 3.10 Windows x86-64

sabyenc3-5.1.3-cp310-cp310-win32.whl (33.0 kB view hashes)

Uploaded CPython 3.10 Windows x86

sabyenc3-5.1.3-cp310-cp310-musllinux_1_1_x86_64.whl (1.0 MB view hashes)

Uploaded CPython 3.10 musllinux: musl 1.1+ x86-64

sabyenc3-5.1.3-cp310-cp310-musllinux_1_1_i686.whl (922.3 kB view hashes)

Uploaded CPython 3.10 musllinux: musl 1.1+ i686

sabyenc3-5.1.3-cp310-cp310-musllinux_1_1_aarch64.whl (679.4 kB view hashes)

Uploaded CPython 3.10 musllinux: musl 1.1+ ARM64

sabyenc3-5.1.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (466.3 kB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ x86-64

sabyenc3-5.1.3-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl (434.2 kB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ i686

sabyenc3-5.1.3-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (146.6 kB view hashes)

Uploaded CPython 3.10 manylinux: glibc 2.17+ ARM64

sabyenc3-5.1.3-cp310-cp310-macosx_10_9_universal2.whl (82.7 kB view hashes)

Uploaded CPython 3.10 macOS 10.9+ universal2 (ARM64, x86-64)

sabyenc3-5.1.3-cp39-cp39-win_amd64.whl (37.6 kB view hashes)

Uploaded CPython 3.9 Windows x86-64

sabyenc3-5.1.3-cp39-cp39-win32.whl (32.9 kB view hashes)

Uploaded CPython 3.9 Windows x86

sabyenc3-5.1.3-cp39-cp39-musllinux_1_1_x86_64.whl (1.0 MB view hashes)

Uploaded CPython 3.9 musllinux: musl 1.1+ x86-64

sabyenc3-5.1.3-cp39-cp39-musllinux_1_1_i686.whl (922.1 kB view hashes)

Uploaded CPython 3.9 musllinux: musl 1.1+ i686

sabyenc3-5.1.3-cp39-cp39-musllinux_1_1_aarch64.whl (679.3 kB view hashes)

Uploaded CPython 3.9 musllinux: musl 1.1+ ARM64

sabyenc3-5.1.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (466.0 kB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ x86-64

sabyenc3-5.1.3-cp39-cp39-manylinux_2_17_i686.manylinux2014_i686.whl (434.0 kB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ i686

sabyenc3-5.1.3-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (146.4 kB view hashes)

Uploaded CPython 3.9 manylinux: glibc 2.17+ ARM64

sabyenc3-5.1.3-cp39-cp39-macosx_10_9_universal2.whl (82.7 kB view hashes)

Uploaded CPython 3.9 macOS 10.9+ universal2 (ARM64, x86-64)

sabyenc3-5.1.3-cp38-cp38-win_amd64.whl (37.6 kB view hashes)

Uploaded CPython 3.8 Windows x86-64

sabyenc3-5.1.3-cp38-cp38-win32.whl (33.0 kB view hashes)

Uploaded CPython 3.8 Windows x86

sabyenc3-5.1.3-cp38-cp38-musllinux_1_1_x86_64.whl (1.0 MB view hashes)

Uploaded CPython 3.8 musllinux: musl 1.1+ x86-64

sabyenc3-5.1.3-cp38-cp38-musllinux_1_1_i686.whl (922.3 kB view hashes)

Uploaded CPython 3.8 musllinux: musl 1.1+ i686

sabyenc3-5.1.3-cp38-cp38-musllinux_1_1_aarch64.whl (679.4 kB view hashes)

Uploaded CPython 3.8 musllinux: musl 1.1+ ARM64

sabyenc3-5.1.3-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (466.7 kB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ x86-64

sabyenc3-5.1.3-cp38-cp38-manylinux_2_17_i686.manylinux2014_i686.whl (434.6 kB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ i686

sabyenc3-5.1.3-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (147.0 kB view hashes)

Uploaded CPython 3.8 manylinux: glibc 2.17+ ARM64

sabyenc3-5.1.3-cp38-cp38-macosx_10_9_universal2.whl (82.7 kB view hashes)

Uploaded CPython 3.8 macOS 10.9+ universal2 (ARM64, x86-64)

sabyenc3-5.1.3-cp37-cp37m-win_amd64.whl (37.5 kB view hashes)

Uploaded CPython 3.7m Windows x86-64

sabyenc3-5.1.3-cp37-cp37m-win32.whl (32.9 kB view hashes)

Uploaded CPython 3.7m Windows x86

sabyenc3-5.1.3-cp37-cp37m-musllinux_1_1_x86_64.whl (1.0 MB view hashes)

Uploaded CPython 3.7m musllinux: musl 1.1+ x86-64

sabyenc3-5.1.3-cp37-cp37m-musllinux_1_1_i686.whl (923.0 kB view hashes)

Uploaded CPython 3.7m musllinux: musl 1.1+ i686

sabyenc3-5.1.3-cp37-cp37m-musllinux_1_1_aarch64.whl (680.0 kB view hashes)

Uploaded CPython 3.7m musllinux: musl 1.1+ ARM64

sabyenc3-5.1.3-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (467.6 kB view hashes)

Uploaded CPython 3.7m manylinux: glibc 2.17+ x86-64

sabyenc3-5.1.3-cp37-cp37m-manylinux_2_17_i686.manylinux2014_i686.whl (435.4 kB view hashes)

Uploaded CPython 3.7m manylinux: glibc 2.17+ i686

sabyenc3-5.1.3-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (147.6 kB view hashes)

Uploaded CPython 3.7m manylinux: glibc 2.17+ ARM64

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page