Skip to main content

yEnc decoding of usenet data using SIMD routines

Project description

SABYenc 3 - yEnc decoding of usenet data using SIMD routines

Modification of the original yenc module for use within SABnzbd. The module was extended to do header parsing and full yEnc decoding from a Python list of chunks, the way in which data is retrieved from usenet. This is particularly beneficial when SSL is enabled, which limits the size of each chunk to 16K. Parsing these chunks in python is much more costly. Additionally, this module releases Python's GIL during decoding, greatly increasing performance of the overall download process.

Further improved by using yencode from animetosho, which utilizes x86/ARM SIMD optimised routines if such CPU features are available.

Installing

As simple as running:

pip install sabyenc3 --upgrade

When you want to compile from sources, you can run in the sabyenc directory:

pip install .

SIMD detection

To see which SIMD set was detected on your system, run:

python -c "import sabyenc3; print(sabyenc3.simd);"

Testing

For testing we use pytest (install via pip install -r tests/requirements.txt) and test can simply be executed by browsing to the sabyenc directory and running:

pytest

Note that tests can fail if git modified the line endings of data files when checking out the repository!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sabyenc3-5.4.1.tar.gz (227.5 kB view hashes)

Uploaded source

Built Distributions

sabyenc3-5.4.1-cp310-cp310-win_amd64.whl (40.0 kB view hashes)

Uploaded cp310

sabyenc3-5.4.1-cp310-cp310-win32.whl (35.4 kB view hashes)

Uploaded cp310

sabyenc3-5.4.1-cp39-cp39-win_amd64.whl (40.0 kB view hashes)

Uploaded cp39

sabyenc3-5.4.1-cp39-cp39-win32.whl (35.4 kB view hashes)

Uploaded cp39

sabyenc3-5.4.1-cp38-cp38-win_amd64.whl (40.0 kB view hashes)

Uploaded cp38

sabyenc3-5.4.1-cp38-cp38-win32.whl (35.4 kB view hashes)

Uploaded cp38

sabyenc3-5.4.1-cp37-cp37m-win_amd64.whl (39.9 kB view hashes)

Uploaded cp37

sabyenc3-5.4.1-cp37-cp37m-win32.whl (35.4 kB view hashes)

Uploaded cp37

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page