bool-to-int8-ray

bool to int8 serialization with ray.io

Installation

pip install bool-to-int8-ray

Usage

export B2I8_PCT_CPU=0.6

from bool_to_int8_ray import bool_to_int8_batch, int8_to_bool_batch
import numpy as np

# given lists of binary hashes
hashvalues = np.array([
    [1, 0, 1, 0, 1, 1, 0, 0],
    [1, 1, 0, 0, 0, 0, 0, 1]
])
# convert to list of int8 values
serialized = bool_to_int8_batch(hashvalues)
# convert back
deserialized = int8_to_bool_batch(serialized)

Speed Test

source .venv/bin/activate
export B2I8_PCT_CPU=0.6
python test/speedtest.py

Results

 4.776991 -- numpy version, hash to int8
 6.031101 -- numpy version, int8 to hash
 2.308446 -- ray.io version, hash to int8
 3.023865 -- ray.io version, int8 to hash

Appendix

Installation

The bool-to-int8-ray git repo is available as PyPi package

pip install bool-to-int8-ray
pip install git+ssh://git@github.com/ulf1/bool-to-int8-ray.git

Install a virtual environment

python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt --no-cache-dir
pip install -r requirements-dev.txt --no-cache-dir

(If your git repo is stored in a folder with whitespaces, then don't use the subfolder .venv. Use an absolute path without whitespaces.)

Python commands

• Jupyter for the examples: jupyter lab
• Check syntax: flake8 --ignore=F401 --exclude=$(grep -v '^#' .gitignore | xargs | sed -e 's/ /,/g')
• Run Unit Tests: PYTHONPATH=. pytest
• Run Speed Test: python test/speedtest.py

Publish

# pandoc README.md --from markdown --to rst -s -o README.rst
python setup.py sdist 
twine upload -r pypi dist/*

Clean up

find . -type f -name "*.pyc" | xargs rm
find . -type d -name "__pycache__" | xargs rm -r
rm -r .pytest_cache
rm -r .venv

Support

Please open an issue for support.

Contributing

Please contribute using Github Flow. Create a branch, add commits, and open a pull request.

Acknowledgements

The "Evidence" project was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) - 433249742 (GU 798/27-1; GE 1119/11-1).

Maintenance

• till 31.Aug.2023 (v0.1.1) the code repository was maintained within the DFG project 433249742
• since 01.Sep.2023 (v0.2.0) the code repository is maintained by @ulf1.