rencrypt 0.3.23

A Python encryption library implemented in Rust. It supports AEAD with AES-GCM and ChaCha20Poly1305. It uses ring crate to handle encryption.

rencrypt

[!WARNING] This crate hasn't been audited, but it mostly wraps ring crate which is a well known audited library, so in principle at least the primitives should offer a similar level of security.
This is still under development. Please do not use it with sensitive data just yet. Please wait for a stable release and maybe an audit.
It's mostly ideal for experimental and learning projects.

A Python encryption library implemented in Rust. It supports AEAD with AES-GCM and ChaCha20Poly1305. It uses ring to handle encryption.
If offers slightly higher speed compared to other Python libs, especially for small chunks of data. The API also tries to be easy to use but it's more optimized for speed than usability.

So if you want to achieve the highest possible encryption speed, consider giving it a try.

Benchmark

Some benchmarks comparing it to PyFLocker which from my benchmarks is the fastest among other Python libs like cryptography, NaCl (libsodium), PyCryptodome

Buffer in memory

This is useful when you keep a buffer, set your plaintext/ciphertext in there, and then encrypt/decrypt in-place in that buffer. This is the most performant way to use it, because it does't copy any bytes nor allocate new memory.
rencrypt is faster on small buffers, less than few MB, PyFLocker is comming closer for larger buffers.

Encrypt seconds

Decrypt seconds

Block size and duration in seconds

MB	rencrypt encrypt	PyFLocker encrypt	rencrypt decrypt	PyFLocker decrypt
0.03125	0.00001	0.00091	0.00001	0.00004
0.0625	0.00001	0.00005	0.00001	0.00004
0.125	0.00002	0.00005	0.00003	0.00005
0.25	0.00004	0.00008	0.00005	0.00009
0.5	0.00010	0.00014	0.00011	0.00015
1	0.00021	0.00024	0.00021	0.00029
2	0.00043	0.00052	0.00044	0.00058
4	0.00089	0.00098	0.00089	0.00117
8	0.00184	0.00190	0.00192	0.00323
16	0.00353	0.00393	0.00367	0.00617
32	0.00678	0.00748	0.00749	0.01348
64	0.01361	0.01461	0.01460	0.02697
128	0.02923	0.03027	0.03134	0.05410
256	0.06348	0.06188	0.06136	0.10417
512	0.11782	0.13463	0.12090	0.21114
1024	0.25001	0.24953	0.25377	0.42581

File

Encrypt seconds

Decrypt seconds

File size and duration in seconds

MB	rencrypt encrypt	PyFLocker encrypt	rencrypt decrypt	PyFLocker decrypt
0.031251	0.00010	0.00280	0.00004	0.00479
0.062501	0.00009	0.00218	0.00005	0.00143
0.125	0.00020	0.00212	0.00014	0.00129
0.25	0.00034	0.00232	0.00020	0.00165
0.5	0.00050	0.00232	0.00035	0.00181
1	0.00087	0.00356	0.00065	0.00248
2	0.00215	0.00484	0.00154	0.00363
4	0.00361	0.00765	0.00301	0.00736
8	0.00688	0.01190	0.00621	0.00876
16	0.01503	0.02097	0.01202	0.01583
32	0.02924	0.03642	0.02563	0.02959
64	0.05737	0.06473	0.04431	0.05287
128	0.11098	0.12646	0.08944	0.09926
256	0.22964	0.24716	0.17402	0.19420
512	0.43506	0.46444	0.38143	0.38242
1024	0.97147	0.95803	0.78137	0.87363
2048	2.07143	2.10766	1.69471	2.99210
4096	4.85395	4.69722	5.40580	8.73779
8192	10.76984	11.76741	10.29253	21.00636
16384	21.84490	26.27385	39.56230	43.55530

Usage

There are two ways in which you can use the lib, the first one is a bit faster, the second offers a bit more flexible way to use it sacrificing a bit of performance.

1. With a buffer in memory: using encrypt()/decrypt(), is useful when you keep a buffer (or have it from somewhere), set your plaintext/ciphertext in there, and then encrypt/decrypt in-place in that buffer. This is the most performant way to use it, because it does't copy any bytes nor allocate new memory.
   The buffer has to be a numpy array, so that it's easier for you to collect data with slices that reference to underlying data. This is because the whole buffer needs to be the size of ciphertext (which is plaintext_len + tag_len + nonce_len) but you may pass a slice of the buffer to a BufferedReader to read_into() the plaintext.
   If you can directly collect the data to that buffer, like BufferedReader.read_into(), this is the preffered way to go.
2. From some bytes into the buffer: using encrypt_into()/decrypt_into(), when you have some arbitrary data that you want to work with. It will first copy those bytes to the buffer then do the operation in-place in the buffer. This is a bit slower, especially for large data, because it first needs to copy the bytes to the buffer.

Encryption provider

You will notice in the examples we initiate the Cipher from something like this cipher_meta = CipherMeta.Ring(RingAlgorithm.AES256GCM). The first part CipherMeta.Ring is the encryption provider, for now it's only one but in the future we will add more. The last part is the algorithm for that provider, in this case AES256GCM. Each provier might expose specific algorithms.

Security

For security reasons it's a good practice to lock the memory with mlock() in which you keep sensitive data like passwords or encrryption keys, or any other plaintext sensitive content. Also it's important to zeroize the data when not used anymore.
In the case of Copy-on-write fork you need to zeroize the memory before forking the child process.
In the examples below you will see how to do it.

Examples

You can see more in examples directory and in bench.py which has some benchmarks. Here are few simple examples:

Encrypt and decrypt with a buffer in memory

encrypt()/decrypt()

This is the most performant way to use it as it will not copy bytes to the buffer nor allocate new memory for plaintext and ciphertext.

# This is the most performant way to use it as it will not copy bytes to the buffer nor allocate new memory for plaintext and ciphertext.

from rencrypt import Cipher, CipherMeta, RingAlgorithm
import os
from zeroize import zeroize1, mlock, munlock
import numpy as np


if __name__ == "__main__":
    try:
        # You can use also other algorithms like cipher_meta = CipherMeta.Ring(RingAlgorithm.ChaCha20Poly1305)`.
        cipher_meta = CipherMeta.Ring(RingAlgorithm.AES256GCM)
        key_len = cipher_meta.key_len()
        key = bytearray(key_len)
        # for security reasons we lock the memory of the key so it won't be swapped to disk
        mlock(key)
        cipher_meta.generate_key(key)
        # The key is copied and the input key is zeroized for security reasons.
        # The copied key will also be zeroized when the object is dropped.
        cipher = Cipher(cipher_meta, key)
        # it was zeroized we can unlock it
        munlock(key)

        # we create a buffer based on plaintext block len of 4096
        # the actual buffer needs to be a bit larger as the ciphertext also includes the tag and nonce
        plaintext_len = 4096
        ciphertext_len = cipher.ciphertext_len(plaintext_len)
        buf = np.array([0] * ciphertext_len, dtype=np.uint8)
        # for security reasons we lock the memory of the buffer so it won't be swapped to disk, because it contains plaintext after decryption
        mlock(buf)

        aad = b"AAD"

        # put some plaintext in the buffer, it would be ideal if you can directly collect the data into the buffer without allocating new memory
        # but for the sake of example we will allocate and copy the data
        plaintext = bytearray(os.urandom(plaintext_len))
        # for security reasons we lock the memory of the plaintext so it won't be swapped to disk
        mlock(plaintext)
        # cipher.copy_slice is slighlty faster than buf[:plaintext_len] = plaintext, especially for large plaintext, because it copies the data in parallel
        # cipher.copy_slice takes bytes as input, cipher.copy_slice1 takes bytearray
        cipher.copy_slice(plaintext, buf)
        # encrypt it, this will encrypt in-place the data in the buffer
        print("encryping...")
        ciphertext_len = cipher.encrypt(buf, plaintext_len, 42, aad)
        cipertext = buf[:ciphertext_len]
        # you can do something with the ciphertext

        # decrypt it
        # if you need to copy ciphertext to buffer, we don't need to do it now as it's already in the buffer
        # cipher.copy_slice(ciphertext, buf[:len(ciphertext)])
        print("decryping...")
        plaintext_len = cipher.decrypt(buf, ciphertext_len, 42, aad)
        plaintext2 = buf[:plaintext_len]
        # for security reasons we lock the memory of the plaintext so it won't be swapped to disk
        mlock(plaintext2)
        assert plaintext == plaintext2

    finally:
        # best practice, you should always zeroize the plaintext and keys after you are done with it (key will be zeroized when the enc object is dropped)
        zeroize1(plaintext)
        zeroize1(buf)

        munlock(buf)
        munlock(plaintext)

        print("bye!")

You can use other ciphers like cipher = Cipher.ChaCha20Poly1305.

Encrypt and decrypt a file

import errno
import io
import os
from pathlib import Path
import shutil
from rencrypt import Cipher, CipherMeta, RingAlgorithm
import hashlib
from zeroize import zeroize1, mlock, munlock
import numpy as np


def read_file_in_chunks(file_path, buf):
    with open(file_path, "rb") as file:
        buffered_reader = io.BufferedReader(file, buffer_size=len(buf))
        while True:
            read = buffered_reader.readinto(buf)
            if read == 0:
                break
            yield read


def hash_file(file_path):
    hash_algo = hashlib.sha256()
    with open(file_path, "rb") as f:
        for chunk in iter(lambda: f.read(4096), b""):
            hash_algo.update(chunk)
    return hash_algo.hexdigest()


def compare_files_by_hash(file1, file2):
    return hash_file(file1) == hash_file(file2)


def silentremove(filename):
    try:
        os.remove(filename)
    except OSError as e:  # this would be "except OSError, e:" before Python 2.6
        if e.errno != errno.ENOENT:  # errno.ENOENT = no such file or directory
            raise  # re-raise exception if a different error occurred


def create_directory_in_home(dir_name):
    # Get the user's home directory
    home_dir = Path.home()

    # Create the full path for the new directory
    new_dir_path = home_dir / dir_name

    # Create the directory
    try:
        new_dir_path.mkdir(parents=True, exist_ok=True)
    except Exception as e:
        print(f"Error creating directory: {e}")

    return new_dir_path.absolute().__str__()


def create_file_with_size(file_path_str, size_in_bytes):
    with open(file_path_str, "wb") as f:
        for _ in range((size_in_bytes // 4096) + 1):
            f.write(os.urandom(min(4096, size_in_bytes)))
        f.flush()


def delete_dir(path):
    if os.path.exists(path):
        shutil.rmtree(path)
    else:
        print(f"Directory {path} does not exist.")


if __name__ == "__main__":
    try:
        tmp_dir = create_directory_in_home("Cipher_tmp")
        fin = tmp_dir + "/" + "fin"
        fout = tmp_dir + "/" + "fout.enc"
        create_file_with_size(fin, 10 * 1024 * 1024)

        chunk_len = 256 * 1024

        # You can use also other algorithms like cipher_meta = CipherMeta.Ring(RingAlgorithm.ChaCha20Poly1305)`.
        cipher_meta = CipherMeta.Ring(RingAlgorithm.AES256GCM)
        key_len = cipher_meta.key_len()
        key = bytearray(key_len)
        # for security reasons we lock the memory of the key so it won't be swapped to disk
        mlock(key)
        cipher_meta.generate_key(key)
        # The key is copied and the input key is zeroized for security reasons.
        # The copied key will also be zeroized when the object is dropped.
        cipher = Cipher(cipher_meta, key)
        # it was zeroized we can unlock it
        munlock(key)

        plaintext_len = chunk_len
        ciphertext_len = cipher.ciphertext_len(plaintext_len)
        buf = np.array([0] * ciphertext_len, dtype=np.uint8)
        mlock(buf)

        aad = b"AAD"

        # encrypt
        print("encryping...")
        with open(fout, "wb", buffering=plaintext_len) as file_out:
            i = 0
            for read in read_file_in_chunks(fin, buf[:plaintext_len]):
                ciphertext_len = cipher.encrypt(buf, read, i, aad)
                file_out.write(buf[:ciphertext_len])
                i += 1
            file_out.flush()

        # decrypt
        print("decryping...")
        tmp = fout + ".dec"
        with open(tmp, "wb", buffering=plaintext_len) as file_out:
            i = 0
            for read in read_file_in_chunks(fout, buf):
                plaintext_len2 = cipher.decrypt(buf, read, i, aad)
                file_out.write(buf[:plaintext_len2])
                i += 1
            file_out.flush()

        assert compare_files_by_hash(fin, tmp)

        delete_dir(tmp_dir)

    finally:
        # best practice, you should always zeroize the plaintext and keys after you are done with it (key will be zeroized when the enc object is dropped)
        # buf will containt the last block plaintext so we need to zeroize it
        zeroize1(buf)

        munlock(buf)

    print("bye!")

Encrypt and decrypt from some bytes into the buffer

encrypt_from()/decrypt_from()

This is a bit slower than handling data only via the buffer, especially for large plaintext, but there are situations when you can't directly collect the data to the buffer but have some data from somewhere else.

# This is a bit slower than handling data only via the buffer, especially for large plaintext,
# but there are situations when you can't directly collect the data to the buffer but have some bytes from somewhere else.

from rencrypt import Cipher, CipherMeta, RingAlgorithm
import os
from zeroize import zeroize1, mlock, munlock
import numpy as np


if __name__ == "__main__":
    try:
        # You can use also other algorithms like cipher_meta = CipherMeta.Ring(RingAlgorithm.ChaCha20Poly1305)`.
        cipher_meta = CipherMeta.Ring(RingAlgorithm.AES256GCM)
        key_len = cipher_meta.key_len()
        key = bytearray(key_len)
        # for security reasons we lock the memory of the key so it won't be swapped to disk
        mlock(key)
        cipher_meta.generate_key(key)
        # The key is copied and the input key is zeroized for security reasons.
        # The copied key will also be zeroized when the object is dropped.
        cipher = Cipher(cipher_meta, key)
        # it was zeroized we can unlock it
        munlock(key)

        # we create a buffer based on plaintext block len of 4096
        # the actual buffer needs to be a bit larger as the ciphertext also includes the tag and nonce
        plaintext_len = 4096
        ciphertext_len = cipher.ciphertext_len(plaintext_len)
        buf = np.array([0] * ciphertext_len, dtype=np.uint8)
        # for security reasons we lock the memory of the buffer so it won't be swapped to disk, because it contains plaintext after decryption
        mlock(buf)

        aad = b"AAD"

        plaintext = bytearray(os.urandom(plaintext_len))
        # for security reasons we lock the memory of the plaintext so it won't be swapped to disk
        mlock(plaintext)

        # encrypt it, after this will have the ciphertext in the buffer
        print("encryping...")
        ciphertext_len = cipher.encrypt_from(plaintext, buf, 42, aad)
        cipertext = bytes(buf[:ciphertext_len])

        # decrypt it
        print("decryping...")
        plaintext_len = cipher.decrypt_from(cipertext, buf, 42, aad)
        plaintext2 = buf[:plaintext_len]
        # for security reasons we lock the memory of the plaintext so it won't be swapped to disk
        mlock(plaintext2)
        assert plaintext == plaintext2

    finally:
        # best practice, you should always zeroize the plaintext and keys after you are done with it (key will be zeroized when the enc object is dropped)
        zeroize1(plaintext)
        zeroize1(buf)

        munlock(buf)
        munlock(plaintext)

        print("bye!")

Zeroing memory before forking child process

This mitigates the problems that appears on Copy-on-write fork. You need to zeroize the data before forking the child process.

""" In the case of [Copy-on-write fork](https://en.wikipedia.org/wiki/Copy-on-write) you need to zeroize the memory before forking the child process. """

import os
from zeroize import zeroize1, mlock, munlock


if __name__ == "__main__":
    try:
        # Maximum you can mlock is 4MB
        sensitive_data = bytearray(b"Sensitive Information")
        mlock(sensitive_data)

        print("Before zeroization:", sensitive_data)

        zeroize1(sensitive_data)
        print("After zeroization:", sensitive_data)

        # Forking after zeroization to ensure no sensitive data is copied
        pid = os.fork()
        if pid == 0:
            # This is the child process
            print("Child process memory after fork:", sensitive_data)
        else:
            # This is the parent process
            os.wait()  # Wait for the child process to exit
        
        print("all good, bye!")

    finally:
        # Unlock the memory
        print("unlocking memory")
        munlock(sensitive_data)

Build from source

Browser

Geting sources from GitHub

Skip this if you're starting it in browser.

git clone https://github.com/radumarias/rencrypt-python && cd Cipher-python

Compile and run

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

To configure your current shell, you need to source the corresponding env file under $HOME/.cargo. This is usually done by running one of the following (note the leading DOT):

. "$HOME/.cargo/env"

python -m venv .env
source .env/bin/activate
pip install -r requirements.txt
maturin develop --release
pytest
python examples/encrypt.py
python examples/encrypt_into.py
python examples/encrypt_from.py
python examples/encrypt_file.py
python benches/bench.py

More benchmarks

Different ways to use the lib

Encrypt

Decrypt

Block size and duration in seconds

MB	rencrypt encrypt	PyFLocker encrypt update_into	rencrypt encrypt_from	PyFLocker encrypt update	rencrypt decrypt	PyFLocker decrypt update_into	rencrypt decrypt_from	PyFLocker decrypt update
0.03125	0.00001	0.00091	0.00001	0.00009	0.00001	0.00004	0.00001	0.00005
0.0625	0.00001	0.00005	0.00002	0.00005	0.00001	0.00004	0.00002	0.00008
0.125	0.00002	0.00005	0.00003	0.00011	0.00003	0.00005	0.00003	0.00013
0.25	0.00004	0.00008	0.00007	0.00019	0.00005	0.00009	0.00007	0.00023
0.5	0.0001	0.00014	0.00015	0.00035	0.00011	0.00015	0.00014	0.00043
1	0.00021	0.00024	0.0008	0.00082	0.00021	0.00029	0.00044	0.00103
2	0.00043	0.00052	0.00082	0.00147	0.00044	0.00058	0.00089	0.00176
4	0.00089	0.00098	0.00174	0.00284	0.00089	0.00117	0.0013	0.0034
8	0.00184	0.0019	0.00263	0.00523	0.00192	0.00323	0.00283	0.00571
16	0.00353	0.00393	0.00476	0.0141	0.00367	0.00617	0.00509	0.01031
32	0.00678	0.00748	0.00904	0.0244	0.00749	0.01348	0.01014	0.02543
64	0.01361	0.01461	0.01595	0.05064	0.0146	0.02697	0.0192	0.05296
128	0.02923	0.03027	0.03343	0.10362	0.03134	0.0541	0.03558	0.1138
256	0.06348	0.06188	0.07303	0.20911	0.06136	0.10417	0.07572	0.20828
512	0.11782	0.13463	0.14283	0.41929	0.1209	0.21114	0.14434	0.41463
1024	0.25001	0.24953	0.28912	0.8237	0.25377	0.42581	0.29795	0.82588

Speed throughput

256KB seems to be the sweet spot for buffer size that offers the max MB/s speed for encryption, on benchmarks that seem to be the case. We performed 10.000 encryption operations for each size varying from 1KB to 16MB.

MB	MB/s
0.0009765625	1083
0.001953125	1580
0.00390625	2158
0.0078125	2873
0.015625	3348
0.03125	3731
0.0625	4053
0.125	4156
0.25	4247
0.5	4182
1.0	3490
2.0	3539
4.0	3684
8.0	3787
16.0	3924

For the future

• Add more AES ciphers like AES128-GCM and AES-GCM-SIV
• Add other encryption providers like RustCrypto and libsodium
• Maybe add support for RSA and Elliptic-curve cryptography
• Saving and loading keys from file

Considerations

This lib hasn't been audited, but it mostly wraps ring crate which is a well known library, so in principle it should offer as similar level of security.

Contribute

Feel free to fork it, change and use it in any way that you want. If you build something interesting and feel like sharing pull requests are always appreciated.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache License, shall be dual-licensed as above, without any additional terms or conditions.

How to contribute

1. Fork the repo
2. Make the changes in your fork
3. Add tests for you changes, if applicable
4. cargo build --all --all-features and fix any issues
5. cargo fmt --all, you can cnofigure your IDE to do this on save RustRover and VSCode
6. cargo check --all --all-features and fix any errors and warnings
7. cargo clippy --all --all-features -- -D warnings and fix any errors
8. cargo test --all --all-features and fix any issues
9. cargo bench --all --all-features and fix any issues
10. Create a PR
11. Monitor the checks (GitHub actions runned)
12. Respond to any comments
13. In the end ideally it will be merged to main