A package that is able to constantly write encrypted bytes to disk
GnuPG Eternal - Large File Encryption with Line Modification
GnuPG Eternal enables developers and data scientists to encrypt and decrypt large files while modifying them in memory. It is specifically designed for de-tokenization + encryption of sensitive data such as account numbers and social insurance numbers, where raw data is not allowed to live on the local storage of the system.
One primary example is for transferring and processing PANs. Provided is a file of customers' PANs that are tokenized. The requirement is to send the encrypted and de-tokenized PANs to the receiver.
However, because PANs are highly sensitive, the de-tokenized/raw PANs cannot touch the local storage of the system. Hence, de-tokenization and encryption need to happen in system memory.
Traditionally, the content of the whole file can be loaded into memory if wanted, and the developer/data scientist can manipulate the data depending on his/her needs.
As the file size increases to a certain point where the system cannot hold it any more, the program would crash.
GnuPG Eternal allows the developer/data scientist to manipulate and encrypt files regardless of their sizes: that is, the maximum space consumption is constant, provided that the file is not one massive line of data.
Install via pip:
pip install gpgeternal
With the file to manipulate on the local storage, define the path to the file and the path to the processed/manipulated file. The file to process must exist.
Define the recipient of the encrypted file and the manipulation method for each line (ex. de-tokenization method that returns the de-tokenized line).
import gpgeternal def manipulation_func(line): # The example manipulation method line = line.replace('Foobar', 'cipher') return line def main(): file_to_encrypt = 'test.txt' # File to manipulate and encrypt recipient = 'email@example.com' # Email of the recipient (GnuPG) output_file = 'test.pgp' # Name of the encrypted & manipulated file gpgeternal.encrypt_large_file(recipient, file_to_encrypt, output_file, manipulation_func) if __name__ == '__main__': main()
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size gpgeternal-0.9.tar.gz (3.1 kB)||File type Source||Python version None||Upload date||Hashes View hashes|