Python module for detecting password, api keys hashes and any other string that resembles a randomly generated character sequence.
Project description
stringlifier
String-classifier - is a python module for detecting random string and hashes text/code.
Typical usage scenarios include:
- Sanitizing application or security logs
- Detecting accidentally exposed credentials (complex passwords or api keys)
Interactive notebook
You can see Stringlifier in action by checking out this interactive notebook hosted on Colaboratory.
Quick start guide
You can quickly use stringlifier via pip-installation:
$ pip install stringlifier
In case you are using the pip3 installation that comes with Python3, use pip3 instead of pip in the above command.
$ pip3 install stringlifier
API example:
from stringlifier.api import Stringlifier
stringlifier=Stringlifier()
s = stringlifier("com.docker.hyperkit -A -u -F vms/0/hyperkit.pid -c 8 -m 8192M -b 127.0.0.1 --pass=\"NlcXVpYWRvcg\" -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-vpnkit,path=vpnkit.eth.sock,uuid=45172425-08d1-41ec-9d13-437481803412 -U c6fb5010-a83e-4f74-9a5a-50d9086b9")
After this, s
should be:
'com.docker.hyperkit -A -u -F vms/0/hyperkit.pid -c 8 -m 8192M -b <IP_ADDR> --pass="<RANDOM_STRING>" -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-vpnkit,path=vpnkit.eth.sock,uuid=<UUID> -U <UUID>'
You can also choose to see the full tokenization and classification output:
s, tokens = stringlifier("com.docker.hyperkit -A -u -F vms/0/hyperkit.pid -c 8 -m 8192M -b 127.0.0.1 --pass=\"NlcXVpYWRvcg\" -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-vpnkit,path=vpnkit.eth.sock,uuid=45172425-08d1-41ec-9d13-437481803412 -U c6fb5010-a83e-4f74-9a5a-50d9086b9", return_tokens=True)
s
will be the same as before and tokens
will contain the following data:
[[('0', 33, 34, '<NUMERIC>'),
('8', 51, 52, '<NUMERIC>'),
('8192', 56, 60, '<NUMERIC>'),
('127.0.0.1', 65, 74, '<IP_ADDR>'),
('NlcXVpYWRvcg', 83, 95, '<RANDOM_STRING>'),
('0', 100, 101, '<NUMERIC>'),
('0', 102, 103, '<NUMERIC>'),
('31', 118, 120, '<NUMERIC>'),
('1', 128, 129, '<NUMERIC>'),
('0', 130, 131, '<NUMERIC>'),
('45172425-08d1-41ec-9d13-437481803412', 172, 208, '<UUID>'),
('c6fb5010-a83e-4f74-9a5a-50d9086b9', 212, 244, '<UUID>')]]
Building your own classifier
You can also train your own model if you want to detect different types of strings. For this you can use the Command Line Interface for the string classifier:
$ python3 stringlifier/modules/stringc.py --help
Usage: stringc.py [options]
Options:
-h, --help show this help message and exit
--interactive
--train
--resume
--train-file=TRAIN_FILE
--dev-file=DEV_FILE
--store=OUTPUT_BASE
--patience=PATIENCE (default=20)
--batch-size=BATCH_SIZE
(default=32)
--device=DEVICE
For instructions on how to generate your training data, use this link.
Important note: This model might not scale if detecting a type of string depends on the surrounding tokens. In this case, you can look at a more advanced tool for sequence processing such as NLP-Cube
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file stringlifier39-0.1.1.8.tar.gz
.
File metadata
- Download URL: stringlifier39-0.1.1.8.tar.gz
- Upload date:
- Size: 2.0 MB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | e92bfbc1028dc0d7e81aaf092ae4ccf5cc46c8526c8ebf514e9f9584eca6fd79 |
|
MD5 | c12fbe0d48a56623c30bb845e4963f00 |
|
BLAKE2b-256 | 8071ee7383f6d9a5f9e9e648b3b359a480da4af71b21d804fea1bf029aebd6fe |
File details
Details for the file stringlifier39-0.1.1.8-py3-none-any.whl
.
File metadata
- Download URL: stringlifier39-0.1.1.8-py3-none-any.whl
- Upload date:
- Size: 2.0 MB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.9.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0cba30ec5212ab69601fa4ee82bda7c0c7d14ee4190e400f35fc1748e0e8f93f |
|
MD5 | 7bd3762172075a156790858384664dda |
|
BLAKE2b-256 | 6a663b1892b01e10384cce806dcb89e18bde137c8866b3798e1d27ead4032808 |