Skip to main content

Python module for detecting password, api keys hashes and any other string that resembles a randomly generated character sequence.

Project description

Downloads Downloads Weekly daily Version Python 3 GitHub stars

stringlifier

String-classifier - is a python module for detecting random string and hashes text/code.

Typical usage scenarios include:

  • Sanitizing application or security logs
  • Detecting accidentally exposed credentials (complex passwords or api keys)

Interactive notebook

You can see Stringlifier in action by checking out this interactive notebook hosted on Colaboratory.

Quick start guide

You can quickly use stringlifier via pip-installation:

$ pip install stringlifier

In case you are using the pip3 installation that comes with Python3, use pip3 instead of pip in the above command.

$ pip3 install stringlifier

API example:

from stringlifier.api import Stringlifier

stringlifier=Stringlifier()

s = stringlifier("com.docker.hyperkit -A -u -F vms/0/hyperkit.pid -c 8 -m 8192M -b 127.0.0.1 --pass=\"NlcXVpYWRvcg\" -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-vpnkit,path=vpnkit.eth.sock,uuid=45172425-08d1-41ec-9d13-437481803412 -U c6fb5010-a83e-4f74-9a5a-50d9086b9")

After this, s should be:

'com.docker.hyperkit -A -u -F vms/0/hyperkit.pid -c 8 -m 8192M -b <IP_ADDR> --pass="<RANDOM_STRING>" -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-vpnkit,path=vpnkit.eth.sock,uuid=<UUID> -U <UUID>'

You can also choose to see the full tokenization and classification output:

s, tokens = stringlifier("com.docker.hyperkit -A -u -F vms/0/hyperkit.pid -c 8 -m 8192M -b 127.0.0.1 --pass=\"NlcXVpYWRvcg\" -s 0:0,hostbridge -s 31,lpc -s 1:0,virtio-vpnkit,path=vpnkit.eth.sock,uuid=45172425-08d1-41ec-9d13-437481803412 -U c6fb5010-a83e-4f74-9a5a-50d9086b9", return_tokens=True)

s will be the same as before and tokens will contain the following data:

[[('0', 33, 34, '<NUMERIC>'),
   ('8', 51, 52, '<NUMERIC>'),
   ('8192', 56, 60, '<NUMERIC>'),
   ('127.0.0.1', 65, 74, '<IP_ADDR>'),
   ('NlcXVpYWRvcg', 83, 95, '<RANDOM_STRING>'),
   ('0', 100, 101, '<NUMERIC>'),
   ('0', 102, 103, '<NUMERIC>'),
   ('31', 118, 120, '<NUMERIC>'),
   ('1', 128, 129, '<NUMERIC>'),
   ('0', 130, 131, '<NUMERIC>'),
   ('45172425-08d1-41ec-9d13-437481803412', 172, 208, '<UUID>'),
   ('c6fb5010-a83e-4f74-9a5a-50d9086b9', 212, 244, '<UUID>')]]

Building your own classifier

You can also train your own model if you want to detect different types of strings. For this you can use the Command Line Interface for the string classifier:

$ python3 stringlifier/modules/stringc.py --help

Usage: stringc.py [options]

Options:
  -h, --help            show this help message and exit
  --interactive
  --train
  --resume
  --train-file=TRAIN_FILE
  --dev-file=DEV_FILE
  --store=OUTPUT_BASE
  --patience=PATIENCE   (default=20)
  --batch-size=BATCH_SIZE
                        (default=32)
  --device=DEVICE

For instructions on how to generate your training data, use this link.

Important note: This model might not scale if detecting a type of string depends on the surrounding tokens. In this case, you can look at a more advanced tool for sequence processing such as NLP-Cube

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

stringlifier39-0.1.1.8.tar.gz (2.0 MB view details)

Uploaded Source

Built Distribution

stringlifier39-0.1.1.8-py3-none-any.whl (2.0 MB view details)

Uploaded Python 3

File details

Details for the file stringlifier39-0.1.1.8.tar.gz.

File metadata

  • Download URL: stringlifier39-0.1.1.8.tar.gz
  • Upload date:
  • Size: 2.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.6

File hashes

Hashes for stringlifier39-0.1.1.8.tar.gz
Algorithm Hash digest
SHA256 e92bfbc1028dc0d7e81aaf092ae4ccf5cc46c8526c8ebf514e9f9584eca6fd79
MD5 c12fbe0d48a56623c30bb845e4963f00
BLAKE2b-256 8071ee7383f6d9a5f9e9e648b3b359a480da4af71b21d804fea1bf029aebd6fe

See more details on using hashes here.

File details

Details for the file stringlifier39-0.1.1.8-py3-none-any.whl.

File metadata

File hashes

Hashes for stringlifier39-0.1.1.8-py3-none-any.whl
Algorithm Hash digest
SHA256 0cba30ec5212ab69601fa4ee82bda7c0c7d14ee4190e400f35fc1748e0e8f93f
MD5 7bd3762172075a156790858384664dda
BLAKE2b-256 6a663b1892b01e10384cce806dcb89e18bde137c8866b3798e1d27ead4032808

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page