Parse a list of indicators into a dictionary or JSON structure for programmatic use.

Navigation

Verified details

These details have been verified by PyPI
Maintainers
Avatar for bonifield_tools from gravatar.com bonifield_tools

Unverified details

These details have not been verified by PyPI
Meta
  • Requires: Python >=3.11
Report project as malware

Project description

ParseIOC

Parse a list of indicators into a dictionary or JSON structure for programmatic use.

Converts a plain list of indicators into a dictionary of:

  • emails and their domains
  • URL domains, and username+password and ports if present
  • ipv4 and ipv6 addresses and networks
  • md5, sha256, and sha512 hashes

...with the goal of mapping this dictionary to fields in a SIEM or similar database.

This enables programatic use of an IOC file to quickly query specific values in a SIEM, such as source.ip or file.hash.sha256.

This tool will support more formats (imphash, ssdeep, ja3+) "eventually".

Otherwise-unidentifiable strings are categorized as domains.

Setup

uv add parse_ioc
# or
pip3 install parse_ioc

Usage

from parse_ioc import ParseIOC, map_fields, parse_multi

Categorize a single indicator

i = ParseIOC("https://192.168.20.20/bad.txt")
i.to_dict
i.to_json

Categorize a list or file of indicators

setting mode="single" will produce the same results as calling ParseIOC(ioc) by itself

p = parse_multi(ioc_list, mode="combined")
# or
p = parse_multi("ioc_examples.txt", mode="combined")

print(json.dumps(p, indent=4))

Map a list or file of indicators to a TOML of SIEM fields

m = map_fields("ioc_examples.txt", "map_ecs.toml")
print(json.dumps(m, indent=4))

Output from running parse_ioc.py directly

======================== categorize a single indicator =========================
.to_dict: <class 'dict'> {'ioc': '192.168.20.20', 'ioc_type': 'ipv4'}
.to_json: <class 'str'> {"ioc": "192.168.20.20", "ioc_type": "ipv4"}
============ parse a list or file of IOCs into a combined structure ============
{
    "email": [
        "bob@email.local"
    ],
    "domain": [
        "securewebsite.local",
        "email.local",
        "anotherwebsite.local",
        "n--nhk-u63b1cko2lyc6jrwxgom6k.com",
        "website.local"
    ],
    "port": [
        9443,
        8443
    ],
    "credentials": [
        "username:password"
    ],
    "ipv4": [
        "192.168.1.1",
        "192.168.20.20"
    ],
    "ipv4_network": [
        "192.168.1.0/24"
    ]
}
{
    "email": [
        "bad.username@subdomain.bad.local",
        "bob@email.local"
    ],
    "domain": [
        "securewebsite.local",
        "email.local",
        "bob documents.pdf",
        "file.txt",
        "bad.local",
        "not-an-ioc",
        "anotherwebsite.local",
        "n--nhk-u63b1cko2lyc6jrwxgom6k.com",
        "subdomain.bad.local",
        "website.local",
        "aaaaaaaaaaaaaaaa"
    ],
    "ipv4": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4"
    ],
    "port": [
        9443,
        8443
    ],
    "credentials": [
        "username:password"
    ],
    "ipv4_network": [
        "8.8.8.0/24",
        "192.168.1.0/24"
    ],
    "ipv6": [
        "fe80::"
    ],
    "file_path_linux": [
        "/home/bob/file.txt"
    ],
    "unknown": [
        "rc:\\users\\bob smith\\desktop\\file.txt:alt.exe"
    ],
    "file_path_windows": [
        "c:\\users\\bob\\desktop\\test.txt",
        "c:/users/bob smith/desktop/file.txt:alt.exe",
        "c:\\users\\bob smith\\desktop\\file.txt:alt.exe",
        "file.txt:alt.exe",
        "c:\\users\\bob smith\\desktop\\file.txt",
        "c:\\users\\bob smith\\desktop\\file2.txt:alt.exe"
    ],
    "md5": [
        "6cd3556deb0da54bca060b4c39479839"
    ],
    "sha256": [
        "315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3"
    ],
    "sha512": [
        "c1527cd893c124773d811911970c8fe6e857d6df5dc9226bd8a160614c0cd963a4ddea2b94bb7d36021ef9d865d5cea294a82dd49a0bb269f51f6e7a57f79421"
    ]
}
=== yield dictionaries from a list or file, instead of a combined structure ====
{'ioc': 'bob@email.local', 'ioc_type': 'email', 'extra': [{'ioc': 'email.local', 'ioc_type': 'domain'}]}
{'ioc': 'bad.username@subdomain.bad.local', 'ioc_type': 'email', 'extra': [{'ioc': 'subdomain.bad.local', 'ioc_type': 'domain'}]}
{'ioc': '8.8.8.8', 'ioc_type': 'ipv4'}
{'ioc': '1.2.3.4', 'ioc_type': 'ipv4'}
{'ioc': 'website.local', 'ioc_type': 'domain'}
{'ioc': 'anotherwebsite.local', 'ioc_type': 'domain', 'extra': [{'ioc': 9443, 'ioc_type': 'port'}]}
{'ioc': 'securewebsite.local', 'ioc_type': 'domain', 'extra': [{'ioc': 'username:password', 'ioc_type': 'credentials'}, {'ioc': 8443, 'ioc_type': 'port'}]}
{'ioc': '192.168.1.1', 'ioc_type': 'ipv4'}
{'ioc': '192.168.1.0/24', 'ioc_type': 'ipv4_network'}
{'ioc': '8.8.8.0/24', 'ioc_type': 'ipv4_network'}
{'ioc': '192.168.20.20', 'ioc_type': 'ipv4'}
{'ioc': 'fe80::', 'ioc_type': 'ipv6'}
{'ioc': 'n--nhk-u63b1cko2lyc6jrwxgom6k.com', 'ioc_type': 'domain'}
{'ioc': 'bad.local', 'ioc_type': 'domain'}
{'ioc': '/home/bob/file.txt', 'ioc_type': 'file_path_linux'}
{'ioc': 'rc:\\users\\bob smith\\desktop\\file.txt:alt.exe', 'ioc_type': 'unknown'}
{'ioc': 'file.txt', 'ioc_type': 'domain'}
{'ioc': 'file.txt:alt.exe', 'ioc_type': 'file_path_windows'}
{'ioc': 'bob documents.pdf', 'ioc_type': 'domain'}
{'ioc': '/home/bob/file.txt', 'ioc_type': 'file_path_linux'}
{'ioc': 'c:\\users\\bob\\desktop\\test.txt', 'ioc_type': 'file_path_windows'}
{'ioc': 'c:\\users\\bob smith\\desktop\\file.txt', 'ioc_type': 'file_path_windows'}
{'ioc': 'c:\\users\\bob smith\\desktop\\file.txt', 'ioc_type': 'file_path_windows'}
{'ioc': 'c:\\users\\bob smith\\desktop\\file.txt:alt.exe', 'ioc_type': 'file_path_windows'}
{'ioc': 'c:\\users\\bob smith\\desktop\\file.txt:alt.exe', 'ioc_type': 'file_path_windows'}
{'ioc': 'c:\\users\\bob smith\\desktop\\file.txt:alt.exe', 'ioc_type': 'file_path_windows'}
{'ioc': 'c:\\users\\bob smith\\desktop\\file2.txt:alt.exe', 'ioc_type': 'file_path_windows'}
{'ioc': 'c:/users/bob smith/desktop/file.txt:alt.exe', 'ioc_type': 'file_path_windows'}
{'ioc': 'not-an-ioc', 'ioc_type': 'domain'}
{'ioc': 'aaaaaaaaaaaaaaaa', 'ioc_type': 'domain'}
{'ioc': '6cd3556deb0da54bca060b4c39479839', 'ioc_type': 'md5'}
{'ioc': '315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3', 'ioc_type': 'sha256'}
{'ioc': 'c1527cd893c124773d811911970c8fe6e857d6df5dc9226bd8a160614c0cd963a4ddea2b94bb7d36021ef9d865d5cea294a82dd49a0bb269f51f6e7a57f79421', 'ioc_type': 'sha512'}
======== provide IOC (file or list) and TOML config (path) map_fields() ========
{
    "email.from.address": [
        "bad.username@subdomain.bad.local",
        "bob@email.local"
    ],
    "email.sender.address": [
        "bad.username@subdomain.bad.local",
        "bob@email.local"
    ],
    "email.to.address": [
        "bad.username@subdomain.bad.local",
        "bob@email.local"
    ],
    "email.reply_to.address": [
        "bad.username@subdomain.bad.local",
        "bob@email.local"
    ],
    "email.cc.address": [
        "bad.username@subdomain.bad.local",
        "bob@email.local"
    ],
    "email.bcc.address": [
        "bad.username@subdomain.bad.local",
        "bob@email.local"
    ],
    "destination.domain": [
        "securewebsite.local",
        "email.local",
        "bob documents.pdf",
        "file.txt",
        "bad.local",
        "not-an-ioc",
        "anotherwebsite.local",
        "n--nhk-u63b1cko2lyc6jrwxgom6k.com",
        "subdomain.bad.local",
        "website.local",
        "aaaaaaaaaaaaaaaa"
    ],
    "url.domain": [
        "securewebsite.local",
        "email.local",
        "bob documents.pdf",
        "file.txt",
        "bad.local",
        "not-an-ioc",
        "anotherwebsite.local",
        "n--nhk-u63b1cko2lyc6jrwxgom6k.com",
        "subdomain.bad.local",
        "website.local",
        "aaaaaaaaaaaaaaaa"
    ],
    "tls.client.server_name": [
        "securewebsite.local",
        "email.local",
        "bob documents.pdf",
        "file.txt",
        "bad.local",
        "not-an-ioc",
        "anotherwebsite.local",
        "n--nhk-u63b1cko2lyc6jrwxgom6k.com",
        "subdomain.bad.local",
        "website.local",
        "aaaaaaaaaaaaaaaa"
    ],
    "dns.question.registered_domain": [
        "securewebsite.local",
        "email.local",
        "bob documents.pdf",
        "file.txt",
        "bad.local",
        "not-an-ioc",
        "anotherwebsite.local",
        "n--nhk-u63b1cko2lyc6jrwxgom6k.com",
        "subdomain.bad.local",
        "website.local",
        "aaaaaaaaaaaaaaaa"
    ],
    "file.origin_referrer_url": [
        "securewebsite.local",
        "email.local",
        "bob documents.pdf",
        "file.txt",
        "bad.local",
        "not-an-ioc",
        "anotherwebsite.local",
        "n--nhk-u63b1cko2lyc6jrwxgom6k.com",
        "subdomain.bad.local",
        "website.local",
        "aaaaaaaaaaaaaaaa"
    ],
    "file.origin_url": [
        "securewebsite.local",
        "email.local",
        "bob documents.pdf",
        "file.txt",
        "bad.local",
        "not-an-ioc",
        "anotherwebsite.local",
        "n--nhk-u63b1cko2lyc6jrwxgom6k.com",
        "subdomain.bad.local",
        "website.local",
        "aaaaaaaaaaaaaaaa"
    ],
    "source.ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "destination.ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "dns.resolved_ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "host.ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "network.forwarded_ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "related.ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "client.ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "server.ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "server.nat.ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "threat.enrichments.indicator.ip": [
        "8.8.8.8",
        "192.168.1.1",
        "192.168.20.20",
        "1.2.3.4",
        "8.8.8.0/24",
        "192.168.1.0/24",
        "fe80::"
    ],
    "file.path": [
        "/home/bob/file.txt",
        "c:\\users\\bob\\desktop\\test.txt",
        "c:/users/bob smith/desktop/file.txt:alt.exe",
        "c:\\users\\bob smith\\desktop\\file.txt:alt.exe",
        "file.txt:alt.exe",
        "c:\\users\\bob smith\\desktop\\file.txt",
        "c:\\users\\bob smith\\desktop\\file2.txt:alt.exe"
    ],
    "file.name": [
        "/home/bob/file.txt",
        "c:\\users\\bob\\desktop\\test.txt",
        "c:/users/bob smith/desktop/file.txt:alt.exe",
        "c:\\users\\bob smith\\desktop\\file.txt:alt.exe",
        "file.txt:alt.exe",
        "c:\\users\\bob smith\\desktop\\file.txt",
        "c:\\users\\bob smith\\desktop\\file2.txt:alt.exe"
    ],
    "dll.hash.md5": [
        "6cd3556deb0da54bca060b4c39479839"
    ],
    "email.attachments.file.hash.md5": [
        "6cd3556deb0da54bca060b4c39479839"
    ],
    "file.hash.md5": [
        "6cd3556deb0da54bca060b4c39479839"
    ],
    "process.hash.md5": [
        "6cd3556deb0da54bca060b4c39479839"
    ],
    "dll.hash.sha256": [
        "315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3"
    ],
    "email.attachments.file.hash.sha256": [
        "315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3"
    ],
    "file.hash.sha256": [
        "315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3"
    ],
    "process.hash.sha256": [
        "315f5bdb76d078c43b8ac0064e4a0164612b1fce77c869345bfc94c75894edd3"
    ],
    "dll.hash.sha512": [
        "c1527cd893c124773d811911970c8fe6e857d6df5dc9226bd8a160614c0cd963a4ddea2b94bb7d36021ef9d865d5cea294a82dd49a0bb269f51f6e7a57f79421"
    ],
    "email.attachments.file.hash.sha512": [
        "c1527cd893c124773d811911970c8fe6e857d6df5dc9226bd8a160614c0cd963a4ddea2b94bb7d36021ef9d865d5cea294a82dd49a0bb269f51f6e7a57f79421"
    ],
    "file.hash.sha512": [
        "c1527cd893c124773d811911970c8fe6e857d6df5dc9226bd8a160614c0cd963a4ddea2b94bb7d36021ef9d865d5cea294a82dd49a0bb269f51f6e7a57f79421"
    ],
    "process.hash.sha512": [
        "c1527cd893c124773d811911970c8fe6e857d6df5dc9226bd8a160614c0cd963a4ddea2b94bb7d36021ef9d865d5cea294a82dd49a0bb269f51f6e7a57f79421"
    ],
    "source.port": [
        9443,
        8443
    ],
    "destination.port": [
        9443,
        8443
    ]
}

Known Issues

  • need to streamline and optimize code
  • remove both regex statements created by AI

Project details

Verified details

These details have been verified by PyPI
Maintainers
Avatar for bonifield_tools from gravatar.com bonifield_tools

Unverified details

These details have not been verified by PyPI
Meta
  • Requires: Python >=3.11

Release history Release notifications | RSS feed

0.3.0

This version

0.2.0

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

parse_ioc-0.2.0.tar.gz (19.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

parse_ioc-0.2.0-py3-none-any.whl (20.1 kB view details)

Uploaded Python 3

File details

Details for the file parse_ioc-0.2.0.tar.gz.

File metadata

  • Download URL: parse_ioc-0.2.0.tar.gz
  • Upload date:
  • Size: 19.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.7.3

File hashes

Hashes for parse_ioc-0.2.0.tar.gz
Algorithm Hash digest
SHA256 18a827bdf5e4a10d945cd2f2d0de493a58068af5e104fbaaf76fed7934e8c1b4
MD5 032de886b714056e74564596bf430139
BLAKE2b-256 9c5093557f284ee7f527fe9285bb8d2f93eaf51abbbd992ff29d915a634c60d4

See more details on using hashes here.

File details

Details for the file parse_ioc-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: parse_ioc-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 20.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.7.3

File hashes

Hashes for parse_ioc-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f407d34f19565a95f3672d3b88b7aec6511dd980a98f38728cb11c8eb3ee2891
MD5 6b70b8c1ad22bb9ff8f1f7f61cede99c
BLAKE2b-256 b428197cb37415851b5da8457a0b42d1815ed3c6280e3a13cec7199a1a17f73b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page