programming language for security made easy
Project description
When you need to create complex Workflows and need to communicate different tools working together, maybe you need MIST
.
Why is MIST
MIST
is a high level programming language for defining executions workflows easily.
Ab execution is a command line tool you can invoke from MIST
. They will connect the tools and manage executions and synchronization fo you.
Installing
> pip install mist-lang
Quick Start
Demo 1 - The simplest scenario
Explanation
In this scenario we'll do:
CLI Input
- Read a domain as a parameter from CLI.Search Domains
- Use MIST function for search related domains / sub-domains from a start domain.Fin OpenPorts
- Search open port for each new domain / sub-domain found.Screen (Pring)
- Displays the results into the screen (by using MIST 'print' function).
Use case diagram
MIST code
# examples/demo/scenario-01.mist
include "searchDomains" "findOpenPorts"
searchDomains(%domain) => foundDomains
findOpenPorts(:foundDomains, "80,443") => openPortsFound
print(:openPortsFound)
Execute
> mist examples/demo/scenario-01.mist domain=example.com
Demo 2 - Sending results to Kafka
Explanation
In this scenario we'll do:
CLI Input
- Read a domain as a parameter from CLI.Search Domains
- Use MIST function for search related domains / sub-domains from a start domain.FindOpenPorts
- Search open port for each new domain / sub-domain found.Kafka output
- Send results to a Kafka topic.
Use case diagram
MIST code
# examples/demo/scenario-02.mist
include "searchDomains" "findOpenPorts" "kafkaProducer"
searchDomains(%domain) => foundDomains
findOpenPorts(:foundDomains, "80,443") => openPortsFound
kafkaProducer($KAFKA_SERVER, "domainsTopic", :openPortsFound)
Execute
> mist examples/demo/scenario-02.mist domain=example.com
Demo 3 - Adding new tool and remove duplicate domains
Explanation
In this scenario we'll do:
CLI Input
- Read a domain as a parameter from CLI.- Search domains:
Search Domains
- Use MIST function for search related domains / sub-domains from a start domain.Festin
- Use MIST integration for Festin for search related domains / sub-domains from a start domain.
Filter Repeated
- Use MIST function to detect and remove repeated found domains.Fin OpenPorts
- Search open port for each new domain / sub-domain get fromFitler Repeated
.Kafka output
- Send results to a Kafka topic.
Use case diagram
MIST code
# examples/demo/scenario-03.mist
include "searchDomains" "festin" "findOpenPorts" "filterRepeated" "kafkaProducer"
searchDomains(%domain) => foundDomains
festin(%domain, $DNS_SERVER, True) => foundDomains
filterRepeated(:foundDomains, False) => nonRepeatedFoundDomains
findOpenPorts(:nonRepeatedFoundDomains, "80,443") => openPortsFound
kafkaProducer($KAFKA_SERVER, "domainsTopic", :openPortsFound)
Execute
> mist examples/demo/scenario-03.mist domain=example.com
Demo 4 - Send results to Kafka and S3 through a dispatcher
Explanation
In this scenario we'll do:
CLI Input
- Read a domain as a parameter from CLI.- Search domains:
Search Domains
- Use MIST function for search related domains / sub-domains from a start domain.Festin
- Use MIST integration for Festin for search related domains / sub-domains from a start domain.
Filter Repeated
- Use MIST function to detect and remove repeated found domains.Find OpenPorts
- Search open port for each new domain / sub-domain get fromFitler Repeated
.Dispatcher (80 / 443)
- Split results and send each port to a different queue.- Send results:
Kafka output
- Send found 80 ports to a Kafka topic.S3 output
- Send found 443 ports to a AWS S3 bucket.
Use case diagram
MIST code
# examples/demo/scenario-04.mist
include "searchDomains" "festin" "findOpenPorts" "filterRepeated" "kafkaProducer" "S3Store"
function dispacher(p) {
if (isEqual(p.port, "80")) {
send(p, "kafkaOutput")
} else {
send(p, "S3Output")
}
}
searchDomains(%domain) => foundDomains
festin(%domain, $DNS_SERVER, True) => foundDomains
filterRepeated(:foundDomains, False) => nonRepeatedFoundDomains
findOpenPorts(:nonRepeatedFoundDomains, "80,443") => openPortsFound
dispacher(:openPortsFound)
kafkaProducer($KAFKA_SERVER, "domainsTopic", :kafkaOutput)
S3Store(:S3Output, $BUCKET_URI)
Execute
> mist examples/demo/scenario-04.mist domain=example.com
Demo 5 - Read from Kafka and a File
Explanation
In this scenario we'll do:
1 Input from multiple sources:
File Input
- Read domains from an external file.Kafka Input
- Read domains from Kafka topics.CLI Input
- Read domains from CLI.- Search domains:
Search Domains
- Use MIST function for search related domains / sub-domains from a start domain.Festin
- Use MIST integration for Festin for search related domains / sub-domains from a start domain.
Filter Repeated
- Use MIST function to detect and remove repeated found domains.Find OpenPorts
- Search open port for each new domain / sub-domain get fromFitler Repeated
.Dispatcher (80 / 443)
- Split results and send each port to a different queue.- Send results:
Kafka output
- Send found 80 ports to a Kafka topic.S3 output
- Send found 443 ports to a AWS S3 bucket.
Use case diagram
MIST code
# examples/demo/scenario-05.mist
include "searchDomains" "festin" "findOpenPorts" "filterRepeated" "kafkaProducer" "S3Store" "kafkaConsumer" "tail"
function dispacher(p) {
if (isEqual(p.port, "80")) {
send(p, "kafkaOutput")
} else {
send(p, "S3Output")
}
}
kafkaConsumer($KAFKA_SERVER, "inputTopic", "*END*", False) => inputDomains
tail("domains.txt", "*END*") => inputDomains
send(%domain, "inputDomains")
searchDomains(:inputDomains) => foundDomains
festin(:inputDomains, $DNS_SERVER, True) => foundDomains
filterRepeated(:foundDomains, False) => nonRepeatedFoundDomains
findOpenPorts(:nonRepeatedFoundDomains, "80,443") => openPortsFound
dispacher(:openPortsFound)
kafkaProducer($KAFKA_SERVER, "domainsTopic", :kafkaOutput)
S3Store(:S3Output, $BUCKET_URI)
Execute
> mist examples/demo/scenario-05.mist domain=example.com
Authors
MIST is being developed by BBVA-Labs Security team members.
Contributions
Contributions are of course welcome. See CONTRIBUTING or skim existing tickets to see where you could help out.
License
MIST is Open Source Software and available under the Apache 2 license
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for mist_lang-0.3.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 70e4020d8b069cb6a6046cd269ce079bb80ae1e404a356a1f1e95fa9de64dc57 |
|
MD5 | e798361213db16830f4522f7be9b5536 |
|
BLAKE2b-256 | 0506413d2e3d3b1ed49397dd801999bc568e5993fc6cee901d51b036ebc8348e |