Rule based dependency scanning and service fingerprinting via DNS
Project description
DNSMule
Package for rule based dependency scanning and service fingerprinting via DNS.
This package provides utilities for writing and evaluating verbose and easy to read rule definitions in YAML-format. There are two builtin rule formats with more available as plugins.
Overview
The DNSMule tool is composed in the following way:
- DNSMule
- Backend
- Rules
- Rule
- Plugins
Backends
Any backend can be used and registered in YAML, the only requirement is extending the dnsmule.backends.Backend
class.
A backend is responsible for creating dnsmule.definition.Record
instances for processing with dnsmule.rules.Rules
.
Plugins
Plugins provide DNSMule with additional functionality and/or rule(types).
Plugins can be registered in YAML.
It is required that all plugins extend the dnsmule.plugins.Plugin
class.
Rules
This class is responsible for processing everyRecord
and producing a dnsmule.definition.Result
for them.
The actual processing happens in individual dnsmule.rules.Rule
instances that are ordered and orchestrated by this
class.
Rule
These are user defined rules that process individual Record
instances and append information to record.data
or record.tags
. The record.data
is a storage for result metadata and additional information. The record.tags
set
is a collection of identifications for an input Record
.
Current State
States (Complete, Testing, Developing, Refining) for every feature planned at this moment.
- (Complete) Backends
- (Complete) DNSPythonBackend
- (Complete) YAML definition
- (Complete) Rules
- (Complete) Builtin types
- (Complete) Custom rules
- (Complete) Custom rule factories
- (Complete) YAML definition
- (Testing) Plugins
- (Complete) Registration in program
- (Complete) Registration in YAML
- (Complete) YAML definition
- (Developing) Increasing test coverage
- (Refining) Result Storage
- (Refining) Registration in program
- (Refining) Registration in YAML
- (Refining) YAML definition
- (Developing) DNSMule
- (Complete) Rules from YAML
- (Complete) Backend from YAML
- (Testing) Plugins from YAML
- (Developing) Processing and gathering results
- (Refining) Result storage
- (Refining) Result storage from YAML
- (Refining) Combined YAML definition
Example server
The repo has a Dockerfile
for easily running the tool using an example server in Docker:
$ ./build-image
$ ./run-server
Notice
This package is under active development.
Additional
Rules
Summary
The tool configuration is done through one or multiple rule configuration files. The file structure is defined in the schema file. In addition to some builtin rule types, it is possible to create new types by registering handlers or rules programmatically.
Rules support registration per DNS record type and priority for controlling invocation order.
version: '0.0.1'
rules:
- name: o365
priority: 10
type: dns.regex
record: txt
config:
pattern: '^MS=ms'
identification: MICROSOFT::O365
- name: ses
type: dns.regex
record: txt
config:
pattern: '^amazonses:'
identification: AMAZON::SES
Example
from dnsmule.definitions import Record, Result
from dnsmule.rules import Rules, Rule
rules: Rules
...
@rules.add.A[10]
async def my_scan(record: Record) -> Result:
from dnsmule.config import get_logger
get_logger().info('Address %s', record)
return record.identify('MY::SCAN')
@rules.register('my.rule')
def create_my_rule(**arguments) -> Rule:
...
Here the add
is used to directly register a new rule into the ruleset with a given priority. The register
call
creates a new handler for rules of type my.rule
. Any future my.rule
creations from yml
or code would be routed to
this factory function.
Other Components in YAML
Plugins
It is possible to register plugins using the YAML file:
plugins:
- name: dnsmule_plugins.CertCheckPlugin
config:
callback: false
These are required to extend the dnsmule.plugins.Plugin
class.
Plugins are evaluated and initialized before rules.
Any rules requiring a plugin should list their plugin in this block.
Plugins are only initialized once and if a plugin already exists in the receiving DNSMule instance
it will be ignored.
Backends
It is possible to define a single backend in a YAML file:
backend:
name: 'dnspython.DNSPythonBackend'
config:
timeout: 5.5
resolver: 8.8.8.8
The backend must extend the dnsmule.backends.Backend
class.
This declaration is ignored if this is not used in DNSMule.load
or DNSMule(file=file)
.
Editor Support
Type Hints and JSON Schema (IntelliJ IDEA, PyCharm, etc.)
It is possible to register the schema file as a custom JSON schema in IntelliJ editors. This will give access to typehints and schema validation inside rule files and is especially nice for dynamic rule definitions as you get full editor features for python inside the snippets.
- settings...
- Languages & Frameworks > Schemas and DTDs > JSON Schema Mappings
- Add a new mapping with the schema file and specified file or pattern
This is configured in the schema using the custom intellij language injection tag:
x-intellij-language-injection:
language: Python
prefix: |+0
from dnsmule.rules.entities import Type, Record, Result
def add_rule(record_type: Any, rule_type: str, name: str, priority: int = 0, **options) -> None:
pass
suffix: ''
Currently, this supports dns.regex
pattern regex language injection and dns.dynamic
rule code language injection.
Type hints and quick documentation are available.
Builtin types
Regex rules
Regex rules can be defined with either one pattern
or multiple patterns
.
An example is in the following snippet:
rules:
- name: test
type: dns.regex
record: txt
config:
pattern: '^.*\.hello_world\.'
identification: HELLO::WORLD
flags:
- UNICODE
- DOTALL
- name: generic_verification
type: dns.regex
record: txt
priority: 10
description: Generic Site Regex Collection
config:
patterns:
- '^(.+)(?:-(?:site|domain))?-verification='
- '^(.+)(?:site|domain)verification'
- '^(.+)_verify_'
- '^(\w+)-code:'
group: 1
The full definition is available from the schema file.
Dynamic Rules
Dynamic rules are defined as code snippets with one or two methods
An init method that is invoked once after creation
def init() -> None:
add_rule(...)
A process function that is invoked once for each record
def process(record: Record) -> Result:
add_rule(...)
return record.result()
Both of these functions have access to the following rule creation method:
def add_rule(
record_type: Any,
rule_type: str,
name: str,
priority: int = 0,
**options,
) -> None:
"""
:param record_type: Valid DNS record type as text, int, or type
:param rule_type: Valid rule type factory e.g. dns.regex
:param name: Name of the created rule
:param priority: Priority for the created rule, default 0
:param options: Any additional options for the rule factory
"""
pass
The only globals passed to these methods are:
- __builtins__
- from dnsmule.definitions import RRType, Record, Result
- add_rule
- Any additional globals created by the code itself
When the code is exec'd the result is inspected for:
- init function without parameters
- process function with a single parameter
Some notes:
- The init function is invoked exactly once.
- The process function is invoked exactly once for every single Record.
- Any rules created from the init method will be invoked for every suitable record.
- Any rules created from the process method will be invoked for suitable records found after creation.
- Creating DynamicRules from init or process is considered undefined behaviour and care should be taken
- The user should call init manually and include fail-safes for only calling it once
- The add_rule callback might not be available so you need to pass it manually to the rule
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for dnsmule-0.1.0.post1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f47cecb5e7b0bfa7147d84345b6ca5c8346ebea05bbe9929939dee616ef17ab5 |
|
MD5 | 3790c4094355e92fc3d0f06b4626641f |
|
BLAKE2b-256 | e5027440100fa14e849dca509b58baa40708e5a712d424eba4c4e005d43228d0 |