dnstap-receiver 0.6.0

Python Dnstap receiver

Dnstap streams receiver

This Python module acts as a DNS tap streams receiver for DNS servers. Input streams can be a unix socket or multiple remote tcp sender The output is printed directly to stdout or send to remote tcp address in JSON, YAML or one line text format.

Table of contents

• Installation
• Start dnstap receiver
  • Unix socket mode
  • TCP socket mode
• More options
  • Verbose mode
  • Quiet text output
  • External config file
  • JSON-formatted output
  • YAML-formatted output
  • Forward to remote destination
  • Filtering by dnstap identity
• Tested DNS servers
  • ISC - bind
    • Build with dnstap support
    • Unix socket
  • PowerDNS - pdns-recursor
    • Unix socket
    • TCP stream
  • PowerDNS - dnsdist
    • Unix socket
    • TCP stream
  • NLnet Labs - nsd
    • Build with dnstap support
    • Unix socket
    • TCP stream
  • NLnet Labs - unbound
    • Build with dnstap support
    • Unix socket
    • TCP stream
• Tested Logs Collectors
  • Logstash
• Systemd service file configuration
• About

Installation

Deploy the dnstap receiver in your DNS server with the pip command.

pip install dnstap_receiver

Start dnstap receiver

TCP socket mode

This mode enable to receive dnstap messages from multiple dns servers. By default, the receiver is listening on the ip 0.0.0.0 and the tcp port 6000.

./dnstap_receiver

Unix socket mode

In this mode, the dnstap_receiver binary takes in input a unix socket

./dnstap_receiver -u /var/run/dnstap.sock

More options

Verbose mode

You can execute the binary in verbose mode with the -v argument

./dnstap_receiver -v
2020-09-12 23:47:35,833 Start dnstap receiver...
2020-09-12 23:47:35,833 Using selector: EpollSelector
2020-09-12 23:47:35,834 Listening on 0.0.0.0:6000

Quiet text output

By default the output will be print in quiet text format.

2020-09-12 14:15:00.551 dnsdist01 CLIENT_QUERY NOERROR 192.168.1.114 46528 IP4 TCP 43b www.google.com. A
2020-09-12 14:15:00.551 dnsdist01 CLIENT_RESPONSE NOERROR 192.168.1.114 46528 IP4 TCP 101b www.google.com. A

External config file

The dnstap_receiver binary can takes an external config file with the -c argument

./dnstap_receiver -c /etc/dnstap-receiver/dnstap.conf

Example of configuration file

# enable verbose mode
verbose: true

# read and decode dnstap messages from
input-mode:
  # read dnstap message from tcp socket
  local-address: 0.0.0.0
  local-port: 6000
  # read dnstap message fom unix socket
  unix-socket: null
#  # checking the dnstap identify received and ignores-it 
#  # if the message does not match the expected list of identities
#  dnstap-identity:
#    - dnsdist01
#    - unbound01

# format dnstap message output
output-format:
  text: true
  yaml: false
  json: false

# forward decoded messages to a remote tcp destination
forward-to:
  enable: false
  remote-address: null
  remote-port: null

JSON-formatted output

JSON output can be activated through the external configuration file

# format dnstap message output
output-format:
  text: false
  yaml: false
  json: true

Output example:

{
    "identity": "dev-centos8",
    "query-name": "www.google.com.",
    "query-type": "A",
    "source-ip": "192.168.1.114",
    "message": "CLIENT_QUERY",
    "protocol": "IP4",
    "transport": "UDP",
    "source-port": 42222,
    "length": 43,
    "timestamp": "2020-09-12 22:24:34.132",
    "code": "NOERROR"
}

YAML-formatted output

YAML output can be activated through the external configuration file

# format dnstap message output
output-format:
  text: false
  yaml: true
  json: false

Output example:

code: NOERROR
length: 49
message: RESOLVER_QUERY
protocol: IP4
query-name: dns4.comlaude-dns.eu.
query-type: AAAA
source-ip: '-'
source-port: '-'
timestamp: '2020-09-12 14:13:53.948'
transport: UDP

Forward to remote destination

Forward dnstap message to a remote tcp collector can be done through the external configuration file

# forward decoded messages to a remote tcp destination
forward-to:
  enable: true
  remote-address: 10.0.0.2
  remote-port: 8192

Filtering by dnstap identity

You can filtering dnstap messages according to the dnstap identity field. A list of expected identities can be configured in the external configuration file like below:

input-mode:
  # dnstap identify filtering feature
  # list of expected dnstap identities
  dnstap-identities:
    - dnsdist01
    - unbound01

Tested DNS servers

This dnstap receiver has been tested with success with the following dns servers:

• ISC - bind
• PowerDNS - dnsdist, pdns-recursor
• NLnet Labs - nsd, unbound

bind

Dnstap messages supported:

• RESOLVER_QUERY
• RESOLVER_RESPONSE
• CLIENT_QUERY
• CLIENT_RESPONSE
• AUTH_QUERY
• AUTH_RESPONSE

Build with dnstap support

Download latest source and build-it with dnstap support:

./configure --enable-dnstap
make && make install

Unix socket

Update the configuration file /etc/named.conf to activate the dnstap feature:

options {
    dnstap { client; auth; resolver; forwarder; };
    dnstap-output unix "/var/run/named/dnstap.sock";
    dnstap-identity "dns-bind";
    dnstap-version "bind";
}

Execute the dnstap receiver with named user:

su - named -s /bin/bash -c "dnstap_receiver -u "/var/run/named/dnstap.sock""

pdns-recursor

Dnstap messages supported:

• RESOLVER_QUERY
• RESOLVER_RESPONSE

Unix socket

Update the configuration file to activate the dnstap feature:

vim /etc/pdns-recursor/recursor.conf
lua-config-file=/etc/pdns-recursor/recursor.lua

vim /etc/pdns-recursor/recursor.lua
dnstapFrameStreamServer("/var/run/pdns-recursor/dnstap.sock")

Execute the dnstap receiver with pdns-recursor user:

su - pdns-recursor -s /bin/bash -c "dnstap_receiver -u "/var/run/pdns-recursor/dnstap.sock""

TCP stream

Update the configuration file to activate the dnstap feature with tcp mode and execute the dnstap receiver in listening tcp socket mode:

vim /etc/pdns-recursor/recursor.conf
lua-config-file=/etc/pdns-recursor/recursor.lua

vim /etc/pdns-recursor/recursor.lua
dnstapFrameStreamServer("10.0.0.100:6000")

dnsdist

Dnstap messages supported:

• CLIENT_QUERY
• CLIENT_RESPONSE

Unix socket

Create the dnsdist folder where the unix socket will be created:

mkdir -p /var/run/dnsdist/
chown dnsdist.dnsdist /var/run/dnsdist/

Update the configuration file /etc/dnsdist/dnsdist.conf to activate the dnstap feature:

fsul = newFrameStreamUnixLogger("/var/run/dnsdist/dnstap.sock")
addAction(AllRule(), DnstapLogAction("dnsdist", fsul))
addResponseAction(AllRule(), DnstapLogResponseAction("dnsdist", fsul))

Execute the dnstap receiver with dnsdist user:

su - dnsdist -s /bin/bash -c "dnstap_receiver -u "/var/run/dnsdist/dnstap.sock""

TCP stream

Update the configuration file /etc/dnsdist/dnsdist.conf to activate the dnstap feature with tcp stream and execute the dnstap receiver in listening tcp socket mode:

fsul = newFrameStreamTcpLogger("127.0.0.1:8888")
addAction(AllRule(), DnstapLogAction("dnsdist", fsul))
addResponseAction(AllRule(), DnstapLogResponseAction("dnsdist", fsul))

nsd

Dnstap messages supported:

• AUTH_QUERY
• AUTH_RESPONSE

Build with dnstap support

Download latest source and build-it with dnstap support:

./configure --enable-dnstap
make && make install

Unix socket

Update the configuration file /etc/nsd/nsd.conf to activate the dnstap feature:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: "/var/run/nsd/dnstap.sock"
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-auth-query-messages: yes
    dnstap-log-auth-response-messages: yes

Execute the dnstap receiver with nsd user:

su - nsd -s /bin/bash -c "dnstap_receiver -u "/var/run/nsd/dnstap.sock""

unbound

Dnstap messages supported:

• CLIENT_QUERY
• CLIENT_RESPONSE
• RESOLVER_QUERY
• RESOLVER_RESPONSE
• CLIENT_QUERY
• CLIENT_RESPONSE

Build with dnstap support

Download latest source and build-it with dnstap support:

./configure --enable-dnstap
make && make install

Unix socket

Update the configuration file /etc/unbound/unbound.conf to activate the dnstap feature:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: "dnstap.sock"
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-resolver-query-messages: yes
    dnstap-log-resolver-response-messages: yes
    dnstap-log-client-query-messages: yes
    dnstap-log-client-response-messages: yes
    dnstap-log-forwarder-query-messages: yes
    dnstap-log-forwarder-response-messages: yes

Execute the dnstap receiver with unbound user:

su - unbound -s /bin/bash -c "dnstap_receiver -u "/usr/local/etc/unbound/dnstap.sock""

TCP stream

Update the configuration file /etc/unbound/unbound.conf to activate the dnstap feature with tcp mode and execute the dnstap receiver in listening tcp socket mode:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: ""
    dnstap-ip: "10.0.0.100@6000"
    dnstap-tls: no
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-client-query-messages: yes
    dnstap-log-client-response-messages: yes

Tested Logs Collectors

Logstash with json input

vim /etc/logstash/conf.d/00-dnstap.conf

input {
  tcp {
      port => 8192
      codec => json
  }
}

filter {
  date {
     match => [ "dt_query" , "yyyy-MM-dd HH:mm:ss.SSS" ]
     target => "@timestamp"
  }
}

output {
   elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "dnstap-lb"
  }
}

Systemd service file configuration

System service file for CentOS:

vim /etc/systemd/system/dnstap_receiver.service

[Unit]
Description=Python DNS tap Service
After=network.target

[Service]
ExecStart=/usr/local/bin/dnstap_receiver -u /etc/dnsdist/dnstap.sock -f 10.0.0.2:8192
Restart=on-abort
Type=simple
User=root

[Install]
WantedBy=multi-user.target

systemctl daemon-reload
systemctl start dnstap_receiver
systemctl status dnstap_receiver
systemctl enable dnstap_receiver

About

Author	Denis Machard [e-mail]
License	MIT
PyPI	https://pypi.org/project/dnstap_receiver/