dnstap-receiver 1.1.0

Python Dnstap receiver

Dnstap streams receiver

This Python module acts as a DNS tap streams receiver for DNS servers. Input streams can be a unix socket or multiple remote dns servers. The output is printed directly to stdout or send to remote tcp address in JSON, YAML or one line text format.

Table of contents

• Installation
• Start dnstap receiver
  • Unix socket mode
  • TCP socket mode
  • TLS socket mode
• More options
  • Verbose mode
  • External config file
  • Formatted output
  • Filtering feature
  • Forward feature
• Tested DNS servers
  • ISC - bind
  • PowerDNS - pdns-recursor
  • PowerDNS - dnsdist
  • NLnet Labs - nsd
  • NLnet Labs - unbound
• Tested Logs Collectors
  • Rsyslog
  • Logstash
• Systemd service file configuration
• About

Installation

Deploy the dnstap receiver in your DNS server with the pip command.

pip install dnstap_receiver

Start dnstap receiver

TCP socket mode

This mode enable to receive dnstap messages from multiple dns servers. By default, the receiver is listening on the ip 0.0.0.0 and the tcp port 6000.

./dnstap_receiver

Unix socket mode

In this mode, the dnstap_receiver binary takes in input a unix socket

./dnstap_receiver -u /var/run/dnstap.sock

TLS socket mode

This mode enable to receive dnstap messages from multiple dns servers with tcp/tls transport. By default, the receiver is listening on the ip 0.0.0.0 and the tcp port 6000.

Generate a certificate and private key for the dnstap receiver:

openssl req -x509 -newkey rsa:4096 -sha256  -nodes -keyout \
server.key -out server.crt  -subj "/CN=dnstap_receiver.com" -days 3650

Create the external configuration file and enable tls:

input:
  # read dnstap messages from tcp socket
  tcp-socket:
    # listening address ipv4 0.0.0.0 or ipv6 [::]
    local-address: 0.0.0.0
    # listing on port
    local-port: 6000
    # enable tls on socket
    tls-support: true
    # provide certificate server path
    tls-server-cert: /etc/dnstap_receiver/server.crt
    # provide certificate key path
    tls-server-key: /etc/dnstap_receiver/server.key

Finally execute the dnstap receiver with the configuration file:

./dnstap_receiver -c /etc/dnstap-receiver/dnstap.conf

More options

Verbose mode

You can execute the binary in verbose mode with the -v argument

./dnstap_receiver -v
2020-09-12 23:47:35,833 Start dnstap receiver...
2020-09-12 23:47:35,833 Using selector: EpollSelector
2020-09-12 23:47:35,834 Listening on 0.0.0.0:6000

External config file

The dnstap_receiver binary can takes an external config file with the -c argument See config file example.

./dnstap_receiver -c /etc/dnstap-receiver/dnstap.conf

Formatted output

Output can be formatted in different way:

• text (default one)
• json
• yaml

Quiet text

By default the output will be print in quiet text format.

2020-09-16T18:51:53.547352+00:00 centos RESOLVER_QUERY NOERROR - - IP4 UDP 43b ns2.google.com. A
2020-09-16T18:51:53.591736+00:00 centos RESOLVER_RESPONSE NOERROR - - IP4 UDP 59b ns2.google.com. A

JSON-formatted

JSON output can be activated with the external configuration file

output:
  sdtout:
    format: json

Output example:

{
    "identity": "dev-centos8",
    "query-name": "www.google.com.",
    "query-type": "A",
    "source-ip": "192.168.1.114",
    "message": "CLIENT_QUERY",
    "protocol": "IP4",
    "transport": "UDP",
    "source-port": 42222,
    "length": 43,
    "timestamp": "2020-09-12 22:24:34.132",
    "code": "NOERROR"
}

YAML-formatted

YAML output can be activated with the external configuration file

output:
  sdtout:
    format: yaml

Output example:

code: NOERROR
length: 49
message: RESOLVER_QUERY
protocol: IP4
query-name: dns4.comlaude-dns.eu.
query-type: AAAA
source-ip: '-'
source-port: '-'
timestamp: '2020-09-12 14:13:53.948'
transport: UDP

Filtering feature

This feature can be useful if you want to ignore some messages and keep just what you want. Several filter are available:

• by qname field
• by dnstap identity field.

By dnstap identity

You can filtering incoming dnstap messages according to the dnstap identity field. A regex can be configured in the external configuration file to do that

filter:
  # dnstap identify filtering feature with regex support
  dnstap-identities: dnsdist01|unbound01

By qname

You can filtering incoming dnstap messages according to the query name. A regex can be configured in the external configuration file to do that

filter: 
  # qname filtering feature with regex support
  qname-regex: ".*.com"

Forward feature

Output can be configured to forward messages in several modes.

• stdout
• remote tcp socket
• syslog

Forward to STDOUT

This mode is the default one.

Forward to remote TCP socket

Forward dnstap message to a remote tcp collector can be done through the external configuration file

Enable the tcp-socket output handler in the external config file like below

output:
  # forward to remote tcp destination
  tcp-socket:
    # enable or disable
    enable: true
    # format available text|json|yaml
    format: text
    # delimiter
    delimiter: "\n"
    # retry interval in seconds to connect
    retry: 5
    # remote ipv4 or ipv6 address
    remote-address: 10.0.0.2
    # remote tcp port
    remote-port: 8192

Forward to syslog server

Enable the syslog output handler in the external config file

output:
  syslog:
    # enable or disable
    enable: false
    # syslog over tcp or udp
    transport: udp
    # format available text|json
    format: text
    # retry interval in seconds to connect
    retry: 5
    # remote ipv4 or ipv6 address of the syslog server
    remote-address: 10.0.0.2
    # remote port of the syslog server
    remote-port: 514

Example of output on syslog server

Sep 22 12:43:01 bind CLIENT_RESPONSE NOERROR 192.168.1.100 51717 IP4 UDP 173b www.netflix.fr. A
Sep 22 12:43:01 bind CLIENT_RESPONSE NOERROR 192.168.1.100 51718 IP4 UDP 203b www.netflix.fr. AAAA

Tested DNS servers

This dnstap receiver has been tested with success with the following dns servers:

• ISC - bind
• PowerDNS - dnsdist, pdns-recursor
• NLnet Labs - nsd, unbound

bind

Dnstap messages supported:

• RESOLVER_QUERY
• RESOLVER_RESPONSE
• CLIENT_QUERY
• CLIENT_RESPONSE
• AUTH_QUERY
• AUTH_RESPONSE

Build with dnstap support

Download latest source and build-it with dnstap support:

./configure --enable-dnstap
make && make install

Unix socket

Update the configuration file /etc/named.conf to activate the dnstap feature:

options {
    dnstap { client; auth; resolver; forwarder; };
    dnstap-output unix "/var/run/named/dnstap.sock";
    dnstap-identity "dns-bind";
    dnstap-version "bind";
}

Execute the dnstap receiver with named user:

su - named -s /bin/bash -c "dnstap_receiver -u "/var/run/named/dnstap.sock""

pdns-recursor

Dnstap messages supported:

• RESOLVER_QUERY
• RESOLVER_RESPONSE

Unix socket

Update the configuration file to activate the dnstap feature:

vim /etc/pdns-recursor/recursor.conf
lua-config-file=/etc/pdns-recursor/recursor.lua

vim /etc/pdns-recursor/recursor.lua
dnstapFrameStreamServer("/var/run/pdns-recursor/dnstap.sock")

Execute the dnstap receiver with pdns-recursor user:

su - pdns-recursor -s /bin/bash -c "dnstap_receiver -u "/var/run/pdns-recursor/dnstap.sock""

TCP stream

Update the configuration file to activate the dnstap feature with tcp mode and execute the dnstap receiver in listening tcp socket mode:

vim /etc/pdns-recursor/recursor.conf
lua-config-file=/etc/pdns-recursor/recursor.lua

vim /etc/pdns-recursor/recursor.lua
dnstapFrameStreamServer("10.0.0.100:6000")

dnsdist

Dnstap messages supported:

• CLIENT_QUERY
• CLIENT_RESPONSE

Unix socket

Create the dnsdist folder where the unix socket will be created:

mkdir -p /var/run/dnsdist/
chown dnsdist.dnsdist /var/run/dnsdist/

Update the configuration file /etc/dnsdist/dnsdist.conf to activate the dnstap feature:

fsul = newFrameStreamUnixLogger("/var/run/dnsdist/dnstap.sock")
addAction(AllRule(), DnstapLogAction("dnsdist", fsul))
addResponseAction(AllRule(), DnstapLogResponseAction("dnsdist", fsul))

Execute the dnstap receiver with dnsdist user:

su - dnsdist -s /bin/bash -c "dnstap_receiver -u "/var/run/dnsdist/dnstap.sock""

TCP stream

Update the configuration file /etc/dnsdist/dnsdist.conf to activate the dnstap feature with tcp stream and execute the dnstap receiver in listening tcp socket mode:

fsul = newFrameStreamTcpLogger("127.0.0.1:8888")
addAction(AllRule(), DnstapLogAction("dnsdist", fsul))
addResponseAction(AllRule(), DnstapLogResponseAction("dnsdist", fsul))

nsd

Dnstap messages supported:

• AUTH_QUERY
• AUTH_RESPONSE

Build with dnstap support

Download latest source and build-it with dnstap support:

./configure --enable-dnstap
make && make install

Unix socket

Update the configuration file /etc/nsd/nsd.conf to activate the dnstap feature:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: "/var/run/nsd/dnstap.sock"
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-auth-query-messages: yes
    dnstap-log-auth-response-messages: yes

Execute the dnstap receiver with nsd user:

su - nsd -s /bin/bash -c "dnstap_receiver -u "/var/run/nsd/dnstap.sock""

unbound

Dnstap messages supported:

• CLIENT_QUERY
• CLIENT_RESPONSE
• RESOLVER_QUERY
• RESOLVER_RESPONSE
• CLIENT_QUERY
• CLIENT_RESPONSE

Build with dnstap support

Download latest source and build-it with dnstap support:

./configure --enable-dnstap
make && make install

Unix socket

Update the configuration file /etc/unbound/unbound.conf to activate the dnstap feature:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: "dnstap.sock"
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-resolver-query-messages: yes
    dnstap-log-resolver-response-messages: yes
    dnstap-log-client-query-messages: yes
    dnstap-log-client-response-messages: yes
    dnstap-log-forwarder-query-messages: yes
    dnstap-log-forwarder-response-messages: yes

Execute the dnstap receiver with unbound user:

su - unbound -s /bin/bash -c "dnstap_receiver -u "/usr/local/etc/unbound/dnstap.sock""

TCP stream

Update the configuration file /etc/unbound/unbound.conf to activate the dnstap feature with tcp mode and execute the dnstap receiver in listening tcp socket mode:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: ""
    dnstap-ip: "10.0.0.100@6000"
    dnstap-tls: no
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-client-query-messages: yes
    dnstap-log-client-response-messages: yes

TLS stream

Update the configuration file /etc/unbound/unbound.conf to activate the dnstap feature with tls mode and execute the dnstap receiver in listening tcp/tls socket mode:

dnstap:
    dnstap-enable: yes
    dnstap-socket-path: ""
    dnstap-ip: "10.0.0.100@6000"
    dnstap-tls: yes
    dnstap-send-identity: yes
    dnstap-send-version: yes
    dnstap-log-client-query-messages: yes
    dnstap-log-client-response-messages: yes

Tested Logs Collectors

Rsyslog

Edit the config file /etc/rsyslog.conf to activate tcp mode

module(load="imtcp")
input(type="imtcp" port="514")

Logstash

vim /etc/logstash/conf.d/00-dnstap.conf

input {
  tcp {
      port => 8192
      codec => json
  }
}

filter {
  date {
     match => [ "dt_query" , "yyyy-MM-dd HH:mm:ss.SSS" ]
     target => "@timestamp"
  }
}

output {
   elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "dnstap-lb"
  }
}

Systemd service file configuration

System service file for CentOS:

vim /etc/systemd/system/dnstap_receiver.service

[Unit]
Description=Python DNS tap Service
After=network.target

[Service]
ExecStart=/usr/local/bin/dnstap_receiver -u /etc/dnsdist/dnstap.sock -f 10.0.0.2:8192
Restart=on-abort
Type=simple
User=root

[Install]
WantedBy=multi-user.target

systemctl daemon-reload
systemctl start dnstap_receiver
systemctl status dnstap_receiver
systemctl enable dnstap_receiver

About

Author	Denis Machard [e-mail]
License	MIT
PyPI	https://pypi.org/project/dnstap_receiver/