svc-tracer 0.0.4

Service tracer base on scapy

svc-tracer

Table of Contents

• About
• Installing
• Usage

About

svc-tracer is a command line tool based on scapy to trace service traffic, such as REST, thrift, .etc.

Installing

• Windows

1. Install Python 2.7 or 3.4+:

   When installing, option Add python.exe to Path must be selected and enabled. Or after installation, manually add the Python installation directory and its Scripts subdirectory to your PATH. Depending on your Python version, the defaults would be C:\Python27 and C:\Python27\Scripts respectively.

2. Install the latest version of Npcap:

   Default values are recommended. Scapy will also work with Winpcap.

3. Install svc-tracer via pip

   pip install -i https://pypi.tuna.tsinghua.edu.cn/simple/ svc-tracer
   

4. Reboot the system

• Linux

1. Install Python 2.7 or 3.4+

2. Install tcpdump and make sure it is in the $PATH

   # Debian/Ubuntu:
   sudo apt-get install tcpdump
   
   # Fedora:
   yum install tcpdump
   

3. Install svc-tracer via pip

   pip install -i https://mirrors.aliyun.com/pypi/simple/ svc-tracer
   

• MacOSX

1. Install Python 2.7 or 3.4+

2. Install libpcap

   brew update
   brew install libpcap
   

3. Install svc-tracer via pip

   pip install -i https://pypi.douban.com/simple/ svc-tracer
   

• Known Issues

Refer scapy install guide for known problems.

Usage

• General command help

  svc-tracer --help
  usage: svc-tracer [-h] {show,thrift,http} ...
  
  positional arguments:
    {show,thrift,http}
      show              Show all interfaces
      thrift            Trace thrift service
      http              Trace http service
  
  optional arguments:
    -h, --help          show this help message and exit
  

• Command: show

  Sub command show will show all available network interface's name in user's computer, so that they can use it as the parameter of --iface:

• Command: http

  Sub command http will trace http service requests and responses on specific network interface and specific port, command line help is below:

  svc-tracer http --help
  usage: svc-tracer http [-h] -i <iface> -p <port> [-f <ip> [<ip> ...]] [-t <ip> [<ip> ...]] [-m <regex> [<regex> ...]] [--unpaired]
                         [--print <mode>] [--check_interval <interval>] [--check_timeout <timeout>] [-l <level>] [-o <path>] [-c]
  
  optional arguments:
    -h, --help            show this help message and exit
    -i <iface>, --iface <iface>
                          The interface of sniff from
    -p <port>, --port <port>
                          The port of the service listens to
    -f <ip> [<ip> ...], --from <ip> [<ip> ...]
                          Only record the messages from this IP(s)
    -t <ip> [<ip> ...], --to <ip> [<ip> ...]
                          Only record the messages to this IP(s)
    -m <regex> [<regex> ...], --method <regex> [<regex> ...]
                          Only record the messages match this method(s)
    --unpaired            Print the messages as they arrive, possibly out of order
    --print <mode>        Print parts of the message. Options: header, data, all, none
    --check_interval <interval>
                          Interval in seconds for checking unpair message
    --check_timeout <timeout>
                          Timeout in seconds for checking unpair alert
    -l <level>, --log-level <level>
                          The output level of messages. Options: debug, info, warn, error
    -o <path>, --log-file <path>
                          The output file of messages
    -c, --log-clear       The output file will be cleared
  

  • iface: Required. Specific the name of network interface, such as: eth0, ``

  • port: Required. Specific the port of remote or local service listens to

  • from: Optional. Make tracer only record the request from specific ip(s) and the response to specific ip(s).

    Default: do not filter message by from ip.

  • to: Optional. Make tracer only record the request to specific ip(s) and from response to specific ip(s).

    Default: do not filter message by to ip.

  • method: Optional. Make tracer only record the request and response with http url matching specific regex expression(s).

    Default: do not filter message by request url.

  • unpaired: Optional. This option will not pair the request and response, and print the message separately as they arrive. Note: possibly out of order because the pcap limitation.

    Default: Pair the request and response, and print them together.

  • print: Optional. This option will define the print parts of message. Options: header, data, all, none

    Default: all, print all parts of message.

  • check_interval: Optional. Tracer in paired mode will check the unpaired messages in queue for the specific interval. It is useful for finding requests without responses. This option will not work in unpaired mode.

    Default: 5 seconds.

  • check_timeout: Optional. Unpaired message's timeout value. Tracer will alert and print the timeout unpaired messages in error level. This option will not work in unpaired mode.

    Default: 5 seconds.

  • log-level: Optional. The output level of messages. Tracer will only print the messages with level higher than this level.

    Default: info, print all messages.

  • log-file: Optional. The path of output file for messages.

    Default: None, only output to console.

  • log-clear: Optional. Whether the output file will be cleared first.

    Default: false, append to output file.

• Command: thrift

  Sub command thrift will trace thrift service requests and responses on specific network interface and specific port, command line help is below:

  svc-tracer thrift --help
  usage: svc-tracer thrift [-h] -i <iface> -p <port> [-f <ip> [<ip> ...]] [-t <ip> [<ip> ...]] [-m <regex> [<regex> ...]]
                           [--unpaired] [--print <mode>] [--check_interval <interval>] [--check_timeout <timeout>] [-l <level>]
                           [-o <path>] [-c] [--transport <trans>] [--protocol <proto>] [--finagle] [--idl-file <path>]
  
  optional arguments:
    -h, --help            show this help message and exit
    -i <iface>, --iface <iface>
                          The interface of sniff from
    -p <port>, --port <port>
                          The port of the service listens to
    -f <ip> [<ip> ...], --from <ip> [<ip> ...]
                          Only record the messages from this IP(s)
    -t <ip> [<ip> ...], --to <ip> [<ip> ...]
                          Only record the messages to this IP(s)
    -m <regex> [<regex> ...], --method <regex> [<regex> ...]
                          Only record the messages match this method(s)
    --unpaired            Print the messages as they arrive, possibly out of order
    --print <mode>        Print parts of the message. Options: header, data, all, none
    --check_interval <interval>
                          Interval in seconds for checking unpair message
    --check_timeout <timeout>
                          Timeout in seconds for checking unpair alert
    -l <level>, --log-level <level>
                          The output level of messages. Options: debug, info, warn, error
    -o <path>, --log-file <path>
                          The output file of messages
    -c, --log-clear       The output file will be cleared
    --transport <trans>   Use a specific transport. Options: auto, buffered, framed
    --protocol <proto>    Use a specific protocol. Options: auto, binary, compact, json
    --finagle             Detect finagle-thrift traffic (i.e.: with request headers)
    --idl-file <path>     Use .thrift file to resolve types
  

  • iface: Required. Specific the name of network interface, such as: eth0, ``

  • port: Required. Specific the port of remote or local service listens to

  • from: Optional. Make tracer only record the request from specific ip(s) and the response to specific ip(s).

    Default: do not filter message by from ip.

  • to: Optional. Make tracer only record the request to specific ip(s) and from response to specific ip(s).

    Default: do not filter message by to ip.

  • method: Optional. Make tracer only record the request and response with http url matching specific regex expression(s).

    Default: do not filter message by request url.

  • unpaired: Optional. This option will not pair the request and response, and print the message separately as they arrive. Note: possibly out of order because the pcap limitation.

    Default: Pair the request and response, and print them together.

  • print: Optional. This option will define the print parts of message. Options: header, data, all, none

    Default: all, print all parts of message.

  • check_interval: Optional. Tracer in paired mode will check the unpaired messages in queue for the specific interval. It is useful for finding requests without responses. This option will not work in unpaired mode.

    Default: 5 seconds.

  • check_timeout: Optional. Unpaired message's timeout value. Tracer will alert and print the timeout unpaired messages in error level. This option will not work in unpaired mode.

    Default: 5 seconds.

  • log-level: Optional. The output level of messages. Tracer will only print the messages with level higher than this level.

    Default: info, print all messages.

  • log-file: Optional. The path of output file for messages.

    Default: None, only output to console.

  • log-clear: Optional. Whether the output file will be cleared first.

    Default: false, append to output file.

  • transport: Optional. The transport frame mode of the thrift message.

    Default: auto, auto detect the transport mode.

  • protocol: Optional. The protocol of the thrift message.

    Default: auto, auto detect the protocol.

  • finagle: Optional. Try to detect finagle-thrift's request headers.

    Default: false, auto detect the protocol.

  • idl-file: Optional. The path of .thrift file for resolving types of method's arguments.

    Default: None, do not use .thrift file.