IpSurv

IpSurv are investigation tools for surveying IP addresses, network investigation, test and debugging - "ipsurv, ipscap, ipsend". Those tools allow for packet capture, such as tcpdump, and packet sending tests. Each tools and internal program are extensible using Python.

Installation

PyPI

$ pip install ipsurv
or
$ pip3 install ipsurv

Conda

$ conda install conda-forge::ipsurv

ipscap is also installed with it.

Requirements

• python and pip command
• Python 3.0 or later version.

If you use in Python 3.0 - 3.2, please run pip install ipaddress.

If you'd like to use in Python 2.7, you can refactor to Python 2.7 code easily. See "development_debug.md".

Commands

It’s best to refer to the help to recognize the functions.

$ ipsurv --help

# ipscap --help

$ ipsend --help

Documentation site

IpSurv's documentation site is https://deer-hunt.github.io/ipsurv/.

"ipsurv" command

ipsurv command reference is here.

Features

• Grouping by IP or Subnet.
• Skip duplicate by the group.
• Autodetect IP in line. Trying to resolve the name automatically.
• Autodetect delimiter-char.
• Customize output format. There are many format.
• Output JSON format.
• Show headers.
• Check ICMP, TCP, UDP, HTTP.
• Set timeout.
• Load env variable. And changing arguments and internal configures.
• Use GeoIP2 via IpSurv optionally.

IPv6 is not supported.

Usage

Specify the target using Argument

$ ipsurv 192.168.1.10
$ ipsurv 192.168.1.10 192.168.1.11
$ ipsurv test-example-sample-ipsurv.com --add_ip
$ ipsurv x.x.x.x --format=geo

Specify the target using PIPE

$ cat ips.txt|ipsurv
$ cat apache.log|ipsurv --add_ip

$ cat ./example_data/government.txt|ipsurv --sequence --add_ip
1,www.whitehouse.gov,192.0.66.168,OK,US,AUTOMATTIC,192.0.64.0,192.0.127.255
2,www.state.gov,3.165.39.61,OK,US,AMAZON-CF,3.165.0.0,3.165.255.255
3,www.treasury.gov,23.50.118.187,OK,US,AKAMAI,23.32.0.0,23.67.255.255
4,www.gov.uk,151.101.192.144,OK,US,SKYCA-3,151.101.0.0,151.101.255.255

Output by JSON

$ ipsurv wikipedia.org --format=default --json=2 --add_ip
{
  "original": "wikipedia.org",
  "ip": "103.102.166.224",
  "status": "OK",
  "group": "",
  "country": "US",
  "name": "WIKIMEDIA-AP",
  "network_start": "103.102.166.0",
  "network_end": "103.102.166.255"
}

Format profile/parameters

You can customize "Output Format" by --format option as follows. There are parameter - {} and profile - <> in --format. For more information, please read --format description, Profiles, Parameters.

$ ipsurv github.io --format=heavy            # Profile
$ ipsurv github.io --format=simple           # Profile

$ ipsurv 8.8.8.8 --format="{status},{ip},{country},{address}"   # Parameters

Check HTTP response

$ ipsurv https://www.reddit.com --format="{ip},{http},{http_status},{http_size},{http_mime},{http_server},{http_h2},{http_time}" --http=1
https://www.reddit.com,151.101.129.140,HTTP_OK,200,707634,text/html,snooserv,N/A,130.2

GeoIP

$ ipsurv 8.8.8.8 --geoip_only
8.8.8.8,NA,North America,US,United States,America/Chicago,AS15169,37.751;-97.822

Command options

ipsurv have many options. Please read Command arguments(.md) reference.

usage: ipsurv [-h] [--verbose {0,1,2,3}] [--debug] [--log {string}]
              [--disable_env] [--resolve {0,1}] [--identify_int]
              [--autodetect] [--begin {number}] [--end {number}]
              [--collect {string}] [--all_collect] [--timeout {string}]
              [--group {string}] [--skip_duplicate {0,1,2}] [--range {string}]
              [--format {string}] [--no_original] [--sequence] [--add_ip]
              [--ident] [--enclose {string}] [--delimiter {string}]
              [--alt_delimiter {string}] [--headers {0,1,2,3}]
              [--json {0,1,2}] [--json_list] [--exhaustive] [--icmp {0,1}]
              [--tcp {number}] [--udp {number}] [--http {0,1,2}] [--json_all]
              [--geoip_only] [--host_only] [--version]
              [target [target ...]]

"ipscap" command

ipscap command reference is here.

Features

• Capture TCP, UDP, ICMP packets
• Show IP-header values and protocol's header values.
• Output the binary data of headers in HEX format.
• Filter by strings or various criteria.
• Allows tracking matched transfers.
• Various output mode.
• Dump to files.

IPv6 is not supported.

Usage

# ipscap --exclude_ssh
# ipscap --force
  
# ipscap --port="80;53" --find="GET"
# ipscap --condition="port=80,443,53,-1" --protocol=TCP,UDP,ICMP
# ipscap --find="HTTP" --tracking

# ipscap --port=80,443 --stat_group=1

Filterings

# ipscap --find="HTTP/1.1 \d01"
# ipscap --find="http" --find_mode=MATCH
# ipscap --find="00 99 f0 e0 78 4e 23 70 a1" --find_mode=HEX
# ipscap --find="Accept-Ranges: bytes\r\n\r\n\x00\x00\x01\x00\x01\x00" --find_mode=BINARY
# ipscap --find="HTTP" --tracking
# ipscap --condition="port!=22"
# ipscap --condition="src_port>=80;src_port<=500;flags=SYN,PSH"
# ipscap --condition="ttl>=120"

Dump files

# ipscap --port=80 --dumpfile=1

# ipscap --output=HEADER # HEADER only
# ipscap --output=BINARY --port="80" # BINARY
# ipscap --output=binary --port="80" # BINARY
# ipscap --output=BINARY_ALL --port="80" # BINARY with headers
# ipscap --output=LINE --port="80" #LINE
# ipscap --output=HEX --port="80" # HEX
# ipscap --output=hex --port="80" # HEX
# ipscap --output=BASE64 --port="80" # BASE64

Capture 80 port

# ipscap --port=80

Time:           2025-01-04 17:23:46.7809 / 1736011426.7809, Passage number: 1
IP header:      Version: 4, IP header length: 20, Total length: 44, Checksum: 17625, TTL: 64, IP protocol: TCP[6]
TCP header:     TCP header length: 24, Checksum: 36766, Sequence: 132160001, Acknowledgement: 57321677, Window: 65535, Flags: ['SYN', 'ACK']
TCP options:    mss:1460
Source:         IP: 151.101.129.140           Port: 80
Destination:    IP: 10.0.2.15                 Port: 38158
Direction:      RECEIVE [ <<< ]
Data length:    2 byte
IP-H data:      45 00 00 2c 10 f3 00 00 40 06 44 d9 97 65 81 8c 0a 00 02 0f 
TCP-H data:     00 50 95 0e 07 e0 9a 01 03 6a a8 cd 60 12 ff ff 8f 9e 00 00 02 04 05 b4 

Time:           2025-01-04 17:23:46.7812 / 1736011426.7812, Passage number: 2
IP header:      Version: 4, IP header length: 20, Total length: 40, Checksum: 31449, TTL: 64, IP protocol: TCP[6]
TCP header:     TCP header length: 20, Checksum: 9499, Sequence: 57321677, Acknowledgement: 132160002, Window: 29200, Flags: ['ACK']
TCP options:    -
Source:         IP: 10.0.2.15                 Port: 38158
Destination:    IP: 151.101.129.140           Port: 80
Direction:      SEND [ >>> ]
Data length:    0 byte
IP-H data:      45 00 00 28 9a f6 40 00 40 06 7a d9 0a 00 02 0f 97 65 81 8c 
TCP-H data:     95 0e 00 50 03 6a a8 cd 07 e0 9a 02 50 10 72 10 25 1b 00 00 

Time:           2025-01-04 17:23:46.7814 / 1736011426.781, Passage number: 3
IP header:      Version: 4, IP header length: 20, Total length: 118, Checksum: 31370, TTL: 64, IP protocol: TCP[6]
TCP header:     TCP header length: 20, Checksum: 9577, Sequence: 57321677, Acknowledgement: 132160002, Window: 29200, Flags: ['PSH', 'ACK']
TCP options:    -
Source:         IP: 10.0.2.15                 Port: 38158
Destination:    IP: 151.101.129.140           Port: 80
Direction:      SEND [ >>> ]
Data length:    78 byte
IP-H data:      45 00 00 76 9a f7 40 00 40 06 7a 8a 0a 00 02 0f 97 65 81 8c 
TCP-H data:     95 0e 00 50 03 6a a8 cd 07 e0 9a 02 50 18 72 10 25 69 00 00 

GET / HTTP/1.1
User-Agent: curl/7.29.0
Host: www.reddit.com
Accept: */*

Output line format

# ipscap --port=80 --output=LINE
2025-01-02 14:55:55.7247, 1,  4, 20, 64, 60,      TCP, 40, 1165755664, 0, 29200,          ['SYN'],              0,      10.0.2.15:57910,         151.101.129.140:80,      SEND,          mss:1460;sack;nop;wscale:7
2025-01-02 14:55:55.7275, 1,  4, 20, 64, 44,      TCP, 24, 3072001, 1165755665, 65535,    ['SYN', 'ACK'],       2,      151.101.129.140:80,      10.0.2.15:57910,         RECEIVE,       mss:1460
2025-01-02 14:55:55.7277, 2,  4, 20, 64, 40,      TCP, 20, 1165755665, 3072002, 29200,    ['ACK'],              0,      10.0.2.15:57910,         151.101.129.140:80,      SEND,          -
2025-01-02 14:55:55.7278, 3,  4, 20, 64, 118,     TCP, 20, 1165755665, 3072002, 29200,    ['PSH', 'ACK'],       78,     10.0.2.15:57910,         151.101.129.140:80,      SEND,          -
2025-01-02 14:55:55.7278, 2,  4, 20, 64, 40,      TCP, 20, 3072002, 1165755743, 65535,    ['ACK'],              6,      151.101.129.140:80,      10.0.2.15:57910,         RECEIVE,       -
2025-01-02 14:55:55.7322, 3,  4, 20, 64, 982,     TCP, 20, 3072002, 1165755743, 65535,    ['PSH', 'ACK'],       942,    151.101.129.140:80,      10.0.2.15:57910,         RECEIVE,       -
2025-01-02 14:55:55.7324, 4,  4, 20, 64, 40,      TCP, 20, 1165755743, 3072944, 30144,    ['ACK'],              0,      10.0.2.15:57910,         151.101.129.140:80,      SEND,          -
2025-01-02 14:55:55.7325, 5,  4, 20, 64, 40,      TCP, 20, 1165755743, 3072944, 30144,    ['FIN', 'ACK'],       0,      10.0.2.15:57910,         151.101.129.140:80,      SEND,          -
2025-01-02 14:55:55.7326, 4,  4, 20, 64, 40,      TCP, 20, 3072944, 1165755743, 65535,    ['FIN', 'ACK'],       6,      151.101.129.140:80,      10.0.2.15:57910,         RECEIVE,       -
2025-01-02 14:55:55.7327, 6,  4, 20, 64, 40,      TCP, 20, 1165755744, 3072945, 30144,    ['ACK'],              0,      10.0.2.15:57910,         151.101.129.140:80,      SEND,          -
2025-01-02 14:55:55.7327, 5,  4, 20, 64, 40,      TCP, 20, 3072944, 1165755744, 65535,    ['FIN', 'ACK'],       6,      151.101.129.140:80,      10.0.2.15:57910,         RECEIVE,       -

Output HEX

# ipscap --port=80 --output=HEX

Time:           2025-01-02 22:29:48 / 1735787388.957, Passage number: 3
IP header:      Version: 4, IP header length: 20, Total length: 978, Checksum: 18625, TTL: 64, IP protocol: TCP[6]
TCP header:     TCP header length: 20, Checksum: 26766, Sequence: 67904002, Acknowledgement: 172468636, Window: 65535, Flags: ['PSH', 'ACK']
TCP options:    -
Source:         IP: 151.101.129.140           Port: 80
Destination:    IP: 10.0.2.15                 Port: 52386
Direction:      RECEIVE [ <<< ]
Data length:    938 byte
IP-H data:      45 00 03 d2 3e a6 00 00 40 06 13 80 97 65 81 8c 0a 00 02 0f 
TCP-H data:     00 50 cc a2 04 0c 22 02 0a 47 a9 9c 50 18 ff ff 06 2f 00 00 

48 54 54 50 2f 31 2e 31 20 33 30 31 20 52 65 64 69 72 65 63 74 0d 0a 44 61 74 65 3a 20 53 61 74 2c 20 31 31 20 4a 61 6e 20 32 30 32 35 20 31 32 3a 34 35 3a 34 36 20 47 4d 54 0d 0a 
43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 6b 65 65 70 2d 61 6c 69 76 65 0d 0a 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c 3a 20 6e 6f 2d 73 74 6f 72 65 0d 0a 4c 6f 63 61 74 69 6f 6e 3a 20 68 74
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4c 61 6e 67 75 61 67 65 3a 20 65 6e 0d 0a 41 63 63 65 70 74 2d 43 48 3a 20 53 65 63 2d 43 48 2d 55 41 2d 46 75 6c 6c 2d 56 65 72 73 69 6f 6e 2d 4c 69 73 74 2c 20 53 65 63 2d 43
72 63 68 3d 2a 0d 0a 50 65 72 6d 69 73 73 69 6f 6e 73 2d 50 6f 6c 69 63 79 3a 20 75 6e 6c 6f 61 64 3d 28 29 0d 0a 43 6f 6e 74 65 6e 74 2d 4c 65 6e 67 74 68 3a 20 31 0d 0a 0d 0a 20 

Command options

usage: ipscap [-h] [--verbose {0,1,2,3}] [--debug] [--log {string}]
              [--find {string}] [--find_mode [REGEX, MATCH, BINARY, HEX]]
              [--port {int}] [--protocol [ICMP, TCP, UDP]] [--ip {string}]
              [--condition {string}] [--tracking] [--stat_mode {0,1,2}]
              [--stat_group {0,1,2}]
              [--output [NONE, HEADER, TEXT, BINARY, BINARY_ALL, HEX, HEX_ALL, LINE]]
              [--output_raw] [--dumpfile {0,1,2}] [--timeout {float}]
              [--exclude_ssh] [--web_port] [--general_port] [--force]
              [--version]

"ipsend" command

ipsend command reference is here.

Features

• Transmit by TCP, UDP, SSL.
• Transmit by Raw socket.
• Support Instant transmission and Interactive transmission.
• Specify Input and Output format - TEXT, HEX, BINARY, BASE64.
• Set SSL context - SSLv3, TLS1.0, TLS1.1, TLS1.2, TLS1.3.

IPv6 is not supported.

Usage

$ ipsend --dest=google.com --http -I
$ ipsend --dest=google.com --port=80 --interactive=2

$ ipsend "GET /index.html HTTP/1.1\\n" --dest=google.com --http
$ ipsend "GET / HTTP/1.1\\n" --dest=google.com --https
$ ipsend --dest=google.com --https -I --output=BASE64
$ ipsend --mode=UDP --dest=8.8.8.8 --port=53
$ ipsend --mode=TCP --dest=wikipedia.org --http -I --output=BINARY

$ ipsend --mode=SSL --dest=google.com --port=443 -I
$ ipsend --mode=SSL --dest=google.com --https -I --output=BINARY

$ ipsend --dest=google.com --http -I
Mode: TCP
Input: TEXT / Output: TEXT
Destination: google.com
Port: 80

[INTERACTIVE] / Line-break to send

Please input send-data. Input a line break to send.

Command options

usage: ipsend [-h] [--verbose {0,1,2,3}] [--debug] [--log {string}]
                   [--mode {TCP,UDP,SSL,IP_HEADER,TCP_HEADER,UDP_HEADER,ICMP_HEADER,IP_PAYLOAD,TCP_PAYLOAD,UDP_PAYLOAD,ICMP_PAYLOAD}]
                   [--input {TEXT,BINARY,HEX,BASE64}]
                   [--output {NONE,TEXT,BINARY,HEX,BASE64}]
                   [--interactive {int}]
                   [--ssl_context {SSLV3,TLS1.0,TLS1.1,TLS1.2,TLS1.3}]
                   [--output_send {int}] [--auto_nl {bool}] [--dest {string}]
                   [--port {int}] [--timeout {float}] [--dumpfile]
                   [--ip_flags {int}] [--ip_identification {int}]
                   [--ip_ttl {int}] [--ip_protocol {int}] [--src_ip {int}]
                   [--src_port {int}] [--dest_ip {int}] [--dest_port {int}]
                   [--tcp_flags {str}] [--tcp_seq {int}] [--tcp_ack {int}]
                   [--tcp_window {int}] [--icmp_type {int}]
                   [--icmp_code {int}] [--icmp_id {int}] [--icmp_seq {int}]
                   [-I] [--http] [--https] [--version]

Path summary

Debugging

In verbose mode, outputting internal data and behaviors in detail.

$ ipsurv ***** --verbose=2    #INFO
$ ipsurv ***** --verbose=3    #DEBUG

$ ipsurv ***** --debug     #DEBUG  This option is equivalent to "--verbose=3"

# ipscap ***** --verbose=2    #INFO
# ipscap ***** --verbose=3    #DEBUG

# ipscap ***** --debug     #DEBUG  This option is equivalent to "--verbose=3"

Dependencies

• dnspython
• geoip2 [Optional]

Related OSS

• IpServer