Skip to main content

Search network for HTTP servers using a regular expression filter

Project description

The MIT License

Search network for HTTP servers using a regular expression filter.

Use httpfind to obtain the IP addresses of specified devices on a network. HTTP requests for a user specified page are sent in parallel. Responses are compared against a user specified regular expression pattern. Qualified results are returned as a list. The module is readily imported for use in other projects, and it also includes a convenient command line interface.

Installation

pip install httpfind

Examples

Basic import example

import httpfind

result = httpfind.survey(
    network='192.168.0.0/24',
    pattern='(A|a)ccess (P|p)oint',
    path='login.php',
    log=False)

# Results printed as full URLs
print(result)
# Results printed as IP addresses
print([x.hostname for x in result])

Yields

['http://192.168.0.190/login.php', 'http://192.168.0.191/login.php', 'http://192.168.0.192/login.php']
['192.168.0.190', '192.168.0.191', '192.168.0.192']

Command line example

$> httpfind -h
usage: httpfind [-h] [-p PATH] [-f PATTERN] [-l] network

Search 'network' for hosts with a response to 'path' that matches 'filter'

positional arguments:
  network               IP address with optional mask, e.g. 192.168.0.0/24

optional arguments:
  -h, --help            show this help message and exit
  -p PATH, --path PATH  URL path at host, e.g. index.html
  -f PATTERN, --filter PATTERN
                        Regular expression pattern for filter
  -l, --log             Enable logging

$> httpfind 192.168.0.0/24 -f "Access Point" -p login.php
Scanning, please wait ...
Found 3 matches for Access Point on 192.168.0.0/24
192.168.0.190
192.168.0.191
192.168.0.192

Parameters

def survey(network=None, path='', pattern='', log=False):

  • network - IP address and subnet mask compatible with ipaddress library
  • path - Path portion of a URL as defined by url(un)split
  • pattern - A regular expression pattern compatible with re.compile
  • log - boolean to control logging level

Consequently, the network can be defined in either subnet mask (x.x.x.x/255.255.255.0) or CIDR notation (x.x.x.x/24). Presently, httpfind only scans networks of upto 256 addresses as shown in most of the examples. Of course, a single IP address may be specified either by x.x.x.x or x.x.x.x/32.

There are numerous resources for regular expressions, such as the introduction provided by the Python Software Foundation. For the simple cases, using the default or ‘’ will match any pages while a word such as ‘Access’ will match if it’s found in the returned HTML provided it’s the same case.

Performance

As discoverhue utilizes the excellent aiohttp package, requests are sent simultaneously rather than iteratively. More accurately, the requests are sent randomly over a 2.5s interval so as to not spike traffic. The timeout is set for 5.0s, so typical execution time is about 8.0s.

Contributions

Welcome at https://github.com/Overboard/httpfind

Status

Released.

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for httpfind, version 1.0.1
Filename, size File type Python version Upload date Hashes
Filename, size httpfind-1.0.1-py3-none-any.whl (8.9 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size httpfind-1.0.1.tar.gz (5.5 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page