Skip to main content

Retrieve and parse whois data for IPv4 and IPv6 addresses.

Project description

ipwhois is a Python package focused on retrieving and parsing whois data for IPv4 and IPv6 addresses.

RDAP is the recommended query method as of v0.11.0. Please see the upgrade info.

IPWhois.lookup() is deprecated as of v0.12.0 and will be removed. Legacy whois lookups were moved to IPWhois.lookup_whois().

Features

  • Parses a majority of whois fields in to a standard dictionary
  • IPv4 and IPv6 support
  • Referral whois support
  • Supports RDAP queries (recommended method, more detailed information)
  • Proxy support for RDAP queries
  • Recursive network parsing for IPs with parent/children networks listed
  • Python 2.6+ and 3.3+ supported
  • Useful set of utilities
  • BSD license
  • 100% core code coverage (See ‘# pragma: no cover’ for exclusions)

Usage Examples

RDAP (HTTP)

Basic usage

>>>> from ipwhois import IPWhois
>>>> from pprint import pprint

>>>> obj = IPWhois('74.125.225.229')
>>>> results = obj.lookup_rdap(depth=1)
>>>> pprint(results)

{
'asn': '15169',
'asn_cidr': '74.125.225.0/24',
'asn_country_code': 'US',
'asn_date': '2007-03-13',
'asn_registry': 'arin',
'entities': [u'GOGL'],
'network': {'cidr': '74.125.0.0/16',
         'country': None,
         'end_address': '74.125.255.255',
         'events': [{'action': u'last changed',
                     'actor': None,
                     'timestamp': u'2012-02-24T09:44:34-05:00'},
                    {'action': u'registration',
                     'actor': None,
                     'timestamp': u'2007-03-13T12:09:54-04:00'}],
         'handle': u'NET-74-125-0-0-1',
         'ip_version': u'v4',
         'links': [u'https://rdap.arin.net/registry/ip/074.125.000.000',
                   u'https://whois.arin.net/rest/net/NET-74-125-0-0-1'],
         'name': u'GOOGLE',
         'notices': [{'description': u'By using the ARIN RDAP/Whois service, you are agreeing to the RDAP/Whois Terms of Use',
                      'links': [u'https://www.arin.net/whois_tou.html'],
                      'title': u'Terms of Service'}],
         'parent_handle': u'NET-74-0-0-0-0',
         'raw': None,
         'remarks': None,
         'start_address': '74.125.0.0',
         'status': None,
         'type': None},
'objects': {u'ABUSE5250-ARIN': {'contact': {'address': [{'type': None,
                                                      'value': u'1600 Amphitheatre Parkway\nMountain View\nCA\n94043\nUNITED STATES'}],
                                         'email': [{'type': None,
                                                    'value': u'network-abuse@google.com'}],
                                         'kind': u'group',
                                         'name': u'Abuse',
                                         'phone': [{'type': [u'work',
                                                             u'voice'],
                                                    'value': u'+1-650-253-0000'}],
                                         'role': None,
                                         'title': None},
                             'entities': None,
                             'events': [{'action': u'last changed',
                                         'actor': None,
                                         'timestamp': u'2015-11-06T15:36:35-05:00'},
                                        {'action': u'registration',
                                         'actor': None,
                                         'timestamp': u'2015-11-06T15:36:35-05:00'}],
                             'events_actor': None,
                             'handle': u'ABUSE5250-ARIN',
                             'links': [u'https://rdap.arin.net/registry/entity/ABUSE5250-ARIN',
                                       u'https://whois.arin.net/rest/poc/ABUSE5250-ARIN'],
                             'notices': [{'description': u'By using the ARIN RDAP/Whois service, you are agreeing to the RDAP/Whois Terms of Use',
                                          'links': [u'https://www.arin.net/whois_tou.html'],
                                          'title': u'Terms of Service'}],
                             'raw': None,
                             'remarks': [{'description': u'Please note that the recommended way to file abuse complaints are located in the following links.\r\n\r\nTo report abuse and illegal activity: https://www.google.com/intl/en_US/goodtoknow/online-safety/reporting-abuse/ \r\n\r\nFor legal requests: http://support.google.com/legal \r\n\r\nRegards,\r\nThe Google Team',
                                          'links': None,
                                          'title': u'Registration Comments'}],
                             'roles': None,
                             'status': [u'validated']},
         u'GOGL': {'contact': {'address': [{'type': None,
                                            'value': u'1600 Amphitheatre Parkway\nMountain View\nCA\n94043\nUNITED STATES'}],
                               'email': None,
                               'kind': u'org',
                               'name': u'Google Inc.',
                               'phone': None,
                               'role': None,
                               'title': None},
                   'entities': [u'ABUSE5250-ARIN', u'ZG39-ARIN'],
                   'events': [{'action': u'last changed',
                               'actor': None,
                               'timestamp': u'2015-11-06T15:45:54-05:00'},
                              {'action': u'registration',
                               'actor': None,
                               'timestamp': u'2000-03-30T00:00:00-05:00'}],
                   'events_actor': None,
                   'handle': u'GOGL',
                   'links': [u'https://rdap.arin.net/registry/entity/GOGL',
                             u'https://whois.arin.net/rest/org/GOGL'],
                   'notices': None,
                   'raw': None,
                   'remarks': None,
                   'roles': [u'registrant'],
                   'status': None},
         u'ZG39-ARIN': {'contact': {'address': [{'type': None,
                                                 'value': u'1600 Amphitheatre Parkway\nMountain View\nCA\n94043\nUNITED STATES'}],
                                    'email': [{'type': None,
                                               'value': u'arin-contact@google.com'}],
                                    'kind': u'group',
                                    'name': u'Google Inc',
                                    'phone': [{'type': [u'work',
                                                        u'voice'],
                                               'value': u'+1-650-253-0000'}],
                                    'role': None,
                                    'title': None},
                        'entities': None,
                        'events': [{'action': u'last changed',
                                    'actor': None,
                                    'timestamp': u'2015-09-01T14:03:11-04:00'},
                                   {'action': u'registration',
                                    'actor': None,
                                    'timestamp': u'2000-11-30T13:54:08-05:00'}],
                        'events_actor': None,
                        'handle': u'ZG39-ARIN',
                        'links': [u'https://rdap.arin.net/registry/entity/ZG39-ARIN',
                                  u'https://whois.arin.net/rest/poc/ZG39-ARIN'],
                        'notices': [{'description': u'By using the ARIN RDAP/Whois service, you are agreeing to the RDAP/Whois Terms of Use',
                                     'links': [u'https://www.arin.net/whois_tou.html'],
                                     'title': u'Terms of Service'}],
                        'raw': None,
                        'remarks': None,
                        'roles': None,
                        'status': [u'validated']}},
'query': '74.125.225.229',
'raw': None
}

Use a proxy

>>>> from urllib import request
>>>> from ipwhois import IPWhois
>>>> handler = request.ProxyHandler({'http': 'http://192.168.0.1:80/'})
>>>> opener = request.build_opener(handler)
>>>> obj = IPWhois('74.125.225.229', proxy_opener = opener)

Tweaking queries for your network

>>>> from ipwhois import IPWhois
>>>> obj = IPWhois('74.125.225.229', timeout=10)
>>>> results = obj.lookup_rdap(retry_count=5, rate_limit_timeout=60)

Legacy Whois

Basic usage

>>>> from ipwhois import IPWhois
>>>> from pprint import pprint

>>>> obj = IPWhois('74.125.225.229')
>>>> results = obj.lookup_whois()
>>>> pprint(results)

{
'asn': '15169',
'asn_cidr': '74.125.225.0/24',
'asn_country_code': 'US',
'asn_date': '2007-03-13',
'asn_registry': 'arin',
'nets': [{'abuse_emails': 'arin-contact@google.com',
          'address': '1600 Amphitheatre Parkway',
          'cidr': '74.125.0.0/16',
          'city': 'Mountain View',
          'country': 'US',
          'created': '2007-03-13T00:00:00',
          'description': 'Google Inc.',
          'handle': 'NET-74-125-0-0-1',
          'misc_emails': None,
          'name': 'GOOGLE',
          'postal_code': '94043',
          'range': '74.125.0.0 - 74.125.255.255',
          'state': 'CA',
          'tech_emails': 'arin-contact@google.com',
          'updated': '2012-02-24T00:00:00'}],
'query': '74.125.225.229',
'raw': None,
'raw_referral': None,
'referral': None
}

Multiple networks listed and referral whois

>>>> from ipwhois import IPWhois
>>>> from pprint import pprint

>>>> obj = IPWhois('38.113.198.252')
>>>> results = obj.lookup_whois(get_referral=True)
>>>> pprint(results)

{
'asn': '174',
'asn_cidr': '38.0.0.0/8',
'asn_country_code': 'US',
'asn_date': '',
'asn_registry': 'arin',
'nets': [{'abuse_emails': 'abuse@cogentco.com',
          'address': '1015 31st St NW',
          'cidr': '38.0.0.0/8',
          'city': 'Washington',
          'country': 'US',
          'created': '1991-04-16T00:00:00',
          'description': 'PSINet, Inc.',
          'handle': 'NET-38-0-0-0-1',
          'misc_emails': None,
          'name': 'COGENT-A',
          'postal_code': '20007',
          'range': '38.0.0.0 - 38.255.255.255',
          'state': 'DC',
          'tech_emails': 'ipalloc@cogentco.com',
          'updated': '2011-05-20T00:00:00'},
         {'abuse_emails': 'abuse@cogentco.com',
          'address': '1015 31st St NW',
          'cidr': '38.112.0.0/13',
          'city': 'Washington',
          'country': 'US',
          'created': '2003-08-20T00:00:00',
          'description': 'PSINet, Inc.',
          'handle': 'NET-38-112-0-0-1',
          'misc_emails': None,
          'name': 'COGENT-NB-0002',
          'postal_code': '20007',
          'range': None,
          'state': 'DC',
          'tech_emails': 'ipalloc@cogentco.com',
          'updated': '2004-03-11T00:00:00'}],
'query': '38.113.198.252',
'raw': None,
'raw_referral': None,
'referral': {'address': '1015 31st St NW',
             'cidr': '38.113.198.0/23',
             'city': 'Washington',
             'country': 'US',
             'description': 'Cogent communications - IPENG',
             'name': 'NET4-2671C60017',
             'postal_code': '20007',
             'state': 'DC',
             'updated': '2007-09-18 22:02:09'}
}

Utilities

Retrieve host information for an IP address

>>>> from ipwhois import IPWhois
>>>> from pprint import pprint

>>>> obj = IPWhois('74.125.225.229')
>>>> results = obj.get_host()
>>>> pprint(results)

('dfw06s26-in-f5.1e100.net', [], ['74.125.225.229'])

Retrieve the official country name for an ISO 3166-1 country code

>>>> from ipwhois import IPWhois
>>>> from ipwhois.utils import get_countries

>>>> countries = get_countries()
>>>> obj = IPWhois('74.125.225.229')
>>>> results = obj.lookup_whois(False)
>>>> print(countries[results['nets'][0]['country']])

United States

Parse out IP addresses and ports from text or a file

>>>> from ipwhois.utils import unique_addresses
>>>> from pprint import pprint

>>>> input_data = (
    'You can have IPs like 74.125.225.229, or 2001:4860:4860::8888'
    'Put a port at the end 74.125.225.229:80 or for IPv6: '
    '[2001:4860:4860::8888]:443 or even networks like '
    '74.125.0.0/16 and 2001:4860::/32.'
)

>>>> results = unique_addresses(data=input_data, file_path=None)
>>>> pprint(results)

{'2001:4860:4860::8888': {'count': 2, 'ports': {'443': 1}},
 '2001:4860::/32': {'count': 1, 'ports': {}},
 '74.125.0.0/16': {'count': 1, 'ports': {}},
 '74.125.225.229': {'count': 2, 'ports': {'80': 1}}}

Dependencies

Python 2.6, 2.7:

dnspython
ipaddr

Python 3.3+:

dnspython3

Installing

Latest version from PyPi:

pip install --upgrade ipwhois

Latest version from GitHub:

pip install -e git+https://github.com/secynic/ipwhois@master#egg=ipwhois

RDAP (HTTP)

IPWhois.lookup_rdap() is now the recommended lookup method. RDAP provides a far better data structure than legacy whois and REST lookups (previous implementation). RDAP queries allow for parsing of contact information and details for users, organizations, and groups. RDAP also provides more detailed network information.

Upgrading from 0.10 to 0.11

Considerable changes were made between v0.10.3 and v0.11.0. The new RDAP return format was introduced and split off from the legacy whois return format. Using RDAP lookup is the recommended method to maximize indexable values.

RDAP return data is different in nearly every way from the legacy whois data.

For information on raw RDAP responses, please see the RFC: https://tools.ietf.org/html/rfc7483

Here are the new standard keys for RDAP results:

:query: The IP address (String)
:network: Dictionary of values returned by _RDAPNetwork. The raw
        result is included for each entity if the inc_raw parameter is
        True.
:entities: List of entity keys referenced by the top level IP
        address query.
:objects: Dictionary of objects with the handles as keys, and the
        dictionary returned by _RDAPEntity, etc as the values. The raw
        result is included for each object if the inc_raw parameter is
        True.

See the example for more detailed field information.

Legacy Whois Parsing

IPWhois.lookup() is deprecated as of v0.12.0 and will be removed. Legacy whois lookups were moved to IPWhois.lookup_whois().

Parsing is currently limited to CIDR, country, name, handle, range, description, state, city, address, postal_code, abuse_emails, tech_emails, misc_emails, created and updated fields. This is assuming that those fields are present (for both whois and rwhois).

Some IPs have parent networks listed. The parser attempts to recognize this, and break the networks into individual dictionaries. If a single network has multiple CIDRs, they will be separated by ‘, ‘.

Sometimes, you will see whois information with multiple consecutive same name fields, e.g., Description: some text\nDescription: more text. The parser will recognize this and the returned result will have the values separated by ‘\n’.

Country Codes

The legacy country code listing (iso_3166-1_list_en.xml) is no longer available as a free export from iso.org. Support has been added for iso_3166-1.csv, which is now the default.

Use Legacy XML File:

>>>> from ipwhois.utils import get_countries
>>>> countries = get_countries(is_legacy_xml=True)

IP Reputation Support?

This feature is under consideration. Take a look at TekDefense’s Automater:

TekDefense-Automater

Domain Support?

There are no plans for domain whois support in this project.

Look at Sven Slootweg’s python-whois for a library with domain support.

Special Thanks

Thank you JetBrains for the PyCharm open source support!

Changelog

0.12.0 (2016-03-28)

  • Added headers parameter to ipwhois.Net.get_http_json() (issue #98).
  • Fixed ASN HTTP lookup (fallback) Accept headers (issue #98).
  • Fixed HTTP decoding, set to utf-8 (italomaia - issue #97)
  • IPWhois.lookup() deprecated (issue #96), and will be removed in a future release (TBD). Use IPWhois.lookup_whois() instead.
  • Added rate_limit_timeout parameter (issue #99) to Net.get_http_json(), IPWhois.lookup_rdap(), and RDAP.lookup(). New exception HTTPRateLimitError.
  • Added new parameter asn_alts to Net.lookup_asn(), IPWhois.lookup_rdap() and IPWhois.lookup(). Takes an array of lookup types to attempt if the ASN dns lookup fails. Allow permutations must be enabled. Defaults to all [‘whois’, ‘http’] (issue #93).
  • Fixed socket exception handling in Net.get_http_json() for Python 2.6.
  • Fixed assertIsInstance for Python 2.6 tests (issue #100). Implemented unittest._formatMessage and unittest.util.safe_repr for Python 2.6.
  • Moved TestCommon to tests\__init__.py to avoid duplicate code.
  • Replaced remaining % with str.format (issue #95).

0.11.2 (2016-02-25)

  • Added allow_permutations parameter (bool) to net.Net() and ipwhois.IPWhois() to allow alternate ASN lookups if DNS lookups fail. (FirefighterBlu3)
  • Fixed ASN DNS resolver timeout/retry_count support. Retry count is used as a multiplier of timeout, to determine a limetime interval. (FirefighterBlu3)
  • Fixed bug where remarks would return None if missing a title.
  • Added CONTRIBUTING.rst
  • Added tests

0.11.1 (2015-12-17)

  • Re-added CIDR calculation for RDAP lookups.
  • Improved tests - core code coverage now 100%. See ‘# pragma: no cover’ for exclusions. A few bugs were identified in the process, detailed below.
  • Moved IP zero stripping from rdap._RDAPNetwork.parse() to new helper function utils.ipv4_lstrip_zeros().
  • Moved CIDR calculation from rdap._RDAPNetwork.parse() to new helper function utils.calculate_cidr().
  • Fixed utils.ipv4_is_defined() if statement ordering for RFC 1918 conflict.
  • Fixed utils.ipv6_is_defined() if statement ordering for Unspecified and Loopback (conflict with Reserved).
  • Added is_offline parameter to whois.Whois.lookup() primarily for testing.
  • Fixed bug in whois.Whois._parse_fields() that attempted to parse ‘val2’ of regex, which is no longer used. Also fixed the expected Exception to be IndexError.
  • Fixed bug in ipwhois.IPWhois.lookup() where the argument order was mixed up, causing referral lookups to be skipped when get_referral=True.
  • Fixed bug in rdap._RDAPCommon.summarize_notices() output for links.
  • Fixed bug in root entity iteration exception handling in rdap.RDAP.lookup().

0.11.0 (2015-11-02)

  • Support for REST lookups replaced with RDAP.
  • Split code for a more structured system (net, whois, rdap, exceptions).
  • Tests match the data new structure.
  • Split tests for online and offline testing.
  • Performance enhancements for parsing.
  • Added an optional bootstrap parameter for RDAP lookups, in order to replace ASN lookups or use both. Will default to False. Afrinic is currently not supported, so I would not use this for now. ARIN acknowledged my issue for this, and will be adding support back in for Afrinic bootstrap.
  • Added field_list parameter (inclusion list) for WHOIS lookups.
  • Added logging.
  • Added examples directory.

0.10.3 (2015-08-14)

  • Fixed LACNIC lookup_rws() queries, since they switched to RDAP. This is temporary to get it working until the major library transition to RDAP and new parsed formatting is complete.

0.10.2 (2015-05-19)

  • Fixed APNIC parsing for updated field.
  • Fixed datetime parsing and validation when Zulu (Z) is appended.
  • Added RIPE parsing for created and updated fields (whois and RWS).
  • Removed unnecessary parentheses in IPWhois class declaration.
  • Some documentation and comment tweaking to work with Sphinx.
  • Minor PEP 8 tweaks.

0.10.1 (2015-02-09)

  • Fixed setup.py bug.

0.10.0 (2015-02-09)

  • Added .csv support for country code source. You can no longer download country code information from iso.org.
  • Added support for IPv4Address or IPv6Address as the address arg in IPWhois.
  • Fixed file open encoding bug. Moved from open to io.open.
  • Fixed parameter in IPWhois ip defined checks.
  • Fixed TestIPWhois.test_ip_invalid() assertions.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
ipwhois-0.12.0.zip (52.0 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page