This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Netdiff is a simple Python library that provides utilities for parsing network topology data of open source dynamic routing protocols and detecting changes in these topologies.

Current features:

Goals:

  • provide an abstraction layer to facilitate parsing different network topology formats
  • add support for the most popular dynamic open source routing protocols
  • facilitate detecting changes in network topology for monitoring purposes
  • provide standard NetJSON output
  • keep the library small with as few dependencies as possible

Currently used by

Install stable version from pypi

Install from pypi:

pip install netdiff

Install development version

Install tarball:

pip install https://github.com/ninuxorg/netdiff/tarball/master

Alternatively you can install via pip using git:

pip install -e git+git://github.com/ninuxorg/netdiff#egg=netdiff

If you want to contribute, install your cloned fork:

git clone git@github.com:<your_fork>/netdiff.git
cd netdiff
python setup.py develop

Basic Usage Example

Calculate diff of an OLSR 0.6.x topology:

from netdiff import OlsrParser
from netdiff import diff

old = OlsrParser('./stored-olsr.json')
new = OlsrParser('telnet://127.0.0.1:9090')
diff(old, new)

In alternative, you may also use the subtraction operator:

from netdiff import OlsrParser
from netdiff import diff

old = OlsrParser('./stored-olsr.json')
new = OlsrParser('telnet://127.0.0.1:9090')
old - new

The output will be an ordered dictionary with three keys:

  • added
  • removed
  • changed

Each key will contain a dict compatible with the NetJSON NetworkGraph format representing respectively:

  • the nodes and links that have been added to the topology
  • the nodes and links that have been removed from the topology
  • links that are present in both topologies but their cost changed

If no changes are present, keys will contain None.

So if between old and new there are no changes, the result will be:

{
    "added": None
    "removed": None,
    "changed": None
}

While if there are changes, the result will look like:

{
    "added": {
        "type": "NetworkGraph",
        "protocol": "OLSR",
        "version": "0.6.6",
        "revision": "5031a799fcbe17f61d57e387bc3806de",
        "metric": "ETX",
        "nodes": [
            {
                "id": "10.150.0.7"
            },
            {
                "id": "10.150.0.6"
            }
        ],
        "links": [
            {
                "source": "10.150.0.3",
                "target": "10.150.0.7",
                "cost": 1.50390625
            },
            {
                "source": "10.150.0.3",
                "target": "10.150.0.6",
                "cost": 1.0
            }
        ]
    },
    "removed": {
        "type": "NetworkGraph",
        "protocol": "OLSR",
        "version": "0.6.6",
        "revision": "5031a799fcbe17f61d57e387bc3806de",
        "metric": "ETX",
        "nodes": [
            {
                "id": "10.150.0.8"
            }
        ],
        "links": [
            {
                "source": "10.150.0.7",
                "target": "10.150.0.8",
                "cost": 1.0
            }
        ]
    },
    "changed": {
        "type": "NetworkGraph",
        "protocol": "OLSR",
        "version": "0.6.6",
        "revision": "5031a799fcbe17f61d57e387bc3806de",
        "metric": "ETX",
        "nodes": [],
        "links": [
            {
                "source": "10.150.0.3",
                "target": "10.150.0.2",
                "cost": 1.0
            }
        ]
    }
}

Parsers

Parsers are classes that extend netdiff.base.BaseParser and implement a parse method which is in charge of converting a python data structure into networkx.Graph object and return the result.

Parsers also have a json method which returns valid NetJSON output.

The available parsers are:

Initialization arguments

data: the only required argument, different inputs are accepted:

  • JSON formatted string representing the topology
  • python dict (or subclass of dict) representing the topology
  • string representing a HTTP URL where the data resides
  • string representing a telnet URL where the data resides
  • string representing a file path where the data resides

timeout: integer representing timeout in seconds for HTTP or telnet requests, defaults to None

verify: boolean indicating to the request library whether to do SSL certificate verification or not

Initialization examples

Local file example:

from netdiff import BatmanParser
BatmanParser('./my-stored-topology.json')

HTTP example:

from netdiff import NetJsonParser
url = 'https://raw.githubusercontent.com/interop-dev/netjson/master/examples/network-graph.json'
NetJsonParser(url)

Telnet example with timeout:

from netdiff import OlsrParser
OlsrParser('telnet://127.0.1:8080', timeout=5)

HTTPS example with self-signed SSL certificate using verify=False:

from netdiff import NetJsonParser
OlsrParser('https://myserver.mydomain.com/topology.json', verify=False)

NetJSON output

Netdiff parsers can return a valid NetJSON NetworkGraph object:

from netdiff import OlsrParser

olsr = OlsrParser('telnet://127.0.0.1:9090')

# will return a dict
olsr.json(dict=True)

# will return a JSON formatted string
print(olsr.json(indent=4))

Output:

{
    "type": "NetworkGraph",
    "protocol": "OLSR",
    "version": "0.6.6",
    "revision": "5031a799fcbe17f61d57e387bc3806de",
    "metric": "ETX",
    "nodes": [
        {
            "id": "10.150.0.3"
        },
        {
            "id": "10.150.0.2"
        },
        {
            "id": "10.150.0.4"
        }
    ],
    "links": [
        {
            "source": "10.150.0.3",
            "target": "10.150.0.2",
            "cost": 2.4
        },
        {
            "source": "10.150.0.3",
            "target": "10.150.0.4",
            "cost": 1.0
        }
    ]
}

Exceptions

All the exceptions are subclasses of netdiff.exceptions.NetdiffException.

ConversionException

netdiff.exceptions.ConversionException

Raised when netdiff can’t recognize the format passed to the parser.

Not necessarily an error, should be caught and managed in order to support additional formats.

The data which was retrieved from network/storage can be accessed via the “data” attribute, eg:

def to_python(self, data):
    try:
        return super(OlsrParser, self).to_python(data)
    except ConversionException as e:
        return self._txtinfo_to_jsoninfo(e.data)

ParserError

netdiff.exceptions.ParserError

Raised when the format is recognized but the data is invalid.

NetJsonError

netdiff.exceptions.NetJsonError

Raised when the json method of netdiff.parsers.BaseParser does not have enough data to be compliant with the NetJSON NetworkGraph format specification.

TopologyRetrievalError

netdiff.exceptions.TopologyRetrievalError

Raised when it is not possible to retrieve the topology data (eg: the URL might be temporary unreachable).

Known Issues

ConnectionError: BadStatusLine

If you get a similar error when performing a request to the jsoninfo plugin of olsrd (version 0.6 to 0.9) chances are high that http headers are disabled.

To fix it turn on http headers in your olsrd configuration file, eg:

LoadPlugin "olsrd_jsoninfo.so.0.0"
{
    PlParam "httpheaders" "yes"   # add this line
    PlParam "Port" "9090"
    PlParam "accept" "0.0.0.0"
}

Running tests

Install your forked repo:

git clone git://github.com/<your_fork>/netdiff
cd netdiff/
python setup.py develop

Install test requirements:

pip install -r requirements-test.txt

Run tests with:

./runtests.py

Alternatively, you can use the nose command (which has a ton of available options):

nosetests
nosetests tests.test_olsr  # run only olsr related tests
nosetests tests/test_olsr.py  # variant form of the previous command
nosetests tests.test_olsr:TestOlsrParser  # variant form of the previous command
nosetests tests.test_olsr:TestOlsrParser.test_parse  # run specific test

See test coverage with:

coverage run --source=netdiff runtests.py && coverage report

Contributing

  1. Join the ninux-dev mailing list
  2. Fork this repo and install it
  3. Follow PEP8, Style Guide for Python Code
  4. Write code
  5. Write tests for your code
  6. Ensure all tests pass
  7. Ensure test coverage is not under 90%
  8. Document your changes
  9. Send pull request
Release History

Release History

0.4.7

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4.6

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.alpha

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
netdiff-0.4.7-py2.py3-none-any.whl (20.1 kB) Copy SHA256 Checksum SHA256 3.4 Wheel Jan 4, 2016
netdiff-0.4.7.tar.gz (63.1 kB) Copy SHA256 Checksum SHA256 Source Jan 4, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting