Skip to main content

Coverage analyzer for network router configurations

Project description

NetCov

NetCov is an open-source tool that can be used with Batfish to analyze test coverage for network configurations. Given a set of Batfish queries, it analyzes which lines of configurations has/has not been covered. The result can be used to assess the rigorousness of the test suite and help discover the blind spots.

NetCov is written in Python and can be used in concert with pybatfish, Batfish's Python API.

Features

NetCov supports coverage tracking for the following Batfish questions:

NetCov supports configuration formats of the following vendors:

  • Cisco
  • Juniper

NetCov reports configuration coverage as the percentage of configuration lines that are covered, such as:

NetCov can also report fine-grained coverage results as colored annotations on source configurations (lines in blue are covered, lines in red are not covered):

You can find an example of the coverage report here.

Installing NetCov

Install NetCov using pip. A virtual environment and Python version 3.7 is recommended.

$ pip install netcov

NetCov leverages LCOV to generate HTML report. If you would like to use this feature, install LCOV using Homebrew (MacOS) or apt (Ubuntu):

Install lcov on MacOS using Homebrew:

$ brew install lcov

Install lcov on Ubuntu using apt:

$ sudo apt install lcov

Usage

NetCov can be used seamlessly with pybatfish. It provides a hooked version of pybatfish APIs that automatically tracks coverage during the execution of supported pybatfish questions.

Using NetCov for an existing pybatfish script/notebook

It takes only two simple steps to measure coverage for your existing pybatfish scripts/notebooks.

  1. For import, replace pybatfish client session with the one provided by NetCov:
#from pybatfish.client.session import Session
from netcov import NetCovSession as Session
  1. Generate coverage results at the end of your script. To print coverage metrics to the console, use bf.cov.result(). To generate HTML report, use bf.cov.html_report().

Use NetCov for a new pybatfish script/notebook

We provide a demo video and an example to help you get started. If you are new to pybatfish, we recommend reading the pybatfish doc first.

Advanced

Sometimes not all information retrieved by Batfish questions are meant to be tested, for example, when you retrieve all RIB entries but only assert on a subset of them. To help NetCov model coverage more accurately, you can pause coverage tracking and add tested information use a NetCov API:

# pause coverage tracking to avoid over-estimation
bf.cov.pause()
routes = bf.q.routes(nodes="edge-0000").answer().frame()
bf.cov.resume()

# filter RIB entries to test
tested = routes[routes["Network"] == '0.0.0.0/0'].head(1)

# add tested route to coverage trace
bf.cov.add_tested_routes(tested)

bf.cov.result() prints coverage metrics using logging module and writes to stderr by default. To save the coverage report to file, you can customize logger by:

import logging
fh = logging.FileHandler('cov.log')
logging.getLogger('netcov').addHandler(fh)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

netcov-0.1.1.tar.gz (45.5 kB view details)

Uploaded Source

Built Distribution

netcov-0.1.1-py3-none-any.whl (57.7 kB view details)

Uploaded Python 3

File details

Details for the file netcov-0.1.1.tar.gz.

File metadata

  • Download URL: netcov-0.1.1.tar.gz
  • Upload date:
  • Size: 45.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.7.6

File hashes

Hashes for netcov-0.1.1.tar.gz
Algorithm Hash digest
SHA256 8748fb26df0f666d87b38fba17ea1f65ad6fe496cc2c3f94bf73554745d4ce5b
MD5 565a151e88d6b1c0414b4609e0ad5cf5
BLAKE2b-256 99ecedf2275cc7ddc5fe74bcadc6b59588217ac0f2cbb09e47d0153dc4939885

See more details on using hashes here.

File details

Details for the file netcov-0.1.1-py3-none-any.whl.

File metadata

  • Download URL: netcov-0.1.1-py3-none-any.whl
  • Upload date:
  • Size: 57.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.7.6

File hashes

Hashes for netcov-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 cae76613b11e9070b897d96034561f90fe7d7599cdca13372e281b554f3362ef
MD5 74bd065fa2b3d3209820b89dff032b53
BLAKE2b-256 59c8ead38b9ead5d0a202e120fbf2bd73db7e70e90635b115d6d320f0f36757c

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page