Skip to main content

Python interface for Network Source of Truth (nsot)

Project description

https://travis-ci.org/dropbox/pynsot.svg

Python client library and command-line utility for the Network Source of Truth (NSoT) REST API.

Warning

This project is still very much in flux and likely to have backwards-incompatible changes as it evolves with the API for the Network Source of Truth project.

Using

All of the endpoints are dynamic, so it’s like magic. Literally:

$ python
>>> from pynsot.client import EmailHeaderClient
>>> email = 'jathan@localhost'
>>> url = 'http://localhost:8990/api'
>>> api = EmailHeaderClient(url, email=email)
>>> print api.users.get()
>>> print api.sites(1).get()
>>> print api.sites(1).attributes.get()

Authentication

The default method of authentication for NSoT is auth_token and this is what the client uses. You’ll need to retreive your secret_key from the NSoT web interface for this to work:

$ python
>>> from pynsot.client import Client
>>> email = 'jathan@localhost'
>>> secret_key = 'qONJrNpTX0_9v7H_LN1JlA0u4gdTs4rRMQklmQF9WF4='
>>> url = 'http://localhost:8990/api'
>>> api = Client(url, email=email, secret_key=secret_key)

Config

By default, the nsot command-line utility will read your settings from ~/.pynsotrc, which looks like this:

$ cat ~/.pynsotrc
[pynsot]
url = http://localhost:8990/api
email = jathan@localhost
secret_key = qONJrNpTX0_9v7H_LN1JlA0u4gdTs4rRMQklmQF9WF4=
auth_method = auth_token

You can create it yourself, or you’ll be prompted to create one if it isn’t found the first time you use the nsot utility:

$ nsot sites list
/home/jathan/.pynsotrc not found; would you like to create it? [Y/n]: y
Please enter URL: http://localhost:8990/api
Please enter SECRET_KEY: qONJrNpTX0_9v7H_LN1JlA0u4gdTs4rRMQklmQF9WF4=
Please enter EMAIL: jathan@localhost

Default Site

You may optionally specify a default_site setting within your ~/.pynsotrc that will pass along the designated site_id on your behalf any time the -s/--site-id option is required:

[pynsot]
; ... Other settings ...
default_site = 1

Since most users will only have a single site they care about, this is to make your life easier. If you do have multiple sites, you can still provide the -s/--site-id option to override your default_site.

CLI Usage

Every resource type has four eligible actions representing creation, retrieval, update, and removal:

  • add

  • list

  • remove

  • update

Each resource can be manipulated using positional command arguments:

$ nsot
Usage: nsot [OPTIONS] COMMAND [ARGS]...

  Network Source of Truth (NSoT) command-line utility.

Options:
  -v, --verbose  Toggle verbosity.
  --version      Show the version and exit.
  -h, --help     Show this message and exit.

Commands:
  attributes  Attribute objects.
  changes     Change events.
  devices     Device objects.
  networks    Network objects.
  sites       Site objects.

The actions for each resource have varying options and requirements which will be covered a little later. For example, to work with Sites you would run nsot sites:

$ nsot sites
Usage: nsot sites [OPTIONS] COMMAND [ARGS]...

  Site objects.

  Sites are the top-level resource from which all other resources descend.
  In other words, Sites contain Attributes, Changes, Devices, and Networks.

Options:
  -h, --help  Show this message and exit.

Commands:
  add     Add a new Site.
  list    List existing Sites.
  remove  Remove a Site.
  update  Update a Site.

Getting Help

Every resource and action for each resource has help text that can be accessed using the -h/--help option. Use it!

Required Options

When adding objects, certain fields will be required. The required options will be designated as such with a [required] tag in the help text (for example from nsot sites add --help:

-n, --name NAME         The name of the Site.  [required]

If a required option is not provided, nsot will complain:

Error: Missing option "-n" / "--name".

Site ID

For all resources other than Sites, the -s/--site-id option is required to specify which Site you would like the object to be under. (See: default_site)

Updating or Removing Objects

When updating or removing objects, you must provide their unique ID using the -i/--id option. The object’s ID can be obtained by using the list action.

Updating Attributes

When modifying attributes on Device and Network resources, you have three actions to choose from:

  • Add (-A/--add-attributes). This is the default behavior that will add attributes if they don’t exist, or update them if they do.

  • Delete (-d/--delete-attributes). This will cause attributes to be deleted. If combined with --multi the attribute will be deleted if either no value is provided, or if the attribute no longer contains a valid value.

  • Replace (-r/--replace-attributes). This will cause attributes to replaced. If combined with -m/--multi and multiple attributes of the same name are provided, only the last value provided will be used.

Please note that this does not apply when updating Attribute resources themselves. Attribute values attached to Devices and Networks are considered to be “instances” of Attributes.

Viewing Objects

Each resource’s list action supports -i/--id, -l/--limit and -o/--offset options.

  • The -i/--id option will retrieve a single object by the provided unique ID and will override any other list options.

  • The -l/--limit option will limit the set of results to N resources.

  • The -o/--offset option will skip the first N resources.

Set Queries

The Device and Network resources support a -q/--query option that is a representation of set operations for matching attribute/value pairs.

The operations are evaluated from left-to-right, where the first character indicates the set operation:

  • “+” indicates a set union

  • “-” indicates a set difference

  • no marker indicates a set intersection

For example

  • -q "foo=bar" would return the set intersection of objects with foo=bar.

  • -q "foo=bar -owner=jathan" would return the set difference of all objects with foo=bar (that is all foo=bar where owner is not jathan).

  • -q "foo=bar +foo=baz would return the set union of all objects with foo=bar or foo=baz (that is all objects matching either).

The ordering of these operations is important. If you are not familiar with set operations, please check out Basic set theory concepts and notation (Wikipedia).

Bulk Addition of Objects

Attributes, Devices, and Networks may be created in bulk by using the -b/--bulk-add option and specifying a file path to a colon-delimited file.

The format of this file must adhere to the following format:

  • The first line of the file must be the field names.

  • All required fields must be present, however, the order of any of the fields does not matter.

  • Repeat: The fields may be in any order so long as the required fields are present! Missing fields will fallback to their defaults!

  • Attribute pairs must be commma-separated, and in format k=v and the attributes must exist!

  • For any fields that require Boolean values, the following applies:

    • You may specify True or False and they will be evaluated

    • If the value for a field is not set it will evaluate to False

    • Any other value for a field will evaluate to True

Attributes

Sample file for nsot devices add --bulk-add /tmp/attributes:

name:resource_name:required:description:multi:display
owner:Network:True:Network owner:True:True
metro:Device:False:Device metro:False:True

Devices

Sample file for nsot devices add --bulk-add /tmp/devices:

hostname:attributes
device5:foo=bar,owner=team-networking
device6:foo=bar,owner=team-networking

Networks

Sample file for nsot networks add --bulk-add /tmp/networks:

cidr:attributes
10.20.30.0/24:foo=bar,owner=team-networking
10.20.31.0/24:foo=bar,owner=team-networking

Working with Resources

Sites

Sites are the top-level resource from which all other resources descend. In other words, Sites contain Networks, Attributes, Devices, etc. These examples illustrate having many Sites.

Adding a site:

$ nsot sites add --name Spam --description 'Spam Site'
[SUCCESS] added site with args: name=Space, description=Spam Site!

Listing all Sites:

$ nsot sites list
+--------------------------+
| ID   Name    Description |
+--------------------------+
| 1    Foo     Foo Site    |
| 2    Bar     Bar Site    |
| 3    Baz     Baz Site    |
| 4    Spam    Sheep Site  |
| 5    Sheep   Sheep Site  |
+--------------------------+

Listing a single Site:

$ nsot sites list --name Foo
+-------------------------+
| ID   Name   Description |
+-------------------------+
| 1    Foo    Foo Site    |
+-------------------------+

Listing a few Sites:

$ nsot sites list --limit 2
+--------------------------+
| ID   Name    Description |
+--------------------------+
| 1    Foo     Foo Site    |
| 2    Bar     Bar Site    |
+--------------------------+

Updating a Site:

$ nsot sites update --id 2 --name Snickers
[SUCCESS] updated site with args: description=None, name=Snickers!

$ nsot sites list --name Snickers
+-----------------------------+
| ID   Name       Description |
+-----------------------------+
| 2    Snickers   Bar Site    |
+-----------------------------+

Removing a Site:

$ nsot sites remove --id 1
[SUCCESS] removed site with args: id=1!

Attributes

Attributes are flexible key/value pairs or tags you may use to assign arbitrary data to objects.

Adding an Attribute:

$ nsot attributes add --site-id 1 -n owner --r Device -d "Owner of a device." --required
[SUCCESS] Added attribute with args: multi=False, resource_name=Device, name=owner, required=True, display=False, description=Owner of a device.!

Listing all Attributes:

$ nsot attributes list --site-id 1
+-----------------------------------------------------------------------------+
| ID   Name    Resource   Required?   Display?   Multi?   Description         |
+-----------------------------------------------------------------------------+
| 3    owner   Device     True        False      False    Owner of a device.  |
| 4    foo     Network    False       False      False    Foo for devices     |
| 2    owner   Network    False       False      False    Owner of a network. |
+-----------------------------------------------------------------------------+

You may also list Attributes by name:

$ nsot attributes list --site-id 1 --name owner
+-----------------------------------------------------------------------------+
| ID   Name    Resource   Required?   Display?   Multi?   Description         |
+-----------------------------------------------------------------------------+
| 3    owner   Device     False       True       False    Owner of a device.  |
| 2    owner   Network    False       False      False    Owner of a network. |
+-----------------------------------------------------------------------------+

When listing a single Attribute by ID, you get more detail:

$ nsot attributes list --site-id 1 --id 3
+--------------------------------------------------------------------------------------+
| Name    Resource   Required?   Display?   Multi?   Constraints         Description   |
+--------------------------------------------------------------------------------------+
| owner   Device     False       False      False    pattern=            Device owner. |
|                                                    valid_values=                     |
|                                                    allow_empty=False                 |
+--------------------------------------------------------------------------------------+

Updating an Attribute:

$ nsot attributes update --site-id 1 --id 3 --no-required
[SUCCESS] Updated attribute with args: multi=None, description=None, required=False, display=None!

$ nsot attributes list --site-id 1 --id 3
+----------------------------------------------------------------------------+
| ID   Name    Resource   Required?   Display?   Multi?   Description        |
+----------------------------------------------------------------------------+
| 3    owner   Device     False       False      False    Owner of a device. |
+----------------------------------------------------------------------------+

Removing an Attribute:

$ nsot attributes remove --site-id 1 --id 6
[SUCCESS] Removed attribute with args: id=6!

Networks

A Network resource can represent an IP Network or an IP Address. Working with networks is usually done with CIDR notation. Networks can have any number of arbitrary Attributes.

Adding a Network:

$ nsot networks add --site-id 1 --cidr 192.168.0.0/16 --attributes owner=jathan
[SUCCESS] Added network with args: attributes={u'owner': u'jathan'}, cidr=192.168.0.0/16!

Listing Networks:

$ nsot networks list --site-id 1
+-------------------------------------------------------------------------+
| ID   Network       Prefix   Is IP?   IP Ver.   Parent ID   Attributes   |
+-------------------------------------------------------------------------+
| 1    192.168.0.0   16       False    4         None        owner=jathan |
| 2    10.0.0.0      16       False    4         None        owner=jathan |
| 3    172.16.0.0    12       False    4         None                     |
| 4    10.0.0.0      24       False    4         2                        |
| 5    10.1.0.0      24       False    4         2                        |
+-------------------------------------------------------------------------+

You may also optionally include IP addresses with --include-ips:

$ nsot networks list --side-id 1 --include-ips
+-------------------------------------------------------------------------+
| ID   Network       Prefix   Is IP?   IP Ver.   Parent ID   Attributes   |
+-------------------------------------------------------------------------+
| 1    192.168.0.0   16       False    4         None        owner=jathan |
| 2    10.0.0.0      16       False    4         None        owner=jathan |
| 3    172.16.0.0    12       False    4         None                     |
| 4    10.0.0.0      24       False    4         2                        |
| 5    10.1.0.0      24       False    4         2                        |
| 6    192.168.0.1   32       True     4         1                        |
+-------------------------------------------------------------------------+

Or, you may show only IP adddresses by using --include-ips with --no-include-networks:

$ nsot networks list --site-id 1 --include-ips --no-include-networks
+-----------------------------------------------------------------------+
| ID   Network       Prefix   Is IP?   IP Ver.   Parent ID   Attributes |
+-----------------------------------------------------------------------+
| 6    192.168.0.1   32       True     4         1                      |
+-----------------------------------------------------------------------+

Performing a set query on Networks by attribute/value:

$ nsot networks list --site-id 1 --query owner=jathan
10.0.0.0/16
192.168.0.0/16

You may also display the results comma-delimited:

$ nsot networks list --site-id 1 --query owner=jathan --delimited
10.0.0.0/16,192.168.0.0/16

Updating a Network (-a/--attributes can be provide once for each Attribute):

$ nsot networks update --site-id 1 --id 1 -a owner=jathan -a foo=bar
[SUCCESS] Updated network with args: attributes={u'owner': u'nobody', u'foo': u'bar'}!

$ nsot networks list --site-id 1 --id 1
+-------------------------------------------------------------------------+
| ID   Network       Prefix   Is IP?   IP Ver.   Parent ID   Attributes   |
+-------------------------------------------------------------------------+
| 1    192.168.0.0   16       False    4         None        owner=nobody |
|                                                            foo=bar      |
+-------------------------------------------------------------------------+

To delete attributes, reference each attribute by name and include the -d/--delete-attributes flag:

$ nsot networks update --site-id 1 --id 1 -a owner --delete-attributes

$ nsot networks list --site-id 1 --id 1
+-------------------------------------------------------------------------+
| ID   Network       Prefix   Is IP?   IP Ver.   Parent ID   Attributes   |
+-------------------------------------------------------------------------+
| 1    192.168.0.0   16       False    4         None        owner=nobody |
+-------------------------------------------------------------------------+

Removing a Network:

$ nsot networks remove --site-id 1 --id 2
[SUCCESS] Removed network with args: id=2!

Supernets

Given a Network 192.168.0.0/24:

$ nsot networks list --site-id 1 --id 6
+-----------------------------------------------------------------------+
| ID   Network       Prefix   Is IP?   IP Ver.   Parent ID   Attributes |
+-----------------------------------------------------------------------+
| 6    192.168.0.0   24       False    4         1                      |
+-----------------------------------------------------------------------+

You may view the networks that contain that Network (aka supernets):

$ nsot networks list --site-id 1 --id 5 supernets
+-------------------------------------------------------------------------+
| ID   Network       Prefix   Is IP?   IP Ver.   Parent ID   Attributes   |
+-------------------------------------------------------------------------+
| 1    192.168.0.0   16       False    4         None        owner=jathan |
|                                                            cluster=     |
|                                                            foo=baz      |
+-------------------------------------------------------------------------+

Subnets

Given the parent Network from the above example (192.168.0.0/16), you may the view Networks it contains (aka subnets):

$ nsot networks list --site-id 1 --id 1 subnets
+-----------------------------------------------------------------------+
| ID   Network       Prefix   Is IP?   IP Ver.   Parent ID   Attributes |
+-----------------------------------------------------------------------+
| 6    192.168.0.0   24       False    4         1                      |
| 7    192.168.0.0   25       False    4         6                      |
+-----------------------------------------------------------------------+

Devices

A Device represents various hardware components on your network such as routers, switches, console servers, PDUs, servers, etc.

Devices also support arbitrary attributes similar to Networks.

Adding a Device:

$ nsot devices add --site-id 1 --hostname foo-bar1 --attributes owner=neteng
[SUCCESS] Added device with args: attributes={u'owner': u'neteng'}, hostname=foo-bar1!

Listing Devices:

$ nsot devices list --site-id 1
+------------------------------+
| ID   Hostname   Attributes   |
+------------------------------+
| 1    foo-bar1   owner=jathan |
| 2    foo-bar2   owner=neteng |
| 3    bar-baz1   owner=jathan |
| 4    bar-baz2   owner=neteng |
+------------------------------+

Performing a set query on Device by attribute/value:

$ nsot devices list --site-id 1 --query owner=neteng
bar-baz2
foo-bar2

You may also display the results comma-delimited:

$ nsot devices list --site-id 1 --query owner=neteng --delimited
bar-baz2,foo-bar2

Updating a Device:

$ nsot devices update --id 1 --hostname potato
[SUCCESS] Updated device with args: attributes={}, hostname=potato!

$ ./nsot devices list --site-id 1 --id 1
+----------------------------+
| ID   Hostname   Attributes |
+----------------------------+
| 1    potato                |
+----------------------------+

To delete attributes, reference each attribute by name and include the -d/--delete-attributes flag:

$ nsot devices update --site-id 1 --id 2 -a owner --delete-attributes

$ nsot devices list --site-id 1 --id 2
+------------------------------+
| ID   Hostname   Attributes   |
+------------------------------+
| 2    foo-bar2                |
+------------------------------+

Removing a Device:

$ nsot devices remove --site-id 1 --id 1
[SUCCESS] Removed device with args: id=1!

Changes

All Create/Update/Delete events are logged as a Change. A Change includes information such as the change time, user, and the full resource after modification. Changes are immutable and can only be removed by deleting the entire Site.

Listing Changes:

$ nsot changes list --site-id 1 --limit 5
+-----------------------------------------------------------------------+
| ID   Change At             User               Event    Resource   Obj |
+-----------------------------------------------------------------------+
| 73   2015-03-04 11:12:30   jathan@localhost   Delete   Device     1   |
| 72   2015-03-04 11:10:46   jathan@localhost   Update   Device     1   |
| 71   2015-03-04 11:06:03   jathan@localhost   Create   Device     7   |
| 70   2015-03-04 10:56:54   jathan@localhost   Update   Network    6   |
| 69   2015-03-04 10:53:30   jathan@localhost   Create   Network    6   |
+-----------------------------------------------------------------------+

When listing a single Change event by ID, you get more detail:

$ nsot changes list --site-id 1 --id 73
+-----------------------------------------------------------------------------------+
| Change At             User               Event    Resource   ID   Data            |
+-----------------------------------------------------------------------------------+
| 2015-03-04 11:12:30   jathan@localhost   Delete   Device     1    attributes:     |
|                                                                   hostname:potato |
|                                                                   site_id:1       |
|                                                                   id:1            |
+-----------------------------------------------------------------------------------+

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pynsot-0.19.tar.gz (61.3 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page