Skip to main content

Python interface for Network Source of Truth (nsot)

Project description

######
pynsot
######

.. image:: https://travis-ci.org/dropbox/pynsot.svg
:target: https://travis-ci.org/dropbox/pynsot

Python client library and command-line utility for the `Network Source of
Truth <https://github.com/dropbox/nsot>`_ (NSoT) REST API.

Warning
=======

This project is still very much in flux and likely to have
backwards-incompatible changes as it evolves with the API for the `Network
Source of Truth <https://github.com/dropbox/nsot>`_ project.

Using
=====

All of the endpoints are dynamic, so it's like magic. Literally::

$ python
>>> from pynsot.client import EmailHeaderClient
>>> email = 'jathan@localhost'
>>> url = 'http://localhost:8990/api'
>>> api = EmailHeaderClient(url, email=email)
>>> print api.users.get()
>>> print api.sites(1).get()
>>> print api.sites(1).attributes.get()

Authentication
--------------

The default method of authentication for NSoT is ``auth_token`` and this is what
the client uses. You'll need to retreive your ``secret_key`` from the NSoT web
interface for this to work::

$ python
>>> from pynsot.client import Client
>>> email = 'jathan@localhost'
>>> secret_key = 'qONJrNpTX0_9v7H_LN1JlA0u4gdTs4rRMQklmQF9WF4='
>>> url = 'http://localhost:8990/api'
>>> api = Client(url, email=email, secret_key=secret_key)

Config
------

By default, the ``nsot`` command-line utility will read your settings from
``~/.pynsotrc``, which looks like this::

$ cat ~/.pynsotrc
[pynsot]
url = http://localhost:8990/api
email = jathan@localhost
secret_key = qONJrNpTX0_9v7H_LN1JlA0u4gdTs4rRMQklmQF9WF4=
auth_method = auth_token

You can create it yourself, or you'll be prompted to create one if it isn't
found the first time you use the ``nsot`` utility::

$ nsot sites list
/home/jathan/.pynsotrc not found; would you like to create it? [Y/n]: y
Please enter URL: http://localhost:8990/api
Please enter SECRET_KEY: qONJrNpTX0_9v7H_LN1JlA0u4gdTs4rRMQklmQF9WF4=
Please enter EMAIL: jathan@localhost

.. _default_site:

Default Site
~~~~~~~~~~~~

You may optionally specify a ``default_site`` setting within your
``~/.pynsotrc`` that will pass along the designated ``site_id`` on your behalf
any time the ``-s/--site-id`` option is required::

[pynsot]
; ... Other settings ...
default_site = 1

Since most users will only have a single site they care about, this is to make
your life easier. If you do have multiple sites, you can still provide the
``-s/--site-id`` option to override your ``default_site``.

CLI Usage
=========

Every resource type has four eligible actions representing creation,
retrieval, update, and removal:

+ add
+ list
+ remove
+ update

.. note::
The only exception are Change events, which are immutable and can only be
viewed.

Each resource can be manipulated using positional command arguments::

$ nsot
Usage: nsot [OPTIONS] COMMAND [ARGS]...

Network Source of Truth (NSoT) command-line utility.

Options:
-v, --verbose Toggle verbosity.
--version Show the version and exit.
-h, --help Show this message and exit.

Commands:
attributes Attribute objects.
changes Change events.
devices Device objects.
networks Network objects.
sites Site objects.

The actions for each resource have varying options and requirements which will
be covered a little later. For example, to work with Sites you would run ``nsot
sites``::

$ nsot sites
Usage: nsot sites [OPTIONS] COMMAND [ARGS]...

Site objects.

Sites are the top-level resource from which all other resources descend.
In other words, Sites contain Attributes, Changes, Devices, and Networks.

Options:
-h, --help Show this message and exit.

Commands:
add Add a new Site.
list List existing Sites.
remove Remove a Site.
update Update a Site.

Getting Help
------------

Every resource and action for each resource has help text that can be accessed
using the ``-h/--help`` option. Use it!

Required Options
----------------

When adding objects, certain fields will be required. The required options will
be designated as such with a ``[required]`` tag in the help text (for example
from ``nsot sites add --help``::

-n, --name NAME The name of the Site. [required]

If a required option is not provided, ``nsot`` will complain::

Error: Missing option "-n" / "--name".

Site ID
-------

For all resources other than Sites, the ``-s/--site-id`` option is required to
specify which Site you would like the object to be under. (See:
`default_site`)

Updating or Removing Objects
----------------------------

When updating or removing objects, you must provide their unique ID using the
``-i/--id`` option. The object's ID can be obtained by using the ``list``
action.

Updating Attributes
~~~~~~~~~~~~~~~~~~~

When modifying attributes you have three actions to choose from:

+ Add (``-A/--add-attributes``). This is the default behavior that will add
attributes if they don't exist, or update them if they do.

+ Delete (``-d/--delete-attributes``). This will cause attributes to be
deleted. If combined with ``--multi`` the attribute will be deleted if either
no value is provided, or if the attribute no longer contains a valid value.

+ Replace (``-r/--replace-attributes``). This will cause attributes to
replaced. If combined with ``-m/--multi`` and multiple attributes of the same
name are provided, only the last value provided will be used.

Viewing Objects
---------------

Each resource's ``list`` action supports ``-i/--id``, ``-l/--limit`` and
``-o/--offset`` options.

+ The ``-i/--id`` option will retrieve a single object by the provided unique
ID and will override any other list options.
+ The ``-l/--limit`` option will limit the set of results to ``N`` resources.
+ The ``-o/--offset`` option will skip the first ``N`` resources.

Set Queries
~~~~~~~~~~~

The Device and Network resources support a ``-q/--query`` option that is a
representation of set operations for matching attribute/value pairs.

The operations are evaluated from left-to-right, where the first character indicates the
set operation:

+ "+" indicates a set union
+ "-" indicates a set difference
+ no marker indicates a set intersection

For example

+ ``-q "foo=bar"`` would return the set intersection of objects with ``foo=bar``.
+ ``-q "foo=bar -owner=jathan"`` would return the set difference of all objects
with ``foo=bar`` (that is all ``foo=bar`` where ``owner`` is not ``jathan``).
+ ``-q "foo=bar +foo=baz`` would return the set union of all objects with
``foo=bar`` or ``foo=baz`` (that is all objects matching either).

The ordering of these operations is important. If you are not familiar with set
operations, please check out `Basic set theory concepts and notation
<http://en.wikipedia.org/wiki/Set_theory#Basic_concepts_and_notation>`_
(Wikipedia).

Bulk Addition of Objects
------------------------

Attributes, Devices, and Networks may be created in bulk by using the
``-b/--bulk-add`` option and specifying a file path to a colon-delimited file.

The format of this file must adhere to the following format:

+ The first line of the file must be the field names.
+ All required fields must be present, however, the order of any of the fields
does not matter.
+ Repeat: The fields may be in any order so long as the required fields are
present! Missing fields will fallback to their defaults!
+ Attribute pairs must be commma-separated, and in format k=v and the
attributes must exist!
+ For any fields that require Boolean values, the following applies:

- You may specify ``True`` or ``False`` and they will be evaluated
- If the value for a field is not set it will evaluate to ``False``
- Any other value for a field will evaluate to ``True``

Attributes
~~~~~~~~~~

Sample file for ``nsot devices add --bulk-add /tmp/attributes``::

name:resource_name:required:description:multi:display
owner:Network:True:Network owner:True:True
metro:Device:False:Device metro:False:True

Devices
~~~~~~~

Sample file for ``nsot devices add --bulk-add /tmp/devices``::

hostname:attributes
device5:foo=bar,owner=team-networking
device6:foo=bar,owner=team-networking

Networks
~~~~~~~~

Sample file for ``nsot networks add --bulk-add /tmp/networks``::

cidr:attributes
10.20.30.0/24:foo=bar,owner=team-networking
10.20.31.0/24:foo=bar,owner=team-networking

Working with Resources
======================

Sites
-----

Sites are the top-level resource from which all other resources descend. In
other words, Sites contain Networks, Attributes, Devices, etc. These examples
illustrate having many Sites.

Adding a site::

$ nsot sites add --name Spam --description 'Spam Site'
[SUCCESS] added site with args: name=Space, description=Spam Site!

Listing all Sites::

$ nsot sites list
+--------------------------+
| ID Name Description |
+--------------------------+
| 1 Foo Foo Site |
| 2 Bar Bar Site |
| 3 Baz Baz Site |
| 4 Spam Sheep Site |
| 5 Sheep Sheep Site |
+--------------------------+

Listing a single Site::

$ nsot sites list --name Foo
+-------------------------+
| ID Name Description |
+-------------------------+
| 1 Foo Foo Site |
+-------------------------+

Listing a few Sites::

$ nsot sites list --limit 2
+--------------------------+
| ID Name Description |
+--------------------------+
| 1 Foo Foo Site |
| 2 Bar Bar Site |
+--------------------------+

Updating a Site::

$ nsot sites update --id 2 --name Snickers
[SUCCESS] updated site with args: description=None, name=Snickers!

$ nsot sites list --name Snickers
+-----------------------------+
| ID Name Description |
+-----------------------------+
| 2 Snickers Bar Site |
+-----------------------------+

Removing a Site::

$ nsot sites remove --id 1
[SUCCESS] removed site with args: id=1!

Attributes
----------

Attributes are flexible key/value pairs or tags you may use to assign arbitrary
data to objects.

.. note::
Before you may assign Attributes to other resources, you must create the
Attribute first!

Adding an Attribute::

$ nsot attributes add --site-id 1 -n owner --r Device -d "Owner of a device." --required
[SUCCESS] Added attribute with args: multi=False, resource_name=Device, name=owner, required=True, display=False, description=Owner of a device.!

Listing all Attributes::

$ nsot attributes list --site-id 1
+-----------------------------------------------------------------------------+
| ID Name Resource Required? Display? Multi? Description |
+-----------------------------------------------------------------------------+
| 3 owner Device True False False Owner of a device. |
| 4 foo Network False False False Foo for devices |
| 2 owner Network False False False Owner of a network. |
+-----------------------------------------------------------------------------+

You may also list Attributes by name::

$ nsot attributes list --site-id 1 --name owner
+-----------------------------------------------------------------------------+
| ID Name Resource Required? Display? Multi? Description |
+-----------------------------------------------------------------------------+
| 3 owner Device False True False Owner of a device. |
| 2 owner Network False False False Owner of a network. |
+-----------------------------------------------------------------------------+

When listing a single Attribute by ID, you get more detail::

$ nsot attributes list --site-id 1 --id 3
+--------------------------------------------------------------------------------------+
| Name Resource Required? Display? Multi? Constraints Description |
+--------------------------------------------------------------------------------------+
| owner Device False False False pattern= Device owner. |
| valid_values= |
| allow_empty=False |
+--------------------------------------------------------------------------------------+

Updating an Attribute::

$ nsot attributes update --site-id 1 --id 3 --no-required
[SUCCESS] Updated attribute with args: multi=None, description=None, required=False, display=None!

$ nsot attributes list --site-id 1 --id 3
+----------------------------------------------------------------------------+
| ID Name Resource Required? Display? Multi? Description |
+----------------------------------------------------------------------------+
| 3 owner Device False False False Owner of a device. |
+----------------------------------------------------------------------------+

Removing an Attribute::

$ nsot attributes remove --site-id 1 --id 6
[SUCCESS] Removed attribute with args: id=6!

Networks
--------

A Network resource can represent an IP Network or an IP Address. Working with
networks is usually done with CIDR notation. Networks can have any number of
arbitrary Attributes.

Adding a Network::

$ nsot networks add --site-id 1 --cidr 192.168.0.0/16 --attributes owner=jathan
[SUCCESS] Added network with args: attributes={u'owner': u'jathan'}, cidr=192.168.0.0/16!

Listing Networks::

$ nsot networks list --site-id 1
+-------------------------------------------------------------------------+
| ID Network Prefix Is IP? IP Ver. Parent ID Attributes |
+-------------------------------------------------------------------------+
| 1 192.168.0.0 16 False 4 None owner=jathan |
| 2 10.0.0.0 16 False 4 None owner=jathan |
| 3 172.16.0.0 12 False 4 None |
| 4 10.0.0.0 24 False 4 2 |
| 5 10.1.0.0 24 False 4 2 |
+-------------------------------------------------------------------------+

You may also optionally include IP addresses with ``--include-ips``::

$ nsot networks list --side-id 1 --include-ips
+-------------------------------------------------------------------------+
| ID Network Prefix Is IP? IP Ver. Parent ID Attributes |
+-------------------------------------------------------------------------+
| 1 192.168.0.0 16 False 4 None owner=jathan |
| 2 10.0.0.0 16 False 4 None owner=jathan |
| 3 172.16.0.0 12 False 4 None |
| 4 10.0.0.0 24 False 4 2 |
| 5 10.1.0.0 24 False 4 2 |
| 6 192.168.0.1 32 True 4 1 |
+-------------------------------------------------------------------------+

Or, you may show only IP adddresses by using ``--include-ips`` with
``--no-include-networks``::

$ nsot networks list --site-id 1 --include-ips --no-include-networks
+-----------------------------------------------------------------------+
| ID Network Prefix Is IP? IP Ver. Parent ID Attributes |
+-----------------------------------------------------------------------+
| 6 192.168.0.1 32 True 4 1 |
+-----------------------------------------------------------------------+

Performing a set query on Networks by attribute/value::

$ nsot networks list --site-id 1 --query owner=jathan
10.0.0.0/16
192.168.0.0/16

You may also display the results comma-delimited::

$ nsot networks list --site-id 1 --query owner=jathan --delimited
10.0.0.0/16,192.168.0.0/16

Updating a Network (``-a/--attributes`` can be provide once for each Attribute)::

$ nsot networks update --site-id 1 --id 1 -a owner=jathan -a foo=bar
[SUCCESS] Updated network with args: attributes={u'owner': u'nobody', u'foo': u'bar'}!

$ nsot networks list --site-id 1 --id 6
+-------------------------------------------------------------------------+
| ID Network Prefix Is IP? IP Ver. Parent ID Attributes |
+-------------------------------------------------------------------------+
| 1 192.168.0.0 16 False 4 None owner=nobody |
| foo=bar |
+-------------------------------------------------------------------------+

Removing a Network::

$ nsot networks remove --site-id 1 --id 2
[SUCCESS] Removed network with args: id=2!

Supernets
~~~~~~~~~

Given a Network ``192.168.0.0/24``::

$ nsot networks list --site-id 1 --id 6
+-----------------------------------------------------------------------+
| ID Network Prefix Is IP? IP Ver. Parent ID Attributes |
+-----------------------------------------------------------------------+
| 6 192.168.0.0 24 False 4 1 |
+-----------------------------------------------------------------------+

You may view the networks that contain that Network (aka supernets)::

$ nsot networks list --site-id 1 --id 5 supernets
+-------------------------------------------------------------------------+
| ID Network Prefix Is IP? IP Ver. Parent ID Attributes |
+-------------------------------------------------------------------------+
| 1 192.168.0.0 16 False 4 None owner=jathan |
| cluster= |
| foo=baz |
+-------------------------------------------------------------------------+

Subnets
~~~~~~~

Given the parent Network from the above example (``192.168.0.0/16``), you may
the view Networks it contains (aka subnets)::

$ nsot networks list --site-id 1 --id 1 subnets
+-----------------------------------------------------------------------+
| ID Network Prefix Is IP? IP Ver. Parent ID Attributes |
+-----------------------------------------------------------------------+
| 6 192.168.0.0 24 False 4 1 |
| 7 192.168.0.0 25 False 4 6 |
+-----------------------------------------------------------------------+

Devices
-------

A Device represents various hardware components on your network such as
routers, switches, console servers, PDUs, servers, etc.

Devices also support arbitrary attributes similar to Networks.

Adding a Device::

$ nsot devices add --site-id 1 --hostname foo-bar1 --attributes owner=neteng
[SUCCESS] Added device with args: attributes={u'owner': u'neteng'}, hostname=foo-bar1!

Listing Devices::

$ nsot devices list --site-id 1
+------------------------------+
| ID Hostname Attributes |
+------------------------------+
| 1 foo-bar1 owner=jathan |
| 2 foo-bar2 owner=neteng |
| 3 bar-baz1 owner=jathan |
| 4 bar-baz2 owner=neteng |
+------------------------------+

Performing a set query on Device by attribute/value::

$ nsot devices list --site-id 1 --query owner=neteng
bar-baz2
foo-bar2

You may also display the results comma-delimited::

$ nsot devices list --site-id 1 --query owner=neteng --delimited
bar-baz2,foo-bar2

Updating a Device::

$ nsot devices update --id 1 --hostname potato
[SUCCESS] Updated device with args: attributes={}, hostname=potato!

$ ./nsot devices list --site-id 1 --id 1
+----------------------------+
| ID Hostname Attributes |
+----------------------------+
| 1 potato |
+----------------------------+

Removing a Device::

$ nsot devices remove --site-id 1 --id 1
[SUCCESS] Removed device with args: id=1!

Changes
-------

All Create/Update/Delete events are logged as a Change. A Change includes
information such as the change time, user, and the full resource after
modification. Changes are immutable and can only be removed by deleting the
entire Site.

Listing Changes::

$ nsot changes list --site-id 1 --limit 5
+-----------------------------------------------------------------------+
| ID Change At User Event Resource Obj |
+-----------------------------------------------------------------------+
| 73 2015-03-04 11:12:30 jathan@localhost Delete Device 1 |
| 72 2015-03-04 11:10:46 jathan@localhost Update Device 1 |
| 71 2015-03-04 11:06:03 jathan@localhost Create Device 7 |
| 70 2015-03-04 10:56:54 jathan@localhost Update Network 6 |
| 69 2015-03-04 10:53:30 jathan@localhost Create Network 6 |
+-----------------------------------------------------------------------+

When listing a single Change event by ID, you get more detail::

$ nsot changes list --site-id 1 --id 73
+-----------------------------------------------------------------------------------+
| Change At User Event Resource ID Data |
+-----------------------------------------------------------------------------------+
| 2015-03-04 11:12:30 jathan@localhost Delete Device 1 attributes: |
| hostname:potato |
| site_id:1 |
| id:1 |
+-----------------------------------------------------------------------------------+

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

pynsot-0.16.tar.gz (49.6 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page