Skip to main content

Library for working with the PuppetDB REST API.

Project description


.. image::

.. image::

pypuppetdtb is a library to work with PuppetDB's REST API. It is implemented
using the `requests`_ library.
.. _requests:

**pypuppetdb >= 0.2.0 requires PuppetDB 3.0.0 or later. There is no support for
previous versions beyond 0.1.1**

**pypuppetdb >= 0.2.2 supports PuppetDB 4.0.0. Backwards compatability with 3.x
is available.**

This library is a thin wrapper around the REST API providing some convinience
functions and objects to request and hold data from PuppetDB.

To use this library you will need:
* Python 2.6 or 2.7
* Python 3.3


You can install this package from source or from PyPi.

.. code-block:: bash

$ pip install pypuppetdb

.. code-block:: bash

$ git clone
$ python install

If you wish to hack on it clone the repository but after that run:

.. code-block:: bash

$ pip install -r requirements.txt

This will install all the runtime requirements of pypuppetdb and the
dependencies for the test suite and generation of documentation.

Native packages for your operating system will be provided in the near future.

| OS | Status | |
| Debian 6/Squeeze | planned | Requires Backports |
| Debian 7/Wheezy | planned | |
| Ubuntu 13.04 | planned | |
| Ubuntu 13.10 | planned | |
| CentOS/RHEL 5 | n/a | Python 2.4 |
| CentOS/RHEL 6 | planned | |
| CentOS/RHEL 7 | planned | |
| `ArchLinux`_ | available | Maintained by `Tim Meusel`_ |
| `OpenBSD`_ | available | Maintained by `Jasper Lievisse Adriaanse`_ |

.. _ArchLinux:
.. _Tim Meusel:
.. _Jasper Lievisse Adriaanse:
.. _OpenBSD:


Once you have pypuppetdb installed you can configure it to connect to PuppetDB
and take it from there.


The first thing you need to do is to connect with PuppetDB:

.. code-block:: python

>>> from pypuppetdb import connect
>>> db = connect()


The following will return a generator object yielding Node objects for every
returned node from PuppetDB.

.. code-block:: python

>>> nodes = db.nodes()
>>> for node in nodes:
>>> print(node)

To query a single node the singular `node()` can be used:

.. code-block:: python

>>> node = db.node('hostname')
>>> print(node)

Node scope

The Node objects are a bit more special in that they can query for facts and
resources themselves. Using those methods from a node object will automatically
add a query to the request scoping the request to the node.

.. code-block:: python

>>> node = db.node('hostname')
>>> print(node.fact('osfamily'))


.. code-block:: python

>>> facts = db.facts('osfamily')
>>> for fact in facts:
>>> print(fact)

That queries PuppetDB for the 'osfamily' fact and will yield Fact objects,
one per node this fact is known for.


.. code-block:: python

>>> resources = db.resources('file')

Will return a generator object containing all file resources you're managing
across your infrastructure. This is probably a bad idea if you have a big
number of nodes as the response will be huge.


.. code-block:: python

>>> catalog = db.catalog('hostname')
>>> for res in catalog.get_resources():
>>> print(res)

Will return a Catalog object with the latest Catalog of the definded host. This
catalog contains the defined Resources and Edges.

.. code-block:: python

>>> catalog = db.catalog('hostname')
>>> resource = catalog.get_resource('Service','ntp')
>>> for rel in resource.relationships:
>>> print(rel)
Class[Ntp] - contains - Service[ntp]
File[/etc/ntp.conf] - notifies - Service[ntp]
File[/etc/ntp.conf] - required-by - Service[ntp]

Will return all Relationships of a given Resource defined by type and title.
This will list all linked other Resources and the type of relationship.

Query Builder

Starting with version 0.3.0 pypuppetdb comes shipped with a QueryBuilder component
that, as the name suggests, allows users to build PuppetDB AST queries in an
Object-Oriented fashion. Vastly superior to constructing long strings than adding
additional clauses to fulfill new requirements.

The following code will build a query for the Nodes endpoint to find all nodes
belonging to the production environment.

.. code-block:: python

>>> from pypuppetdb.QueryBuilder import *
>>> op = AndOperator()
>>> op.add(EqualsOperator('catalog_environment', 'production'))
>>> op.add(EqualsOperator('facts_environment', 'production'))
>>> print(op)
["and",["=", "catalog_environment", "production"],["=", "facts_environment", "production"]]

This functionality is based on the PuppetDB AST query string syntax documented

.. _here:

Subqueries are implemented using corresponding operators (like documented).

* SubqueryOperator
* InOperator
* ExtractOperator

.. code-block:: python

>>> from pypuppetdb.QueryBuilder import *
>>> op = InOperator('certname')
>>> ex = ExtractOperator()
>>> ex.add_field(str('certname'))
>>> sub = SubqueryOperator('events')
>>> sub.add_query(EqualsOperator('status', 'noop'))
>>> ex.add_query(sub)
>>> op.add_query(ex)
>>> print(op)
["in","certname",["extract",["certname"],["select_events",["=", "status", "noop"]]]]

Getting Help
This project is still very new so it's not inconceivable you'll run into

For bug reports you can file an `issue`_. If you need help with something
feel free to pop by #voxpupuli on `Freenode`_ or the #puppetboard channel.

.. _issue:
.. _Freenode:

API documentation is automatically generated from the docstrings using
Sphinx's autodoc feature.

Documentation will automatically be rebuilt on every push thanks to the
Read The Docs webhook. You can `find it here`_.

.. _find it here:

You can build the documentation manually by doing:

.. code-block:: bash

$ cd docs
$ make html

Doing so will only work if you have Sphinx installed, which you can acheive

.. code-block:: bash

$ pip install -r requirements.txt


We welcome contributions to this library. However, there are a few ground
rules contributors should be aware of.

This project is licensed under the Apache v2.0 License. As such, your
contributions, once accepted, are automatically covered by this license.

Copyright (c) 2013-2014 Daniele Sluijters

Commit messages
Write decent commit messages. Don't use swear words and refrain from
uninformative commit messages as 'fixed typo'.

The preferred format of a commit message:


docs/quickstart: Fixed a typo in the Nodes section.

If needed, elaborate further on this commit. Feel free to write a
complete blog post here if that helps us understand what this is
all about.

Fixes #4 and resolves #2.

If you'd like a more elaborate guide on how to write and format your commit
messages have a look at this post by `Tim Pope`_.

.. _Tim Pope:

Commits are expected to contain tests or updates to tests if they add to or
modify the current behaviour.

The test suite is powered by `pytest`_ and requires `pytest`_, `pytest-pep8`_,
`httpretty`_ and `pytest-httpretty`_ which will be installed for you if you

.. code-block:: bash

$ pip install -r requirements.txt

.. _pytest:
.. _pytest-pep8:
.. _httpretty:
.. _pytest-httpretty:

To run the unit tests (the ones that don't require a live PuppetDB):

.. code-block:: bash

$ py.test -v -m unit

If the tests pass, you're golden. If not we'll have to figure out why and
fix that. Feel free to ask for help on this.



* Add support for authentication with tokens
* Fix bug with parsing results from inventory endpoint


* Fixed noop puppet runs reporting unchanged instead of noop.
* Fixed unreported nodes shown as 'noop' in puppetdb > 4.1.0.
* Add Inventory API endpoint for PuppetDB 4.2.0.
* Support for producer field on catalogs, facts and report types.


* Fixed a datetime related bug in :func:`pypuppetdb.api.nodes()` that caused
all returned nodes to be an unreported status


* New QueryBuilder module allows users to build PuppetDB queries in an
Object-Oriented fashion.
* Adding support for new fields provided in PuppetDB 4.1.0.


* Removed deprecation of :func:``. Expanded
resource events data timestamps are not parseable.
* Escaping additional path parameters passed to _url() with urllib.quote


* Fixed URL Encoding found when querying the specific value of a macaddress
* Adding support for PuppetDB 4.0.0 information. Namely Adding a catalog_uuid
attribute to the Catalog type object. Adding code_id, catalog_uuid and
cached_catalog_status attributes to the Report type object.
* Removing unneeded sudo option from .travis.yml, this gave unnecessary
warning in the test environment.
* Updating the files under docs/ so
can be updated
* Deprecating :func:`` in favour of the new
events list variable.
* Renaming test-requirements.txt to requirements.txt


* Adding a version comparison utility function using examples provided in
* Adding a new variable latest_report_hash to the Node object. Default
None but is given a real value from the field of the same name in the
Nodes endpoint available in PuppetDB 3.2 or higher.
* Allowing support for 'GET' AND 'POST' requests in the api _query()
function. This will allow clients to send requests to the PuppetDB that
are too long for a GEt request query string
* Adding a node field, code_id, to the Catalog object using the field of
the same name from the Catalogs endpoint (currently unused as of
PuppetDB 3.2.2)
* Adding test cases for new features EXCEPT the GET and POST update.


* Version bump to 0.2.0
* Adding support for v4 of the Query API
* Removing v2 and v3 api functions as per changelog
* pypuppetdb will no longer support multiple API versions, removing the
api_version attribute from pypuppetdb.connect()
* All clients must remove the api_version attribute from the connect function,
or the starting number, since it is no longer supported
* Removing all NotImplemented errors in the function of BaseAPI and filled
them with the real code

New Features

New endpoints:

* ``environments``: ``environments()``
* ``factsets``: ``factsets()``
* ``fact-paths``: ``fact_paths()``
* ``fact-contents``: ``fact_contents()``
* ``edges``: ``edges()``

Changes to Types:

* ``pypupperdb.types.Report`` now requires ``api`` to be passed as the second
argument, this allows to directly query for any events that occurred in this
report object. This functionality was proposed and denied because of backward
compatability reasons, since the previous versions are now removed this is no
longer a problem.
* All ``pypupperdb.types.*`` accept the v4 API information as optional parameters.
These parameters are primarily environment related but may include additional
information if provided from that endpoint.
* Functions appearing inside ``pypuppetdb.types`` that run queries against the
PuppetDB now accept and passing additional keyword arguments to the query.
* All ``pypuppetdb.BaseAPI`` functions pass any received keyword arguments to the
``pypuppetdb.api.__init__._query()`` function. This allows for easy integration
with paging functions and parameters.


* Fix the license in our ````. The license shouldn't be longer than
200 characters. We were including the full license tripping up tools like

Significant changes have been made in this release. The complete v3 API is
now supported except for query pagination.

Most changes are backwards compatible except for a change in the SSL
configuration. The previous behaviour was buggy and slightly misleading in
the names the options took:

* ``ssl`` has been renamed to ``ssl_verify`` and now defaults to ``True``.
* Automatically use HTTPS if ``ssl_key`` and ``ssl_cert`` are provided.

For additional instructions about getting SSL to work see the Quickstart
in the documentation.

Support for API v2 will be dropped in the 0.2.x release series.

New features

The following features are **only** supported for **API v3**.

The ``node()`` and ``nodes()`` function have gained the following options:

* ``with_status=False``
* ``unreported=2``

When ``with_status`` is set to ``True`` an additional query will be made using
the ``events-count`` endpoint scoped to the latest report. This will result in
an additional ``events`` and ``status`` keys on the node object. ``status``
will be either of ``changed``, ``unchanged`` or ``failed`` depending on if
``events`` contains ``successes`` or ``failures`` or none.

By default ``unreported`` is set to ``2``. This is only in effect when
``with_status`` is set to ``True``. It means that if a node hasn't checked in
for two hours it will get a ``status`` of ``unreported`` instead.

New endpoints:

* ``events-count``: ``events_count()``
* ``aggregate-event-counts``: ``aggregate_event_counts()``
* ``server-time``: ``server_time()``
* ``version``: ``current_version()``
* ``catalog``: ``catalog()``

New types:

* ``pypuppetdb.types.Catalog``
* ``pypuppetdb.types.Edge``

Changes to types:

* ``pypuppetdb.types.Node`` now has:
* ``status`` defaulting to ``None``
* ``events`` defaulting to ``None``
* ``unreported_time`` defaulting to ``None``


Due to a fairly serious bug 0.0.3 was pulled from PyPi minutes after release.

When a bug was fixed to be able to query for all facts we accidentally
introduced a different bug that caused the ``facts()`` call on a node to
query for all facts because we were resetting the query.

* Fix a bug where ``node.facts()`` was causing us to query all facts because
the query to scope our request was being reset.


With the introduction of PuppetDB 1.5 a new API version, v3, was also
introduced. In that same release the old ``/experimental`` endpoints
were removed, meaning that as of PuppetDB 1.5 with the v2 API you can
no longer get access to reports or events.

In light of this the support for the experimental endpoints has been
completely removed from pypuppetdb. As of this release you can only get
to reports and/or events through v3 of the API.

This release includes preliminary support for the v3 API. Everything that
could be done with v2 plus the experimental endpoints is now possible on
v3. However, more advanced funtionality has not yet been implemented. That
will be the focus of the next release.

* Removed dependency on pytz.
* Fixed the behaviour of ``facts()`` and ``resources()``. We can now
correctly query for all facts or resources.
* Fixed an issue with catalog timestampless nodes.
* Pass along the ``timeout`` option to ``connect()``.
* Added preliminary PuppetDB API v3 support.
* Removed support for the experimental endpoints.
* The ``connect()`` method defaults to API v3 now.

* Fix a bug in ```` preventing successful installation.

Initial release. Implements most of the v2 API.

Project details

Release history Release notifications

This version
History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


History Node


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
pypuppetdb-0.3.3-py2.py3-none-any.whl (36.1 kB) Copy SHA256 hash SHA256 Wheel py2.py3 Oct 15, 2017
pypuppetdb-0.3.3.tar.gz (38.3 kB) Copy SHA256 hash SHA256 Source None Oct 15, 2017

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging CloudAMQP CloudAMQP RabbitMQ AWS AWS Cloud computing Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page