puppetboard 3.2.0rc1

Web frontend for PuppetDB

Puppetboard

Puppetboard is a web interface to PuppetDB aiming to replace the reporting functionality of Puppet Enterprise console (previously: Puppet Dashboard) for the open source Puppet.

(See more screenshots here.)

Table of Contents

• Requirements
• Installation
• Configuration
• Getting Help
• Contributing

Requirements

• PuppetDB v. 3.0-7.5 (will most probably work with newer, but this has not been tested yet)
• Python 3.6 / 3.7 / 3.8 / 3.9 / 3.10

Installation

Puppetboard is packaged and available on PyPi.

With Puppet module

There is a Puppet module originally written by Spencer Krum and currently maintained by Voxpupuli that takes care of installing the Puppetboard for you.

To see how to get it working with RedHat/Centos 7 check out these docs.

Using Docker

We provide an official Docker image in the GitHub Container Registry.

You can run the app using it with:

docker pull ghcr.io/voxpupuli/puppetboard

docker run -it -p 9080:80 -v /etc/puppetlabs/puppet/ssl:/etc/puppetlabs/puppet/ssl \
-e PUPPETDB_HOST=<hostname> \
-e PUPPETDB_PORT=8081 \
-e PUPPETDB_SSL_VERIFY=/etc/puppetlabs/puppetdb/ssl/ca.pem \
-e PUPPETDB_KEY=/etc/puppetlabs/puppetdb/ssl/private.pem \
-e PUPPETDB_CERT=/etc/puppetlabs/puppetdb/ssl/public.pem \
ghcr.io/voxpupuli/puppetboard

We also provide the Dockerfile so you can build the image yourself:

docker build -t puppetboard .

From a package

Actively maintained packages:

• FreeBSD
• OpenBSD maintained by Sebastian Reitenbach

Manually

You can also install the package from PyPI and configure a WSGI-capable application server to serve it.

We recommend using virtualenv to provide a separate environment for the app.

virtualenv -p python3 venv
. venv/bin/activate
pip install puppetboard

Please see an article about more deployment setups here.

Configuration

PuppetDB

Of course you need to configure your Puppet Server to store the Puppet run reports in PuppetDB. If you haven't done that already please follow the PuppetDB documentation about this.

If you run Puppetboard on a different host than PuppetDB then you may want to configure the certificate allow-list for which certificates are allowed to access data from PuppetDB. Please read more about this feature in the PuppetDB documentation here.

App settings

Puppetboard will look for a file pointed at by the PUPPETBOARD_SETTINGS environment variable. The file has to be identical to default_settings.py but should only override the settings you need changed.

If you run PuppetDB and Puppetboard on the same machine the default settings provided will be enough to get you started and you won't need a custom settings file.

Assuming your webserver and PuppetDB machine are not identical you will at least have to change the following settings:

• PUPPETDB_HOST
• PUPPETDB_PORT

By default PuppetDB requires SSL to be used when a non-local client wants to connect. Therefore you'll also have to supply the following settings:

• PUPPETDB_SSL_VERIFY = /path/to/ca/keyfile.pem
• PUPPETDB_KEY = /path/to/private/keyfile.pem
• PUPPETDB_CERT = /path/to/public/keyfile.crt

For information about how to generate the correct keys please refer to the pypuppetdb documentation. Alternatively it is possible to explicitly specify the protocol to be used setting the PUPPETDB_PROTO variable.

Other settings that might be interesting in no particular order:

• SECRET_KEY: Refer to Flask documentation, section "How to generate good secret keys" for more info. Defaults to a random 24-char string generated by os.random(24).
• PUPPETDB_TIMEOUT: Defaults to 20 seconds but you might need to increase this value. It depends on how big the results are when querying PuppetDB. This behaviour will change in a future release when pagination will be introduced.
• UNRESPONSIVE_HOURS: The amount of hours since the last check-in after which a node is considered unresponsive.
• LOGLEVEL: A string representing the loglevel. It defaults to 'info' but can be changed to 'warning' or 'critical' for less verbose logging or 'debug' for more information.
• ENABLE_QUERY: Defaults to True causing a Query tab to show up in the web interface allowing users to write and execute arbitrary queries against a set of endpoints in PuppetDB. Change this to False to disable this. See ENABLED_QUERY_ENDPOINTS to fine-tune which endpoints are allowed.
• ENABLED_QUERY_ENDPOINTS: If ENABLE_QUERY is True, allow to fine tune the endpoints of PuppetDB APIs that can be queried. It must be a list of strings of PuppetDB endpoints for which the query is enabled. See the QUERY_ENDPOINTS constant in the puppetboard.app module for a list of the available endpoints.
• GRAPH_TYPE: Specify the type of graph to display. Default is pie, other good option is donut. Other choices can be found here: _C3JS_documentation`
• GRAPH_FACTS: A list of fact names to tell PuppetBoard to generate a pie-chart on the fact page. With some fact values being unique per node, like ipaddress, uuid, and serial number, as well as structured facts it was no longer feasible to generate a graph for everything.
• INVENTORY_FACTS: A list of tuples that serve as the column header and the fact name to search for to create the inventory page. If a fact is not found for a node then undef is printed.
• ENABLE_CATALOG: If set to True allows the user to view a node's latest catalog. This includes all managed resources, their file-system locations and their relationships, if available. Defaults to False.
• REFRESH_RATE: Defaults to 30 the number of seconds to wait until the index page is automatically refreshed.
• DEFAULT_ENVIRONMENT: Defaults to 'production', as the name suggests, load all information filtered by this environment value.
• REPORTS_COUNT: Defaults to 10 the limit of the number of reports to load on the node or any reports page.
• OFFLINE_MODE: If set to True load static assets (jquery, semantic-ui, etc) from the local web server instead of a CDN. Defaults to False.
• DAILY_REPORTS_CHART_ENABLED: Enable the use of daily chart graphs when looking at dashboard and node view.
• DAILY_REPORTS_CHART_DAYS: Number of days to show history for on the daily report graphs.
• DISPLAYED_METRICS: Metrics to show when displaying node summary. Example: 'resources.total', 'events.noop'.
• TABLE_COUNT_SELECTOR: Configure the dropdown to limit number of hosts to show per page.
• LITTLE_TABLE_COUNT: Default number of reports to show when when looking at a node.
• NORMAL_TABLE_COUNT: Default number of nodes to show when displaying reports and catalog nodes.
• LOCALISE_TIMESTAMP: Normalize time based on localserver time.
• WITH_EVENT_NUMBERS: If set to True then Overview and Nodes list shows exact number of changed resources in the last report. Otherwise shows only 'some' string if there are resources with given status. Setting this to False gives performance benefits, especially in big Puppet environments (more than few hundreds of nodes). Defaults to True.
• DEV_LISTEN_HOST: For use with dev.py for development. Default is localhost
• DEV_LISTEN_PORT: For use with dev.py for development. Default is 5000

Getting Help

For questions or bug reports you can file an issue.

(Previously mentioned here methods of contacting the app maintainers are not used as of now anymore, sorry.)

Contributing

Development

Puppetboard relies on the pypuppetdb library to fetch data from PuppetDB and is built with the help of the Flask microframework.

If you wish to hack on Puppetboard you should fork/clone the Github repository and then install the requirements through:

pip install --upgrade wheel setuptools
pip install --upgrade -r requirements-test.txt
mypy --install-types --non-interactive puppetboard/ test/

You're advised to do this inside a virtualenv specifically created to work on Puppetboard as to not pollute your global Python installation.

You can run the tests with:

pytest --cov=. --cov-report=xml --flake8 --strict-markers --mypy puppetboard test

You can run the app it in development mode by simply executing:

python dev.py

Use PUPPETBOARD_SETTINGS to change the different settings or patch default_settings.py directly. Take care not to include your local changes on that file when submitting patches for Puppetboard. Place a settings.py file inside the base directory of the git repository that will be used, if the environment variable is not set.

We welcome contributions to this project. However, there are a few ground rules contributors should be aware of.

License

This project is licensed under the Apache v2.0 License. As such, your contributions, once accepted, are automatically covered by this license.

Commit messages

Write decent commit messages. Don't use swear words and refrain from uninformative commit messages as 'fixed typo'.

The preferred format of a commit message:

docs/quickstart: Fixed a typo in the Nodes section.

If needed, elaborate further on this commit. Feel free to write a
complete blog post here if that helps us understand what this is
all about.

Fixes #4 and resolves #2.

If you'd like a more elaborate guide on how to write and format your commit messages have a look at this post by Tim Pope.

More Screenshots

Up to date ones:

Older ones, may be a bit outdated:

Changelog

This is the changelog for Puppetboard.

Development

• Add support for Python 3.10

3.1.0

• Improve facts columns balancing (#618)
• Allow to toggle checkboxes by clicking their label (#617)
• Add support for Python 3.9 (#619)
• pypuppetdb: raise version requirement >=2.4.0.rc1 because we need it for Python 3.9 support

3.0.0

This is a bugfix and maintenance release. The major version is bumped because of the Python 3.5 support drop.

Features:

• Change the default sort order of facts table to a-z, instead of z-a (#572)
• Revamp the README (#601)

Bugfixes:

• Fix noop class in _macros.html (#588)
• Fix listing nodes with boolean fact values (#583)
• Fix auto-resize in radiator view (#605)
• Fix issue with no render when facts are empty (#607)

Other:

• Drop Python 3.5 support (#593)
• Update jQuery to 3.5.1 (#592)
• Manage any other exception for get_or_abort (#606)
• Improve getting resources from CDN (#609)
• Migrate from Travis to GitHub Actions (#604)

2.2.0

• Fix default table sort (#444)
• Use a select for endpoint select in query (#575)
• Surround multiline messages with

   in reports (#576)

• Fix CI builds by requiring pytest >= 4.6 (#577)
• Add noop column in overview and nodes (#584)
• Add title to events labels in overview and nodes (#585)

2.1.2

• Puppet DB 5.2.13 requires v2 metrics

2.1.1

• Added support for new metrics API v2 on PuppetDB >= 5.3.11 and < 6.0.0 (#558)
• Added Python 3.5 back into test matrix (#559)
• Fixed bug in dailyreport.js that caused it not to render when served under a non-default virtual host (#557)

2.1.0

• Fixed Puppetboard's usage for the new metrics v2 API both on the home page for computing the average resources/node and the Metrics listing page. This change now supports the changes in PuppetDB >= 6.9.1 (https://puppet.com/security/cve/CVE-2020-7943/)
• Added backwards compatability support for both the metric v1 and v2 endpoints depending on the version of the server. Any PuppetDB >= 6.9.1 will be queried with the v2 endpoint automatically (because v1 is disabled from here forward). Any PuppetDB <= 6.9.0 will use v1.
• pypuppetdb: raise version requirement >=2.1.0 because changes were needed in this library to support the metrics v2 fixes.
• app.py: Added python2 backwards compatability fix for importing urllib.

2.0.0

• Dockerfile: Switch to python:3.7-alpine image
• pypuppetdb: raise version requirement >=1.2.0 to >=2.0.0
• Drop support for python2.7 and python3.5 & Add python3.8 to buildmatrix
• Upgrade of tests requirements + resolving current deprecation warnings
• Ignore facts environment for compatibility and performance
• Adding mypy + Cleanup + CommonMark upgrade to 0.9.1
• Update docker and fix coveralls not running.
• Cast inventory data toString

1.1.0

• Move to Python 3.6 for Docker
• Fix problem loading daily chart on node page
• fix gunicorn parameter and allow to define workers in docker
• Add feature for better performance in big Puppet envs
• bundle requirements.txt for tests and docker

1.0.0

• CI enhancements
• Allow to configure which PuppetDB endpoints are allowed
• Update c3 to 4.22
• Add basic health check endpoint
• Allow to force the PuppetDB connection protocol
• Update jquery-tablesort to 0.0.11
• Fix bug breaking date/time sort
• Fix formatvalue for list of dicts
• Modify date sort to handle failed
• Include template files for altering Semantic css and Google fonts
• Make 320px the max width for columns
• If query is None don't perform add on it
• Query using producer_timestamp index vs. start_time
• Add missing components for building source packages
• Add support for URL prefixes to Docker image

0.3.0

• Core UI Rework
• Update to pypuppetdb 0.3.3
• Fix sorty on data for index
• Update debian documentation
• Offline mode fix
• Fix fact attribute error on paths
• Enhanced testing
• Radiator CSS uses same coloring
• Markdown in config version
• Update Flask
• Cleanup requirements.txt
• Update package maintainer for OpenBSD

0.2.1

• Daily Charts
• Fixed missing javascript files on radiator view.
• TravisCI and Coveralls integration.
• Fixed app crash in catalog view.
• Upgrade pypuppetdb to 0.3.2
• Enhanced queries for Node and Report (#271)
• Optimize Inventory Code.
• Use certname instead of hostname to identify nodes when applicable.
• Add environment filter for facts.
• Update cs.js to 0.4.11
• Fix radiator column alignment
• Security checks with bandit
• Dockerfile now uses gunicorn and environment variables for configuration.
• Handle division by zero errors.
• Implement new Jquery Datatables.
• JSON output for radiator. * Move javascript to head tag.
• Optimize reports and node page queries.
• Fix all environments for PuppetDB 3.2
• Fact graph chart now configurable.
• Support for Flask 0.12 and Jinja2 2.9
• Fix misreporting noops as changes.

0.2.0

• Full support for PuppetDB 4.x
• Updating Semantic UI to 2.1.8
• Updating Flask-WTF requirements to 0.12
• Updating WTForms to 2.x
• Restored CSRF protection on the Query Tab form
• Updating Pypuppetdb requirement to 0.3.x
• New configuration option OVERVIEW_FILTER allows users to add custom PuppetDB query clauses to include/exclude nodes displayed on the index page
• Adding Radiator view similar to what is available in Puppet Dashboard
• Adding a drop-down list in the Reports tab to configure the number of reports displayed
• Removing unneeded report_latest() endpoint. This endpoint was deprecated with the addition of the latest_report_hash field in the Nodes PuppetDB endpoint
• Enhancing Report pagination
• Using the OOP Query Builder available in Pypuppetdb 0.3.x
• Allowing PQL queries in the Query Tab
• Fixing double url-quoting bug on Metric endpodint calls
• Adding a Boolean field to the Query form to prettyprint responses from PuppetDB
• Fixing corner-case where empty environments would trigger a ZeroDivisionError due to the Number of Nodes divided by the Number of Resources calculation
• Adding additional logging in utils.py

0.1.2

• Add configuration option to set the default environment with new configuration option DEFAULT_ENVIRONMENT, defaults to 'production'.
• Loading all available environments with every page load.
• Adding an "All Environments" item to the Environments dropdown to remove all environment filters on PuppetDB data.
• Updating README.rst to update links and describe new configuration options.
• Fixing Query form submission problem by disabling CSRF protection. Needs to be re-implemented.

* Updating the pypuppetdb requirement to >= 0.2.1, using information
available in PuppetDB 3.2 and higher

* latest_report_hash and latest_report_status fields from the Nodes endpoint, this effectively deprecates the report_latest() function* code_id from the Catalogs endpoint (current unused) * Adding a automatic refresh on the overview page to reload the page every X number of seconds, defaults to 30. This is configurable with the configuration option REFRESH_RATE * Fixing the table alignment in the catalog_compare() page by switching to fixed tables from basic tables. * Using colors similar to Puppet Dashboard and Foreman for the status counts sections

0.1.1

• Fix bug where the reports template was not generating the report links with the right environment

0.1.0

• Requires pypuppetdb >= 0.2.0
• Drop support for PuppetDB 2 and earlier
• Full support for PuppetDB 3.x
• The first directory location is now a Puppet environment which is filtered on all supported queries. Users can browse different environments with a select field in the top NavBar
• Using limit, order_by and offset parameters adding pagaination on the Reports page (available in the NavBar). Functionality is available to pages that accept a page attribute.
• The report page now directly queries pypuppetdb to match the report_id value with the report hash or configuration_version fields.
• Catching and aborting with a 404 if the report and report_latest function queries do not return a generator object.
• Adding a Catalogs page (similar to the Nodes page) with a form to compare one node's catalog information with that of another node.
• Updating the Query Endpoints for the Query page.
• Adding to templates/_macros.html status_counts that shows node/report status information, like what is avaiable on the index and nodes pages, available to the reports pages and tables also.
• Showing report logs and metrics in the report page.
• Removing limit_reports from utils.py because this helper function has been replaced by the limit PuppetDB paging function.

Known Issues

• fact_value pages rendered from JSON valued facts return no results. A more sophisticated API is required to make use of JSON valued facts (through the factsets, fact-paths and/or fact-contents endpoints for example)

0.0.5

• Now requires WTForms versions less than 2.0
• Adding a Flask development server in dev.py.
• Adding CSRF protection VIA the flask_wtf CsrfProtect object.
• Allowing users to configure the report limit on pages where reports are listed with the LIMIT_REPORTS configuration option.
• Adding an inventory page to users to be able to see all available nodes and a configure lists of facts to display VIA the INVENTORY_FACTS configuration option.
• Adding a page to view a node's catalog information if enabled, disabled by default. Can be changed with the ENABLE_CATALOG configuration attribute.
• New configuration option GRAPH_FACTS allows the user to choose which graphs will generate pie on the fact pages.
• Replacing Chart.js with c3.js and d3.js.
• Adding Semantic UI 0.16.1 and removing unused bootstrap styles.
• Adding an OFFLINE_MODE configuration option to load local assets or from a CDN service. This is useful in environments without internet access.

0.0.4

• Fix the sorting of the different tables containing facts.
• Fix the license in our setup.py. The license shouldn't be longer than 200 characters. We were including the full license tripping up tools like bdist_rpm.

0.0.3

This release introduces a few big changes. The most obvious one is the revamped Overview page which has received significant love. Most of the work was done by Julius Härtl. The Nodes tab has been given a slight face-lift too.

Other changes:

• This release depends on the new pypuppetdb 0.1.0. Because of this the SSL configuration options have been changed:
  • PUPPETDB_SSL is gone and replaced by PUPPETDB_SSL_VERIFY which now defaults to True. This only affects connections to PuppetDB that happen over SSL.
  • SSL is automatically enabled if both PUPPETDB_CERT and PUPPETDB_KEY are provided.
• Display of deeply nested metrics and query results have been fixed.
• Average resources per node metric is now displayed as a natural number.
• A link back to the node has been added to the reports.
• A few issues with reports have been fixed.
• A new setting called UNRESPONSIVE_HOURS has been added which denotes the amount of hours after which Puppetboard will display the node as unreported if it hasn't checked in. We default to 2 hours.
• The event message can now be viewed by clicking on the event.

Puppetboard is now neatly packaged up and available on PyPi. This should significantly help reduce the convoluted installation instructions people had to follow.

Updated installation instructions have been added on how to install from PyPi and how to configure your HTTPD.

0.0.2

In this release we've introduced a few new things. First of all we now require pypuppetdb version 0.0.4 or later which includes support for the v3 API introduced with PuppetDB 1.5.

Because of changes in PuppetDB 1.5 and therefor in pypuppetdb users of the v2 API, regardless of the PuppetDB version, will no longer be able to view reports or events.

In light of this the following settings have been removed:

• PUPPETDB_EXPERIMENTAL

Two new settings have been added:

• PUPPETDB_API: an integer, defaulting to 3, representing the API version we want to use.
• ENABLE_QUERY: a boolean, defaulting to True, on wether or not to be able to use the Query tab.

We've also added a few new features:

• Thanks to some work done during PuppetConf together with Nick Lewis (from Puppet Labs) we now expose all of PuppetDB's metrics in the Metrics tab. The formatting isn't exactly pretty but it's a start.
• Spencer Krum added the graphing capabilities to the Facts tab.
• Daniel Lawrence added a feature so that facts on the node view are clickable and take you to the complete overview of that fact for your infrastructure and made the nodes in the complete facts list clickable so you can jump to a node.
• Klavs Klavsen contributed some documentation on how to run Puppetboard with Passenger.

0.0.1

Initial release.