Ansible Playbook Scanning Tool
Project description
Steampunk Spotter CLI
Table of Contents
- Introduction
- Installation
- Usage
- Limitations
- Authentication
- Scanning
- Ansible content
- Selecting the target project
- Including values
- Including metadata
- Automated application of suggestions to your code
- Suppressing check result levels
- Applying profiles
- Exporting and importing scan payload
- Scan configuration
- Formatting scan result
- Omitting documentation URLs from the output
- Managing custom policies
- Setting storage folder
- Disabling colorized output
- Setting API endpoint
- CI/CD integrations
- Development
- Acknowledgement
Introduction
Steampunk Spotter provides an Ansible Playbook Scanning Tool that analyzes and offers recommendations for your Ansible Playbooks.
The Steampunk Spotter CLI enables the use from the console with the ability to scan Ansible content such as playbooks, roles, collections, or task files.
Installation
steampunk-spotter
requires Python 3 and is available as a
steampunk-spotter Python package.
$ pip install steampunk-spotter
We suggest installing the package into a clean Python virtual environment.
Usage
After the CLI is installed, you can explore its commands and options by
running spotter --help
.
The --help/-h
optional argument is also available for every command.
Limitations
The current release version of Steampunk Spotter contains the following limitations that also apply to the CLI:
- with the FREE subscription plan, you can perform up to 100 scans/month,
- with the INDIVIDUAL, TEAM or ENTERPRISE subscription plan you can perform an unlimited number of scans,
- for the ENTERPRISE subscription plan, contact us at steampunk@xlab.si to discuss your needs.
Authentication
To use CLI, you have to supply your Steampunk Spotter user account
credentials.
If you don't have an account, use spotter register
command that will direct
you to the page where you can create one.
Steampunk Spotter supports two kinds of credentials:
- Your username and password.
- Your API token. To create and manage tokens, visit the Steampunk Spotter App's web application, open the top-right account menu, click My Settings and switch to the API tokens tab.
After that, you can start scanning right away.
$ spotter --api-token <api-token> scan playbook.yaml
or:
$ export SPOTTER_API_TOKEN=<api-token>
$ spotter scan playbook.yaml
or:
$ spotter --username <username> --password <password> scan playbook.yaml
or:
$ export SPOTTER_USERNAME=<username>
$ export SPOTTER_PASSWORD=<password>
$ spotter scan playbook.yaml
Steampunk Spotter CLI also offers for convenience the spotter login
command,
which saves the chosen credentials (in plaintext) in your user profile:
$ spotter --api-token <api-token> login
or:
$ export SPOTTER_API_TOKEN=<api-token>
$ spotter login
or:
$ spotter --username <username> --password <password> login
or:
$ export SPOTTER_USERNAME=<username>
$ export SPOTTER_PASSWORD=<password>
$ spotter login
After a successful CLI login, the credentials no longer need to be supplied explicitly to the CLI commands.
Scanning
The CLI spotter scan
command is used for scanning Ansible
content (playbooks, roles, collections, or task files) and returning back the
scan results.
Ansible content
The scan
command accepts a positional argument that can be one or many paths
to files or directories.
The CLI will automatically detect the type of your Ansible content and scan it.
The following types of Ansible content file are currently supported:
- tasks
- playbooks
- roles (applies to
tasks
andhandlers
folders) - collections (applies to
roles
,playbooks
andtests/integration/targets/
folders and any playbooks at the root of collection)
# scan task file, which contains the `tasks` section of the playbook
$ spotter scan path/to/taskfile1.yaml
# scan two playbooks
$ spotter scan path/to/playbook1.yaml path/to/playbook2.yaml
# scan multiple playbooks using glob
$ spotter scan path/to/playbook/folder/play_*.yaml
# scan two roles (scans tasks and handlers folders)
$ spotter scan path/to/role1 path/to/role2
# scan collection (scans Ansible content within roles and playbooks folders
and playbooks at the root of collection directory)
$ spotter scan path/to/collection
# scan multiple files at once
$ spotter scan path/to/taskfile.yaml path/to/playbook.yaml
path/to/role path/to/collection
# scan any folder that contains Ansible content
$ spotter scan path/to/folder
Let us assume we have the following Ansible playbook playbook.yaml
file:
---
1 - name: Sample playbook
2 hosts: localhost
3
4 collections:
5 - sensu.sensu_go
6 - ansible.builtin
7
8 tasks:
9 - name: Create Sensu Go user
10 user:
11 name: joe
12 password: "{{ lookup('env', 'SENSU_USER_PASSWORD') }}"
13
14 - name: Read-only command
15 ansible.builtin.command: ls /tmp
16 changed_when: false
In this case, the CLI tool will report something like that back:
$ spotter scan playbook.yaml
playbook.yaml:10:7: ERROR: Potential name clashes for user among
sensu.sensu_go.user, ansible.builtin.user.
playbook.yaml:10:7: ERROR: Use a fully-qualified name, such as
sensu.sensu_go.user instead of user.
playbook.yaml:10:7: HINT: Required collection sensu.sensu_go is missing from
requirements.yml or requirements.yml is missing.
playbook.yaml:15:7: WARNING: Default value for warn parameter changed between
module versions 2.10.17 and 2.11.0 from true to false. Consider setting
value of the parameter explicitly.
------------------------------------------------------------------------
Spotter took 0.190 s to scan your input.
It resulted in 2 error(s), 1 warning(s) and 1 hint(s).
Overall status: ERROR
Selecting the target project
This part is only relevant for users with a TEAM plan or higher.
By default, the scan results are stored in the first project of the user's first organization (in the app).
Users that have multiple organizations or projects in the app can use
--project-id
optional argument to specify the UUID of an existing target
project, where the scan result will be stored.
$ spotter scan --project-id <project-id> .
You can learn your project id by logging into the app, selecting the appropriate organization and navigating to the project's dashboard.
Including values
By default, CLI parses Ansible YAML content without any parameter values.
With values, we can discover additional tips for improvements, so if you want
to parse and send the values, you can use --include-values
optional argument.
$ spotter scan --include-values playbook.yaml
Including metadata
By default, CLI collects and uses metadata (file names, line and column
numbers, YAML markers) from Ansible content just for displaying the scan
output, which means that no data about your Ansible content structure is sent
to the backend server.
For enriched user experience in the app and to get additional tips for
improvements, you can use --include-metadata
optional argument to send the
metadata.
$ spotter scan --include-metadata playbook.yaml
Automated application of suggestions to your code
There is also a --rewrite
optional argument that rewrites your files with
fixes after scanning.
This action will modify your files.
$ spotter scan --rewrite playbook.yaml
Suppressing check result levels
You can use --display-level
optional argument for suppressing check result
levels.
For example, to show only errors (suppress warnings and hints):
$ spotter scan --display-level error playbook.yaml
Applying scan profiles
When we run scans, we might have a particular goal in mind. For example, in one project we might be interested in upgrading our Ansible environment to a newer version of Ansible. In another one, we want to improve the playbooks for the Ansible version that we are currently in. This means that some check results that Steampunk Spotter produces may be relevant in one of the projects, but not in the other one.
Using the --profile
optional argument, we can specify a scan profile that
contains a selected set of checks to be used for scanning. Spotter currently
supports the following profiles:
default
- this profile is suitable for day-to-day testing and improving Ansible playbooks.full
- displays the full range of check results, which would be useful for the Ansible playbook updating to work at a newer version of Ansible.
For example, to run all checks (apply full
profile):
$ spotter scan --profile full playbook.yaml
Exporting and importing scan payload
To see what data is collected from your Ansible content and sent to the
backend server, you can use the optional --export-payload
argument.
$ spotter scan --export-payload payload.json playbook.yaml
Scan data saved to payload.json.
Note: this operation is fully offline. No actual scan was executed.
After that you can also import (with --import-payload
optional argument)
the exported payload and scan it:
$ spotter scan --import-payload payload.json
Scan configuration
Before scanning, it is possible to configure the environment via the configuration file or optional CLI variables. By default, the CLI runs local discovery of the user's environment.
Currently, the supported variables for configuration file and options are:
ansible_version
- sets target Ansible version
When --config
optional argument is used, the variables specified in
JSON/YAML config file will overwrite the ones from local discovery.
For instance, if we want to overwrite the target Ansible version, we can use
the following JSON config file:
{
"ansible_version": "2.9"
}
And after that we can run the scan command:
$ spotter scan --config config.json playbook.yaml
We can also use --option
optional argument that will overwrite local
discovery and config file.
After --option
optional argument, the user should provide configuration
via key=value pairs.
For instance, if we want to set the Ansible version, we can use the following
command:
# scan Ansible playbooks
$ spotter scan --option ansible_version=2.9 playbook.yaml
Formatting scan result
By default, the CLI will output the scan result in plain text format.
The --format
optional argument allows you to specify the alternative output
format of the scan result such as JSON or YAML.
# scan Ansible playbooks
$ spotter scan --format json playbook.yaml
Omitting documentation URLs from the output
In the scan result, the CLI will display a URL to the relevant Ansible content
documentation in whenever possible.
To omit these documentation URLs from all the output, use --no-docs-url
optional argument.
$ spotter scan --no-docs-url playbook.yaml
Managing custom policies
It is possible to create and use policies for custom Spotter checks written in Rego Language for Open Policy Agent (OPA). The use of custom policies is only available in Spotter's ENTERPRISE plan.
Use set-policies
command to set custom OPA policies.
This will override all current policies.
# set one policy
$ spotter set-policies policy.rego
# set a whole dir with policies
$ spotter set-policies policies/
# set policy for a specific project
$ spotter set-policies --project-id <project-id> policy.rego
# set policy for whole organization
$ spotter set-policies --organization-id <organization-id> policy.rego
After that run a scan to see check results you included.
$ spotter scan playbook.yaml
Use clear-policies
command to clear custom policies.
# clear policies
$ spotter clear-policies
# clear policies for a specific project
$ spotter clear-policies --project-id <project-id>
# clear policies for whole organization
$ spotter clear-policies --organization-id <organization-id>
Setting storage folder
The CLI uses local storage for caching access tokens for the Steampunk
Spotter API.
The default location is ~/.config/steampunk-spotter
, but if you want to
change it you can use --storage-path
optional argument.
$ spotter --storage-path /my/project/.storage scan playbook.yaml
Disabling colorized output
The CLI will colorize the scan result by default.
To make the output non-colorized use --no-colors
option.
$ spotter --no-colors scan playbook.yaml
Setting API endpoint
The CLI connects to Steampunk Spotter API (backend server) to perform scanning. The default API endpoint is already set, but if you need to change it, you can do this in different ways.
The precedence of the API endpoint configuration is the following, where the first one specified takes effect:
-
the
--endpoint
global optional argument.$ spotter --endpoint "<spotter-api-url>" scan playbook.yml
-
the
SPOTTER_ENDPOINT
environment variable.$ export SPOTTER_ENDPOINT=<spotter-api-url>
-
configuration file from storage folder (
~/.config/steampunk-spotter/spotter.json
) with the root JSON entry"endpoint": "<endpoint>"
(this allows persistent API endpoint configuration). You should create it manually with the following JSON content:{ "endpoint": "<endpoint>" }
-
the default Spotter SaaS API endpoint:
https://api.spotter.steampunk.si/api
.
CI/CD integrations
The CLI can be used in CI/CD pipelines to set up quality scanning of your Ansible content.
When using the CLI in CI/CD workflows, it is important that you provide Steampunk Spotter credentials as secrets (i.e., pipeline-protected and masked variables).
GitLab
The CLI can be integrated with GitLab CI/CD to display scan results as GitLab’s unit test reports. This means that you will be quickly able to see which checks have failed within your CI/CD pipeline.
This is done by using Spotter CLI tool directly in the CI/CD configuration and
configuring it to output your scan result in JUnit XML format, which allows
GitLab to display check results as green check marks for successful checks and
red crosses for unsuccessful checks.
To do so, you should use spotter scan
CLI command along with
--junit-xml <path-junit-xml>
optional argument that will create JUnit XML
report at the specified location.
Below is a .gilab-ci.yaml
example containing a CI job for the test stage,
where you call the aforementioned CLI command and then upload the created
JUnit report format XML file as an artifact to GitLab, which will then display
it within your pipeline details page or merge request widget.
stages:
- test
spotter-scan:
stage: test
image:
name: registry.gitlab.com/xlab-steampunk/steampunk-spotter-client/spotter-cli:<version>
entrypoint: [""]
variables:
SPOTTER_API_TOKEN: $SPOTTER_API_TOKEN
script:
- spotter scan --junit-xml report.xml .
artifacts:
when: always
reports:
junit: report.xml
GitHub
In your CI/CD pipeline, you can specify the name of the
Steampunk Spotter GitHub Action (xlab-steampunk/spotter-action@master
)
with a tag number as a step within your YAML workflow file.
For example, inside your .github/workflows/ci.yml
file, you can do:
name: test
on: push
jobs:
test:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v3
- name: Run Ansible content scan
uses: xlab-steampunk/spotter-action@<version>
env:
SPOTTER_API_TOKEN: ${{ secrets.SPOTTER_API_TOKEN }}
For comprehensive usage and more examples refer to Steampunk Spotter Action on GitHub Marketplace and Steampunk Spotter GitHub Action repository.
Others
For other CI/CDs, we currently only support using the steampunk-spotter
Python package and setting it up as a regular shell command.
You can also use spotter-cli Docker image that is available in our GitLab
Registry (use registry.gitlab.com/xlab-steampunk/steampunk-spotter-client/spotter-cli:latest
image path and select the appropriate tag).
You can use spotter scan
CLI command along with
--junit-xml <path-junit-xml>
optional argument to export scan result in
JUnit XML format, which is consumed by some CI tools such as Jenkins or Bamboo.
Development
You will first need to clone this repository.
$ git clone https://gitlab.com/xlab-steampunk/steampunk-spotter-client/spotter-cli
$ cd spotter-cli
Running from source
If you want to run directly from the source, run the following commands:
$ python3 -m venv .venv && . .venv/bin/activate
$ pip install -e .
Building Docker image
Before building the Docker image, copy the wheel file distribution to the
root of this repository.
Make sure that you have only one *.whl
file.
You can build the wheel yourself or download one from the latest CI/CD
pipeline.
After that, proceed with the following commands:
$ docker build -t spotter-cli .
After that, you will be able to run scanning in a Docker container.
You can mount your Ansible content to the default /scan
working directory.
The credentials can be specified as environment variables for the Docker
container or later via --api-token
or --username
and --password
CLI
optional arguments.
For instance:
docker run --rm -it -e SPOTTER_API_TOKEN=<api-token> -v "/path/to/your/playbooks/:/scan" spotter-cli scan .
or
docker run --rm -it -e SPOTTER_USERNAME=<username> -e SPOTTER_PASSWORD=<password> -v "/path/to/your/playbooks/:/scan" spotter-cli scan .
Acknowledgement
This tool was created by XLAB Steampunk, IT automation specialist and leading expert in building Enterprise Ansible Collections.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
Built Distribution
Hashes for steampunk_spotter-1.2.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d2292323c90c743c91167e31e50d3123a5158097d5eb3a061a46ac75ef124d98 |
|
MD5 | ebb22c86a3df21a52edab0c08bec418e |
|
BLAKE2b-256 | 2547b9448bd4d47465f67f06ba504a1ca704152ffe6e24741dcfd4531f745b02 |