watchmaker 0.9.3

Applied Configuration Management

Watchmaker

Applied Configuration Management

Overview

Watchmaker is a Python package that helps bootstrap a vanilla OS image and apply an OS configuration. Watchmaker itself reads a simple YAML configuration file, which can be hosted on the local filesystem or on a web server.

Complex configuration management (CM) environments may be layered in as part of the provisioning framework. Watchmaker includes a default configuration that will install Salt and a handful Salt Formulas that can be used to harden a system to DISA STIG standards, as well as integrate with common enterprise services.

Documentation

For more information on installing and using Watchmaker, go to https://watchmaker.readthedocs.io.

Changelog

0.9.3

Commit Delta: Change from 0.9.2 release

Released: 2017.03.08

Summary:

• scap-formula

  • Incorporates content from OpenSCAP Security Guide v0.1.38-1

  • Incorporates content from latest DISA SCAP benchmarks

    • Microsoft Internet Explorer 11 STIG Benchmark - Ver 1, Rel 11

    • Microsoft Windows 10 STIG Benchmark - Ver 1, Rel 10

    • Microsoft Windows 2008 R2 DC STIG Benchmark - Ver 1, Rel 27

    • Microsoft Windows 2008 R2 MS STIG Benchmark - Ver 1, Rel 28

    • Microsoft Windows 2012 and 2012 R2 DC STIG Benchmark - Ver 2, Rel 11

    • Microsoft Windows 2012 and 2012 R2 MS STIG Benchmark - Ver 2, Rel 11

    • Microsoft Windows 8/8.1 STIG Benchmark - Ver 1, Rel 21

    • Microsoft Windows Server 2016 STIG Benchmark - Ver 1, Rel 4

    • Red Hat 6 STIG Benchmark - Ver 1, Rel 18

    • Red Hat 7 STIG Benchmark - Ver 1, Rel 2

• dotnet4-formula

  • Skips dotnet4 hotfix install if a newer version is already installed

  • Creates per-OS maps for hotfix updates, since the hotfix id varies per OS

0.9.2

Commit Delta: Change from 0.9.1 release

Released: 2018.02.20

Summary:

• dotnet4-formula

  • Passes version correctly to module.run

0.9.1

Commit Delta: Change from 0.9.0 release

Released: 2018.02.17

Summary:

• This version was effectively a no-op, as the submodule was not updated as intended

• ~dotnet4-formula~

  • ~Passes version correctly to module.run~

0.9.0

Commit Delta: Change from 0.8.0 release

Released: 2018.02.12

Summary:

• [Issue #499][PR #513] Includes additional details about the platform and python version in the watchmaker log

• [Issue #500][PR #512] Retries file retrieval up to 5 times

• [Issue #501][PR #507] Uses urllib handlers to retrieve all files

  • Deprecates the argument --s3-source; to retrieve a file from an S3 bucket use the syntax: s3://<bucket>/<key>

  • Local files may be specified as absolute or relative paths, and may or may not be prefixed with file://

• [PR #496] Moves CloudFormation and Terraform templates to their own project, terraform-aws-watchmaker

• [PR #491] Improves compatibility of the watchmaker bootstrap.ps1 script when executed by an Azure custom script extension

• [Issue #430][PR #487] Writes watchmaker salt config to a custom path:

  • Windows: C:\Watchmaker\Salt\conf

  • Linux: /opt/watchmaker/salt

• scap-formula

  • Incorporates content from OpenSCAP Security Guide v0.1.37-1

0.8.0

Commit Delta: Change from 0.7.2 release

Released: 2018.01.02

Summary:

• [Issue #415][PR #458] Forwards watchmaker log entries from the Windows Event Log to the EC2 System Log (Windows-only)

• [PR #425] Adds a log handler that writes watchmaker log entries to the Windows Event Log (Windows-only)

• [Issue #434][PR #457] Updates doc build to replace recommonmark functionality entirely with m2r

• [PR #437] Modfies CloudFormation templates to use aws cli utility to retrieve the appscript rather than use the functionality built-in to the cfn bootstrap

• [PR #467] Sets environment variables for aws cli when executing the appscript option in the watchmaker CloudFormation templates

0.7.2

Commit Delta: Change from 0.7.1 release

Released: 2017.12.13

Summary:

• Installs futures only on Python 2 – no functional changes

0.7.1

Commit Delta: Change from 0.7.0 release

Released: 2017.12.04

Summary:

• Fixes readthedocs build – no functional changes

0.7.0

Commit Delta: Change from 0.6.6 release

Released: 2017.11.21

Summary:

• [PR #409] Provides terraform modules that deploy the watchmaker CloudFormation templates

• [Issue #418][PR #419] Adds an exclude-states argument to the SaltWorker; specified states will be excluded from the salt state execution

• ash-windows-formula

  • Incorporates security settings from the DISA October quarterly release

• join-domain-formula

  • (Windows) Adds WMI method to set DNS search suffix

  • (Windows) Tests for the EC2Config XML settings file before modifying it

• scap-formula

  • (Linux) Distributes scap content from SCAP Security Guide v0.1.36-1

  • Distributes scap content from the DISA October quarterly release

• splunkforwarder-formula

  • Supports configuration of splunk log sources from pillar and grains inputs

0.6.6

Commit Delta: Change from 0.6.5 release

Released: 2017.10.18

Summary:

• ash-linux-formula

  • (el7) Fixes typos in the firewalld “safety” scripts that resulted in a failure when firewalld was reloaded

• mcafee-agent-formula

  • (el7) Adds required inbound ports to all firewalld zones, to support the event where the default zone is modified from “public”

• splunkforwarder-formula

  • (el7) Adds required outbound ports to the OUTPUT chain; previously, they were mistakenly being added as inbound rules

0.6.5

Commit Delta: Change from 0.6.4 release

Released: 2017.09.29

Summary:

• [PR #391] Updates CloudFormation templates with a parameter that exposes the option to use the S3 API and the instance role to retrieve the Watchmaker content archive

• ash-linux-formula

  • (el7) Updates firewalld “safety” state so that firewalld remains in the active state; the prior approach left firewalld dead/inactive, until the service was restarted or the system was rebooted

0.6.4

Commit Delta: Change from 0.6.3 release

Released: 2017.09.22

Summary:

• [PR #381] Restricts wheel version on Python 2.6 to be less than or equal to 0.29.0, as wheel 0.30.0 removed support for py26.

0.6.3

Commit Delta: Change from 0.6.2 release

Released: 2017.08.11

Summary:

• ash-linux-formula

  • (el7) Includes a “safety” state for firewalld that ensures SSH inbound access will remain available, in the event the default zone is set to “drop”

0.6.2

Commit Delta: Change from 0.6.1 release

Released: 2017.08.07

Summary:

• ash-linux-formula

  • (el6) Improve the method of disabling the sysctl option ip_forward, to account for the behavior of the aws-vpc-nat rpm

• scap-formula

  • (elX) Updates openscap security guide content to version 0.1.34-1

0.6.1

Commit Delta: Change from 0.6.0 release

Released: 2017.08.01

Summary:

• ash-linux-formula

  • Modified the FIPS custom execution module to discover the boot partition and add the boot= line to the grub configuration

0.6.0

Commit Delta: Change from 0.5.1 release

Released: 2017.07.25

Summary:

• ash-linux-formula

  • Updates the EL7 stig baseline to manage the FIPS state. The state defaults to enabled but can be overridden via a pillar or grain, ash-linux:lookup:fips-state. The grain takes precedence over the pillar. Valid values are enabled or disabled

• ash-windows-formula

  • Updates the STIG baselines for Windows Server 2016 member servers and domain controllers with SCAP content from the DISA v1r1 SCAP benchmark release

• join-domain-formula

  • Fixes an issue when joining Windows 2016 servers to a domain, where the Set-DnsSearchSuffix.ps1 helper would fail because the builtin PowerShell version does not work when $null is used in a ValidateSet. The equivalent value must now be passed as the string, "null"

• scap-formula

  • Adds SCAP content for the Window Server 2016 SCAP v1r1 Benchmark

0.5.1

Commit Delta: Change from 0.5.0 release

Released: 2017.07.08

Summary:

• [Issue #341][PR #342] Manages selinux around salt state execution. In some non-interactive execution scenarios, if selinux is enforcing it can interfere with the execution of privileged commands (that otherwise work fine when executed interactively). Watchmaker now detects if selinux is enforcing and temporarily sets it to permissive for the duration of the salt state execution

0.5.0

Commit Delta: Change from 0.4.4 release

Released: 2017.06.27

Summary:

• [Issue #331][PR #332] Writes the role grain to the key expected by the ash-windows formula. Fixes usage of the --ash-role option in the salt worker

• [Issue #329][PR #330] Outputs watchmaker version at the debug log level

• [Issue #322][PR #323][PR #324] Fixes py2/py3 compatibility bug in how the yum worker handles file opening to check the Linux distro

• [Issue #316][PR #320] Improves logging when salt state execution fails due to failed a state. The salt output is now returned to the salt worker, which processes the output, identifies the failed state, and raises an exception with the state failure

• join-domain-formula

  • (Linux) Reworks the pbis config states to make the logged output more readable

0.4.4

Commit Delta: Change from 0.4.3 release

Released: 2017.05.30

Summary:

• join-domain-formula

  • (Linux) Ignores a bad exit code from pbis config utility. The utility will return exit code 5 when modifying the NssEnumerationEnabled setting, but still sets the requested value. This exit code is now ignored

0.4.3

Commit Delta: Change from 0.4.2 release

Released: 2017.05.25

Summary:

• name-computer-formula

  • (Linux) Uses an alternate method of working around a bad code-path in salt that does not handle quoted values in /etc/sysconfig/network.

0.4.2

Commit Delta: Change from 0.4.1 release

Released: 2017.05.19

Summary:

• [PR #301] Sets the grains for admin_groups and admin_users so the keys are named as expected by the join-domain formula

• ash-linux-formula

  • Adds a custom module that lists users from the shadow file

  • Gets local users from the shadow file rather than user.list_users. Prevents a domain-joined system from attempting to iterate over all domain users (and potentially deadlocking on especially large domains)

• join-domain-formula

  • Modifies PBIS install method to use RPMs directly, rather than the SHAR installer

  • Updates approaches to checking for collisions and current join status to better handle various scenarios: not joined, no collision; not joined, collision; joined, computer object present; joined, computer object missing

  • Disables NSS enumeration to prevent PBIS from querying user info from the domain for every call to getent (or equivalents); domain-based user authentication still works fine

• name-computer-formula

  • (Linux) Does not attempt to retain network settings, to avoid a bug in salt; will be revisited when a patched salt version has been released

0.4.1

Commit Delta: Change from 0.4.0 release

Released: 2017.05.09

Summary:

• (EL7) Running watchmaker against EL7 systems will now pin the resulting configuration to the watchmaker version. See the updates to the two formulas in this version. Previously, ash-linux always used the content from the scap-security-guide rpm, which was updated out-of-sync with watchmaker, and so the resulting configuration could not be pinned by pinning the watchmaker version. With this version, ash-linux uses content distributed by watchmaker, via scap-formula, and so the resulting configuration will always be same on EL7 for a given version of watchmaker (as has always been the case for the other supported operating systems).

• ash-linux-formula

  • Supports getting scap content locations from pillar

• scap-formula

  • Updates stig content with latest benchmark versions

  • Adds openscap ds.xml content, used to support remediate actions

0.4.0

Commit Delta: Change from 0.3.1 release

Released: 2017.05.06

Summary:

• [PR #286] Sets the computername grain with the correct key expected by the formula

• [PR #284] Converts cli argument parsing from argparse to click. This modifies the watchmaker depedencies, which warranted a 0.x.0 version bump. Cli and API arguments remain the same, so the change should be backwards-compatible.

• name-computer-formula

  • Adds support for getting the computername from pillar

  • Adds support for validating the specified computername against a pattern

• pshelp-formula

  • Attempts to address occasional stack overflow exception when updating powershell help

0.3.1

Commit Delta: Change from 0.3.0 release

Released: 2017.05.01

Summary:

• [PR #280] Modifies the dynamic import of boto3 to use only absolute imports, as the previous approach (attempt absolute and relative import) was deprecated in Python 3.3

• ntp-client-windows-formula:

  • Stops using deprecated arguments on reg.present states, which cleans up extraneous log messages in watchmaker runs under some configurations

• join-domain-formula:

  • (Windows) Sets the DNS search suffix when joining the domain, including a new pillar config option, ec2config to enable/disable the EC2Config option that also modifies the DNS suffix list.

0.3.0

Commit Delta: Change from 0.2.4 release

Released: 2017.04.24

Summary:

• [Issue #270] Defaults to a platform-specific log directory when call from the CLI:

  • Windows: ${Env:SystemDrive}\Watchmaker\Logs

  • Linux: /var/log/watchmaker

• [PR #271] Modifies CLI arguments to use explicit log-levels rather than a verbosity count. Arguments have been adjusted to better accommodate the semantics of this approach:

  • Uses -l|--log-level instead of -v|--verbose

  • -v and -V are now both used for --version

  • -d is now used for --log-dir

0.2.4

Commit Delta: Change from 0.2.3 release

Released: 2017.04.20

Summary:

• Fixes a bad version string

0.2.3

Commit Delta: Change from 0.2.2 release

Released: 2017.04.20

Summary:

• [Issue #262] Merges lists in pillar files, rather than overwriting them

• [Issue #261] Manages the enabled/disabled state of the salt-minion service, before and after the install

• splunkforwarder-formula

  • (Windows) Ignores false bad exits from Splunk clone-prep-clear-config

0.2.2

Commit Delta: Change from 0.2.1 release

Released: 2017.04.15

Summary:

• [PR #251] Adds CloudFormation templates that integrate Watchmaker with an EC2 instance or Autoscale Group

• join-domain-formula

  • (Linux) Corrects tests that determine whether the instance is already joined to the domain

0.2.1

Commit Delta: Change from 0.2.0 release

Released: 2017.04.10

Summary:

• ash-linux-formula

  • Reduces spurious stderr output

  • Removes notify script flagged by McAfee scans

• splunkforwarder-formula

  • (Windows) Clears system name entries from local Splunk config files

0.2.0

Commit Delta: Change from 0.1.7 release

Released: 2017.04.06

Summary:

• [Issue #238] Captures all unhandled exceptions and logs them

• [Issue #234] Stops the salt service prior to managing salt formulas, to ensure that the filesystem does not throw any errors about the files being locked

• [Issue #72] Manages salt service so the service state after watchmaker completes is the same as it was prior to running watchmaker. If the service was running beforehand, it remains running afterwards. If the service was stopped (or non-existent) beforehad, the service remains stopped afterwards

• [Issue #163] Modifies the user_formulas config option to support a map of <formula_name>:<formula_url>

• [PR #235] Extracts salt content to the same target srv location for both Window and Linux. Previously, the salt content was extracted to different points in the filesystem hierarchy, which required different content for Windows and Linux. Now the same salt content archive can be used for both

• [PR #242] Renames salt worker param content_source to salt_content

• systemprep-formula

  • Deprecated and removed. Replaced by new salt content structure that uses native salt capabilities to map states to a system

• scc-formula

  • Deprecated and removed. Replaced by scap-formula

• scap-formula

  • New bundled salt formula. Provides SCAP scans using either openscap or scc

• pshelp-formula

  • New bundled salt formula. Installs updated PowerShell help content to Windows systems

0.1.7

Commit Delta: Change from 0.1.6 release

Released: 2017.03.23

Summary:

• Uses threads to stream stdout and stderr to the watchmaker log when executing a command via subproces

• [Issue #226] Minimizes salt output of successful states, to make it easier to identify failed states

• join-domain-formula

  • (Linux) Exits with stateful failure on a bad decryption error

• mcafee-agent-formula

  • (Linux) Avoids attempting to diff a binary file

  • (Linux) Installs ed as a dependency of the McAfee VSEL agent

• scc-formula

  • Retries scan up to 5 times if scc exits with an error

0.1.6

Commit Delta: Change from 0.1.5 release

Released: 2017.03.16

Summary:

• ash-linux-formula

  • Provides same baseline states for both EL6 and EL7

0.1.5

Commit Delta: Change from 0.1.4 release

Released: 2017.03.15

Summary:

• ash-linux-formula

  • Adds policies to disable insecure Ciphers and MACs in sshd_config

• ash-windows-formula

  • Adds scm and stig baselines for Windows 10

  • Adds scm baseline for Windows Server 2016 (Alpha)

  • Updates all scm and stig baselines with latest content

• mcafee-agent-formula

  • Uses firewalld on EL7 rather than iptables

• scc-formula

  • Skips verification of GPG key when install SCC RPM

• splunkforwarder-formula

  • Uses firewalld on EL7 rather than iptables

0.1.4

Commit Delta: Change from 0.1.3 release

Released: 2017.03.09

Summary:

• [Issue #180] Fixes bug where file_roots did not contain formula paths

0.1.3

Commit Delta: Change from 0.1.2 release

Released: 2017.03.08

Summary:

• [Issue #164] Aligns cli syntax for extra_arguments with other cli opts

• [Issue #165] Removes ash_role from default config file

• [Issue #173] Fixes exception when re-running watchmaker

0.1.2

Commit Delta: Change from 0.1.1 release

Released: 2017.03.07

Summary:

• Adds a FAQ page to the docs

• Moves salt formulas to the correct location on the local filesystem

• join-domain-formula:

  • (Linux) Modifies decryption routine for FIPS compliance

• ash-linux-formula:

  • Removes several error exits in favor of warnings

  • (EL7-alpha) Various patches to improve support for EL7

• dotnet4-formula:

  • Adds support for .NET 4.6.2

  • Adds support for Windows Server 2016

• emet-formula:

  • Adds support for EMET 5.52

0.1.1

Commit Delta: Change from 0.1.0 release

Released: 2017.02.28

Summary:

• Adds more logging messages when downloading files

0.1.0

Commit Delta: N/A

Released: 2017.02.22

Summary:

• Initial release!