oxl-ansible-executor 0.8

Simple & Transparent Executor for Ansible-Playbooks

Ansible-Playbook Executor

---

DISCLAIMER: This is an unofficial community project! Do not confuse it with the vanilla Ansible/Ansible-Runner product!

WARNING: This project is still in early development. DO NOT use it in production!

---

Scope

The scope of this project is it to create a simple and transparent Python3-interface that can be used to execute Ansible-playbooks.

It will focus on using docker or podman to execute ansible in an isolated environment. But local execution will also be available.

The implementation will be opinionated and will have a 'narrow' interface.

Source code: oxl-ansible-executor & oxl-ansible-executor-plugins

Motivation

I was not 100% happy with the official ansible-runner library.

• ansible-runner provides a lot more functionality than we actually need (only executing ansible-playbooks) and thus has a lot more complexity added-on. This makes it also hard to troubleshoot.

• ansible-runner has lax input-validation - bad user-input might be silently skipped/ignored and thus lead to unexpected behaviour

As I needed an alternative I wanted to provide it to the community to play with. (:

I will try to create a transparent documentation and a lot of unit- & integration-tests!

All user-inputs are validated for data-type and value.

Feel free to give feedback as GitHub issues or email.

---

Roadmap

• Ansible Execution
  • Config Object & Validation incl. inline-docs
  • Engines
    • Local Executor
    • Container Executor
      • Docker
      • Podman
  • Functionality
    • Playbook targeting local machine
    • Playbook targeting remote Linux server (SSH-Key, Connect-Pass, Become-Pass, Vault-Pass)
      • Pass secrets via one-time-read Pipes/FIFO (in-memory - does not write to disk)
    • Stopping job
    • Redirect output (stdout/stderr) to log-files
    • Playbook Status (per category, per host)
      • Track stati at runtime
• Tests
  • Unit-Tests for all components (>85% coverage)
    • Execution Config-Validation
    • Execution
      • Before
        • Base-Executor
        • Local Executor-Engine
        • Containerized Executor-Engine
      • After
      • Execution-Status
  • Integration-Tests for many practical use-cases
    • Simple execution targeting localhost
    • Passing extra-vars
    • Passing env-vars
    • Enabling output-colors
    • User stopping execution
    • Execution reached timeout
    • Passing secrets (as value & as file)
      • SSH-Key (via ssh-agent)
      • Connect-Pass
      • Become-Pass
      • Ansible-Vault
    • SSH
      • SSH-Key usage
      • Known-Hosts file
    • tbc...
  • Integration-Tests also for containerized executor

---

Install

pip install oxl-ansible-executor

See: pypi.org/oxl-ansible-executor

Custom Container-Image for execution

Use the default/fallback container image as a template.

Execution Stats

If you want to have access to execution-stats - make sure to install and enable the plugin:

# inside the execution-venv or -container
pip install oxl-ansible-executor-plugins

# enable the plugin by adding its path to the env-variable
ANSIBLE_CALLBACK_PLUGINS="${ANSIBLE_CALLBACK_PLUGINS:+${ANSIBLE_CALLBACK_PLUGINS}:}$(oxl-ansible-executor-plugins-callback)"

---

Usage

from oxl_ansible_executor import Execution, ExecutionConfig

c = ExecutionConfig(
  playbook_dir='/home/demo/ansible/',
  playbook_file='test.yml',
  inventory_files='inv/env1/hosts.yml',
  debug=True,  # output infos to stdout (for testing purposes)
  load_log_stdout=True,  # loads the stdout for the execution-result; not recommended for long-running jobs
  load_log_stderr=True,  # loads the stderr for the execution-result
)
e = Execution(c)

e.run(blocking=True)
# [DEBUG] Using executor: local
# [DEBUG] Creating log-files
# [DEBUG] Creating secret-pipes
# [DEBUG] Using log files: /home/demo/.local/share/oxl-ansible-executor/ansible_1775500760_lhWOT_stdout.log & /home/demo/.local/share/oxl-ansible-executor/ansible_1775500760_lhWOT_stderr.log
# [DEBUG] Executing ansible-playbook
# [DEBUG] Command: ['ssh-agent', 'sh', '-c', 'ssh-add /tmp/ar_znjp4bih/.fIWqfDljSw && /home/demo/.venv/bin/ansible-playbook -i inv/abc/hosts.yml -C -D -l srv1 --become-pass-file /tmp/ar_znjp4bih/.SHVTBTAgVJ --vault-pass-file /tmp/ar_znjp4bih/.GwGY8BnyF2 syslog.yml']
# [DEBUG] Process finished
# [DEBUG] Process exit-code: 0
# [DEBUG] Post-execution tasks
# [DEBUG] Cleaning stats-sections from log-files

print(e.status)
# {
#   "finished": true,
#   "playbook_finished": true,
#   "failed": false,
#   "canceled": false,
#   "time_start": 1775500758,
#   "time_finish": 1775500783,
#   "timed_out": false,
#   "time_duration_sec": 25,
#   "stats": {
#     "srv1": {
#       "ok": 7,
#       "failures": 0,
#       "unreachable": 0,
#       "changed": 1,
#       "skipped": 0,
#       "rescued": 0,
#       "ignored": 0
#     }
#   },
#   "log_stdout_file": "/home/demo/.local/share/oxl-ansible-executor/ansible_1775500760_lhWOT_stdout.log",
#   "log_stderr_file": "/home/demo/.local/share/oxl-ansible-executor/ansible_1775500760_lhWOT_stderr.log",
#   "ansible_command": [
#     "ansible-playbook",
#     "-i",
#     "inv/abc/hosts.yml",
#     "-C",
#     "-D",
#     "-l",
#     "srv1",
#     "--become-pass-file",
#     "/tmp/ar_znjp4bih/.SHVTBTAgVJ",
#     "--vault-pass-file",
#     "/tmp/ar_znjp4bih/.GwGY8BnyF2",
#     "syslog.yml"
#   ],
#   "process_command": [
#     "ssh-agent",
#     "sh",
#     "-c",
#     "ssh-add /tmp/ar_znjp4bih/.fIWqfDljSw && /home/demo/.venv/bin/ansible-playbook -i inv/abc/hosts.yml -C -D -l srv1 --become-pass-file /tmp/ar_znjp4bih/.SHVTBTAgVJ --vault-pass-file /tmp/ar_znjp4bih/.GwGY8BnyF2 syslog.yml"
#   ],
#   "process_rc": 0,
#   "process_result": {
#     "failed": false,
#     "rc": 0,
#     "pid": null,
#     "stdout": "<OMITTED FOR DEMO>",
#     "stderr": "<OMITTED FOR DEMO>",
#     "stdout_lines": [
#       "PLAY [all] *********************************************************************",
#       "",
#       "TASK [Install rsyslog & logrotate] *********************************************",
#       "ok: [srv1]",
#       "",
#       "TASK [Add certificates] ********************************************************",
#       "ok: [srv1] => (item={'c': '-----BEGIN CERTIFICATE-----\\nMIIDdjCCAv2gAwIBAgIUUkucdMI33le6pBxGgw6WRP2yPlcwCgYIKoZIzj0EAwIw\\ngZExCzAJBgNVBAYTAkFUMQ8wDQYDVQQIDAZTdHlyaWExDTALBgNVBAcMBEdyYXox\\nGDAWBgNVBAoMD09YTCBJVCBTZXJ2aWNlczESMBAGA1UECwwJTG9nc2VydmVyMRUw\\nEwYDVQQDDAxMb2dzZXJ2ZXIgQ0ExHTAbBgkqhkiG9w0BCQEWDmNvbnRhY3RAb3hs\\nLmF0MB4XDTI0MTIyNDEzMjExOFoXDTQ0MTIxOTEzMjExOFowgZExCzAJBgNVBAYT\\nAkFUMQ8wDQYDVQQIDAZTdHlyaWExDTALBgNVBAcMBEdyYXoxGDAWBgNVBAoMD09Y\\nTCBJVCBTZXJ2aWNlczESMBAGA1UECwwJTG9nc2VydmVyMRUwEwYDVQQDDAxMb2dz\\nZXJ2ZXIgQ0ExHTAbBgkqhkiG9w0BCQEWDmNvbnRhY3RAb3hsLmF0MHYwEAYHKoZI\\nzj0CAQYFK4EEACIDYgAEUwaiRMy1OcN0j5odvoTir6LFAqAlUlDp708Y39ZdFduv\\nXMEJGKWeHUKnoMT4uaDEomlwqEUa5JcG3Z9R0PeCJsNXvns53uK1j1uMY395yIvW\\nyQ8yu1vDhD/OnnOMl6oro4IBEjCCAQ4wDAYDVR0TBAUwAwEB/zAdBgNVHQ4EFgQU\\nvk1BL8cxa/eNRSQDD/T+yTyA7sswgdEGA1UdIwSByTCBxoAUvk1BL8cxa/eNRSQD\\nD/T+yTyA7suhgZekgZQwgZExCzAJBgNVBAYTAkFUMQ8wDQYDVQQIDAZTdHlyaWEx\\nDTALBgNVBAcMBEdyYXoxGDAWBgNVBAoMD09YTCBJVCBTZXJ2aWNlczESMBAGA1UE\\nCwwJTG9nc2VydmVyMRUwEwYDVQQDDAxMb2dzZXJ2ZXIgQ0ExHTAbBgkqhkiG9w0B\\nCQEWDmNvbnRhY3RAb3hsLmF0ghRSS5x0wjfeV7qkHEaDDpZE/bI+VzALBgNVHQ8E\\nBAMCAQYwCgYIKoZIzj0EAwIDZwAwZAIwHbFj7SEsIg3dUBpAkKIfLWVrpKicxDUe\\nKqUd/hcXHX/TTKKd0yehyXhZR0HBYgHEAjAbxJb8pQIVzCowHcXmv790wOiSsaFL\\nQJuH8qil1t0JSAV73EdK9zM+IDdvBVdDlBw=\\n-----END CERTIFICATE-----\\n', 'f': 'log_ca.crt'})",
#       "ok: [srv1] => (item={'c': '-----BEGIN CERTIFICATE-----\\nMIIDgTCCAwagAwIBAgIQX7p1IkBq4b97O8k0caiMfjAKBggqhkjOPQQDAjCBkTEL\\nMAkGA1UEBhMCQVQxDzANBgNVBAgMBlN0eXJpYTENMAsGA1UEBwwER3JhejEYMBYG\\nA1UECgwPT1hMIElUIFNlcnZpY2VzMRIwEAYDVQQLDAlMb2dzZXJ2ZXIxFTATBgNV\\nBAMMDExvZ3NlcnZlciBDQTEdMBsGCSqGSIb3DQEJARYOY29udGFjdEBveGwuYXQw\\nHhcNMjQxMjI0MTQyNzA4WhcNMjkxMjIzMTQyNzA4WjCBjDELMAkGA1UEBhMCQVQx\\nDzANBgNVBAgMBlN0eXJpYTENMAsGA1UEBwwER3JhejEYMBYGA1UECgwPT1hMIElU\\nIFNlcnZpY2VzMRIwEAYDVQQLDAlMb2dzZXJ2ZXIxEDAOBgNVBAMMB2dlbmVyaWMx\\nHTAbBgkqhkiG9w0BCQEWDmNvbnRhY3RAb3hsLmF0MHYwEAYHKoZIzj0CAQYFK4EE\\nACIDYgAEF2uLX4WUd/4byt3aS59XxohILfEdnNvqdg87nG0PXC85yhtB5yKMWryT\\n+cWZeGYSucdLtZ4UxDqvosQ32eql/K+VoVRqkb4UF1Mmgq5+smhHIkVxFjJCUtsg\\ni+G9aPM/o4IBJDCCASAwCQYDVR0TBAIwADAdBgNVHQ4EFgQUmULR2FK41wLFEl/B\\nSGhYkQ0Z5v8wgdEGA1UdIwSByTCBxoAUvk1BL8cxa/eNRSQDD/T+yTyA7suhgZek\\ngZQwgZExCzAJBgNVBAYTAkFUMQ8wDQYDVQQIDAZTdHlyaWExDTALBgNVBAcMBEdy\\nYXoxGDAWBgNVBAoMD09YTCBJVCBTZXJ2aWNlczESMBAGA1UECwwJTG9nc2VydmVy\\nMRUwEwYDVQQDDAxMb2dzZXJ2ZXIgQ0ExHTAbBgkqhkiG9w0BCQEWDmNvbnRhY3RA\\nb3hsLmF0ghRSS5x0wjfeV7qkHEaDDpZE/bI+VzALBgNVHQ8EBAMCBaAwEwYDVR0l\\nBAwwCgYIKwYBBQUHAwIwCgYIKoZIzj0EAwIDaQAwZgIxAKVAFBs4EaCRx/0VEZKA\\n3/n06CaEE5I05v8kN5XBWgbJPxaDi+bcRJrzBMsn/JRjvwIxAMi4cekyGCWZYDtM\\n+8WRliIMba5dbFuR/UGAf/bIfmNWM2sSqtzseSZ/RoPdjaChlQ==\\n-----END CERTIFICATE-----\\n', 'f': 'log_client.crt'})",
#       "",
#       "TASK [Add certificate key] *****************************************************",
#       "ok: [srv1]",
#       "",
#       "TASK [Add graylog forwarding] **************************************************",
#       "ok: [srv1]",
#       "",
#       "TASK [Software log files] ******************************************************",
#       "ok: [srv1]",
#       "",
#       "TASK [Software log rotation] ***************************************************",
#       "ok: [srv1]",
#       "",
#       "TASK [Restart services] ********************************************************",
#       "changed: [srv1] => (item=rsyslog.service)",
#       "changed: [srv1] => (item=logrotate.service)",
#       "",
#       "PLAY RECAP *********************************************************************",
#       "srv1                       : ok=7    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0"
#     ],
#     "stderr_lines": [
#       "Identity added: /tmp/ar_znjp4bih/.fIWqfDljSw (Demo)"
#     ]
#   }
# }

# to stop a running execution
e = Execution(c)
e.run(blocking=False)
# you could 'tail -f' the log files
e.stop()  # executor sends signals to subprocess running ansible

---

Use-Cases

See: Our simple Ansible WebUI

---

Contribute

We are happy to see contributions. (:

• Report issues
• Create feature-requests
• Provide PR's for:
  • more Unit-Tests
  • more Integration-Tests
  • fixing bugs/errors
  • enhancing the input-validation
  • ...

---

Security Considerations

• Secrets are passed to Ansible via one-time-readable FIFO/Pipes
• SSH-agent is used to pass SSH-keys to Ansible
• Files are created with an explicit 0600 file-mode
• By default, the temporary runtime-directory is removed after the execution has finished (only log-files remain - see example above)
• tbc

---

AI-Usage Info

The coding of this project involved minimal AI-usage.