pilfer

Decrypt all ansible vault files in a project in-place recursively for viewing/editing, then re-encrypt them all at once when you're done.

Optionally decrypt/re-encrypt all encrypted variables in-place.

Features

• Python 3 compatible - Modernized for current Python versions
• ansible.cfg integration - Automatically reads vault_password_file from your ansible.cfg
• Change detection - Only re-encrypts files that were actually modified (using SHA256)
• Safe operation - Preserves original encrypted content for unchanged files
• No third-party dependencies - Uses Ansible's official vault implementation directly
• Binary data preservation - Preserves exact line endings and formatting (critical for certificates)
• Inline encrypt_string support - Opt-in via pilfer open --include-encrypted-vars; decrypts YAML !vault scalars in place (with # pilfer:vault:N markers); close always re-encrypts whatever the session opened
• Fail-closed sessions - Refuses double-open, keeps session state if close partially fails, non-zero exit codes on errors

Usage

pilfer [open|close] [-p VAULT_PASSWORD_FILE] [--include-encrypted-vars]

Basic Usage

Option 1: From a clone (no pipx)

• Clone this repository (or install editable: pip install -e .)
• From your Ansible project directory, run python /path/to/pilfer/pilfer.py open
• Edit/search plaintext as needed
• Run python /path/to/pilfer/pilfer.py close to re-encrypt any changed files
• pilfer.py is a thin entry point; the implementation lives in the pilfer/ package

Option 2: Installed via pipx (Recommended)

• Install pilfer via pipx: pipx install pilfer
• Run pilfer open to decrypt all vaulted files recursively
• Edit/search plaintext as needed
• Run pilfer close to re-encrypt any changed files

Any unchanged files will be returned to their original state.

Inline encrypted variables (encrypt_string / !vault)

Whole-file vaults are opened by default. Inline !vault scalars are opt-in:

# Open whole-file vaults AND inline encrypt_string values
pilfer open --include-encrypted-vars

# Edit values in place. pilfer rewrites each !vault block like:
#   db_password: "the-secret"  # pilfer:vault:0
#
# Do NOT remove the `# pilfer:vault:N` comment - close uses it to find
# and re-encrypt each value. Do NOT commit while those markers are present
# (plaintext secrets + session metadata would land in git).

pilfer close   # no flag needed; re-encrypts everything this session opened

close always re-encrypts session entries (whole-file and inline). The --include-encrypted-vars flag is only meaningful on open.

If you delete an entire opened variable line (key + value + marker), close treats that as intentional removal and prints:

🔍 Detected removal of 1 encrypted vars:
  - db_password

Do not strip only the # pilfer:vault:N comment while leaving the key - close will refuse so plaintext is not stranded. Renaming the key and dropping the marker is also refused if the secret value is still present in the file.

Safety notes

• Never commit while a session is open. Add vaultedFileList.json and .vault/ to your project's .gitignore.
• Never commit files that still contain # pilfer:vault:N markers - that means the session is still open.
• pilfer open refuses if a session is already open (prevents destroying encrypted backups).
• pilfer close keeps the session and exits non-zero if any file fails - check the exit code in scripts.
• Leave # pilfer:vault:N markers in place until close; removing them makes close fail closed (session kept).

Vault Password File Detection

The script automatically detects your vault password file in this order:

1. Command line argument: -p /path/to/vault/file
2. ansible.cfg: Reads vault_password_file from [defaults] section
3. Common locations:
   • ~/.ansible-vault/.vault-file
   • ../../vault_password_file
   • .vault_password
   • vault_password_file

Examples

Using the installed version:

# Use ansible.cfg vault_password_file setting (recommended)
pilfer open

# Specify custom vault password file
pilfer open -p ~/.my-vault-password

# Also decrypt inline !vault / encrypt_string values
pilfer open --include-encrypted-vars

# Close and re-encrypt modified files (and any opened inline vars)
pilfer close

Using the standalone script:

# Use ansible.cfg vault_password_file setting (recommended)
python pilfer.py open

# Specify custom vault password file
python pilfer.py open -p ~/.my-vault-password

# Also decrypt inline !vault / encrypt_string values
python pilfer.py open --include-encrypted-vars

# Close and re-encrypt modified files
python pilfer.py close

Installation

Option 1: Standalone Script (No Installation Required)

Download and use the standalone script directly:

# Download the standalone script
curl -O https://raw.githubusercontent.com/aioue/pilfer/main/pilfer.py

# Make it executable (required for ./pilfer.py usage)
chmod +x pilfer.py

# Use it directly
./pilfer.py open
# OR
python pilfer.py open

Option 2: Install via pipx (Recommended for Regular Use)

Python 3.6+ is required. Install pilfer using pipx for isolated CLI tool management:

# Install pilfer via pipx (recommended)
pipx install pilfer

# Verify installation
pilfer --help

Alternative Installation Methods

If you prefer other installation methods:

# Install from source (in development mode)
git clone https://github.com/aioue/pilfer.git
cd pilfer
pip install -e .

# Direct pip installation (not recommended for CLI tools)
pip install pilfer

Requirements

Pilfer requires Ansible to be available. If not already installed:

# Using pipx (recommended for CLI tools)
pipx install ansible

# Using pip
pip install ansible

# System package manager
# Ubuntu/Debian:
sudo apt update && sudo apt install ansible

# RHEL/CentOS/Fedora:
sudo dnf install ansible

# macOS:
brew install ansible

ansible.cfg Setup (Recommended)

Add to your ansible.cfg:

[defaults]
vault_password_file = ~/.ansible-vault/.vault-file

This eliminates the need to manually configure vault password paths.

Development and Publishing

For Developers

To set up for development:

# Clone the repository
git clone https://github.com/aioue/pilfer.git
cd pilfer

# Install in development mode
pip install -e .

# Make changes and test
pilfer --help

Publishing to PyPI

Recommended: bump the version in pyproject.toml, pilfer/__init__.py, and pilfer.py, then commit, push, and tag:

# Bump version in pyproject.toml, pilfer/__init__.py, and pilfer.py first
git commit -am "chore(release): X.Y.Z"
git push origin master
git tag vX.Y.Z
git push origin vX.Y.Z

The Release workflow validates versions, runs tests, creates a GitHub release, and publishes to PyPI via trusted publishing. See .github/workflows/README.md for one-time PyPI setup.

Manual fallback (TestPyPI or local publish):

pip install build twine
chmod +x build_and_publish.sh
./build_and_publish.sh test   # TestPyPI
./build_and_publish.sh prod   # production PyPI

The build script will:

1. Clean previous builds
2. Build the package using modern Python packaging
3. Upload to PyPI/TestPyPI using twine
4. Provide installation instructions

License

This project is licensed under the GNU General Public License v3 or later (GPLv3+). See the LICENSE file for the complete license text from the official GNU website.

Packaging Note

Due to a compatibility issue between modern setuptools (which supports SPDX license expressions) and PyPI's current metadata validation (which doesn't yet support the new format), the license file is renamed to PILFER_LICENSE.txt during packaging to avoid auto-detection issues. This is a temporary workaround until PyPI updates its metadata validation to support the newer standards.

This package heavily borrows from the excellent, but no longer supported Ansible Toolkit.

Credits

• Borrows heavily from the excellent, but no longer supported Ansible Toolkit.