ls-keystore-utils 0.2.0

Wrapper library make it easier to create and modify Logstash Secrets keystores for secure settings

ls-keystore-utils

Wrapper library to make it easier to create and modify Logstash Secrets keystores for secure settings.

Problem Solved

Logstash keystores are encrypted PKCS#12 files used to securely store sensitive configuration values, such as passwords or API keys, for Logstash pipelines. Traditionally, managing these keystores requires manual use of the logstash-keystore CLI binary, which is slow because of the need to spin up a JVM, however small, and lacks programmatic integration. The LogstashKeystore class partly solves this by providing a Python-based interface for creating, reading, updating, and deleting keystore entries programmatically, while ensuring compatibility with Logstash's proprietary format.

How It Works

The LogstashKeystore class actually still leverages the official logstash-keystore binary (typically located at /usr/share/logstash/bin/logstash-keystore or best-guess auto-detected) for all write operations (adding, removing, or updating keys) to maintain full compatibility and security. For read operations, it uses custom cryptographic parsing via the cryptography and asn1crypto modules to efficiently retrieve key values without relying on the binary (which does not reveal key values at all). Unfortunately, these libraries do not appear to be able to write new keys to a logstash-keystore generated file. But it is still quite useful to be able to read the values and cache them in an AES-obfuscated memory store.

To reduce redundant calls to the logstash-keystore binary and minimize disk I/O, the class caches keystore data in memory, including obfuscated values and timestamps. Before each operation, it checks for external modifications by comparing cached timestamps with the keystore file's latest timestamp, and/or detecting additions, removals, or value changes. This caching mechanism ensures data consistency, prevents silent overwrites, and allows for efficient batch operations pass multiple keys in a single CLI call with serialized input. With value obfuscated value caching, it can reduce the need to overwrite existing values or execute a full Logstash pipeline just to see if the value is correct, or needs updating.

Installation

pip install -e .

or

pip install ls-keystore-utils

Usage

Instantiation

Create a new keystore or load an existing one. The class automatically detects the logstash-keystore binary path if not provided.

from ls_keystore_utils import LogstashKeystore

# Create a new keystore
ks = LogstashKeystore.create(
    path_settings="/path/to/logstash/config",
    password="mypassword"
)

# Load an existing keystore
ks = LogstashKeystore.load(
    path_settings="/path/to/logstash/config",
    password="mypassword"
)

Adding Keys

Add single or multiple keys. Batch mode is more efficient as it uses one CLI call for all keys.

# Add a single key
ks.add_key("MY_KEY", "secret_value")

# Add multiple keys in batch mode (must be dict of key/value pairs)
ks.add_key({"KEY1": "value1", "KEY2": "value2"})

Removing Keys

Remove single or multiple keys. Batch mode removes all keys in one CLI call.

# Remove a single key
ks.remove_key("MY_KEY")

# Remove multiple keys in batch mode
ks.remove_key(["KEY1", "KEY2"])

Updating Keys

Update existing keys (overwrites values). Same as adding; batch mode supported.

# Update a single key
ks.update_key("EXISTING_KEY", "new_value")

# Update multiple keys in batch mode
ks.update_key({"KEY1": "new_val1", "KEY2": "new_val2"})

Deleting Keys

Delete keys (alias for removing). Supports single and batch modes like removal.

# Delete a single key
ks.delete_key("MY_KEY")

# Delete multiple keys in batch mode
ks.delete_key(["KEY1", "KEY2"])

Compare value to keystore value

More information on the ObfuscatedValue class may be found in the code in crypto.py or in future documentation.

# Compare a known value to the value in the keystore
key = "MY_KEY"
secret = "my_secret_value"
assert secret == ks._current[key].obfuscated_value.deobfuscate()

Features

• Create/Update/Delete: Full CRUD operations via logstash-keystore binary with batch support for efficiency.
• Read Operations: Retrieve individual or all key values using cached, parsed data.
• List Keys: Get a list of all key names in the keystore.
• Validation: Verify keystore integrity and detect external modifications.
• Backups: Create file backups of the keystore.
• Metadata: Track timestamps for change detection and age monitoring.
• Change Tracking: Automatically detects modifications and flags when Logstash restart is required.
• Caching: In-memory cache reduces redundant binary calls and ensures consistency.

Change Tracking

If a key has been updated with a new value, Logstash might need to be restarted to get that value since it is converted to an environment variable. If this is the case, then the LogstashKeystore class will set property needs_restart to True.

if ks.needs_restart is True:
    # do stuff here