Metadata-Version: 2.3
Name: ls-keystore-utils
Version: 0.3.0
Summary: Wrapper library make it easier to create and modify Logstash Secrets keystores for secure settings
Requires-Dist: asn1crypto>=1.5.0
Requires-Dist: cryptography>=3.0
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# ls-keystore-utils

Wrapper library to make it easier to create and modify Logstash Secrets keystores for secure settings.

## Problem Solved

Logstash keystores are encrypted PKCS#12 files used to securely store sensitive configuration values, such as passwords or API keys, for Logstash pipelines. Traditionally, managing these keystores requires manual use of the `logstash-keystore` CLI binary, which is slow because of the need to spin up a JVM, however small, and lacks programmatic integration. The `LogstashKeystore` class partly solves this by providing a Python-based interface for creating, reading, updating, and deleting keystore entries programmatically, while ensuring compatibility with Logstash's proprietary format.

## How It Works

The `LogstashKeystore` class actually still leverages the official `logstash-keystore` binary (typically located at `/usr/share/logstash/bin/logstash-keystore` or best-guess auto-detected) for all write operations (adding, removing, or updating keys) to maintain full compatibility and security. For read operations, it uses custom cryptographic parsing via the cryptography and asn1crypto modules to efficiently retrieve key values without relying on the binary (which does not reveal key values at all). Unfortunately, these libraries do not appear to be able to *write* new keys to a `logstash-keystore` generated file. But it is still quite useful to be able to read the values and cache them in an AES-obfuscated memory store.

To reduce redundant calls to the `logstash-keystore` binary and minimize disk I/O, the class caches keystore data in memory, including obfuscated values and timestamps. Before each operation, it checks for external modifications by comparing cached timestamps with the keystore file's latest timestamp, and/or detecting additions, removals, or value changes. This caching mechanism ensures data consistency, prevents silent overwrites, and allows for efficient batch operations pass multiple keys in a single CLI call with serialized input. With value obfuscated value caching, it can reduce the need to overwrite existing values or execute a full Logstash pipeline just to see if the value is correct, or needs updating.

## Installation

```bash
pip install -e .
```

or

```bash
pip install ls-keystore-utils
```

## Usage

### Instantiation

Create a new keystore or load an existing one. The automatic detection of the `path.settings` and location of the `logstash-keystore` binary path are attempted if not provided. If not specified, `path_settings` will be guessed as `/etc/logstash`, or `/opt/homebrew/etc/logstash`, and `exepath` will be guessed as `/usr/share/logstash/bin/logstash-keystore` or look for `/opt/homebrew/Cellar/logstash/*/libexec/bin/logstash-keystore`.

If you don't provide a value for `salt_iv`, one will be automatically generated to help obfuscate values in memory. If you provide an `ObfuscatedValue` object for `obvpassword`, you *must* provide a value for `salt_iv` that will de-obfuscate it. LogstashKeystore will continue to use that `salt_iv` value to obfuscate the values read from the Logstash Secret store file.

```python
from ls_keystore_utils import LogstashKeystore, ObfuscatedValue, generate_salt_iv

# PREFERRED
# Create a new keystore with an obfuscated password
salt_iv = generate_salt_iv()
obv = ObfuscatedValue("mypassword", salt_iv)
ks = LogstashKeystore.create(
    path_settings="/path/to/logstash/config",
    exepath="/path/to/bin/logstash-keystore",
    salt_iv=salt_iv,
    obvpassword=obv
) 

# Create a new keystore with a plain text password
ks = LogstashKeystore.create(password="mypassword")

# PREFERRED
# Load an existing keystore using an obfuscated password
salt_iv = generate_salt_iv()
obv = ObfuscatedValue("mypassword", salt_iv)
ks = LogstashKeystore.load(
    path_settings="/path/to/logstash/config",
    exepath="/path/to/bin/logstash-keystore",
    salt_iv=salt_iv,
    obvpassword=obv
)
```

### Adding Keys

Add single or multiple keys. Batch mode is more efficient as it uses one CLI call for all keys.

```python
# Add a single key
ks.add_key("MY_KEY", "secret_value")

# Add multiple keys in batch mode (must be dict of key/value pairs)
ks.add_key({"KEY1": "value1", "KEY2": "value2"})
```

### Removing Keys

Remove single or multiple keys. Batch mode removes all keys in one CLI call.

```python
# Remove a single key
ks.remove_key("MY_KEY")

# Remove multiple keys in batch mode
ks.remove_key(["KEY1", "KEY2"])
```

### Updating Keys

Update existing keys (overwrites values). Same as adding; batch mode supported.

```python
# Update a single key
ks.update_key("EXISTING_KEY", "new_value")

# Update multiple keys in batch mode
ks.update_key({"KEY1": "new_val1", "KEY2": "new_val2"})
```

### Deleting Keys

Delete keys (alias for removing). Supports single and batch modes like removal.

```python
# Delete a single key
ks.delete_key("MY_KEY")

# Delete multiple keys in batch mode
ks.delete_key(["KEY1", "KEY2"])
```

### Compare value to keystore value

More information on the `ObfuscatedValue` class may be found in the code in `crypto.py` or in future documentation.

The `ObfuscatedValue` class has an equality method `__eq__()` that can compare between other `ObfuscatedValue` class objects, or just encrypted similarly with the same `salt_iv` value. With the `salt_iv` value, it's also possible to reveal the unobfuscated value.

```python
# Compare two ObfuscatedValue objects
>>> secret = "my_secret_value"
>>> salt_iv = b'\\x00' * 32
>>> ov1 = ObfuscatedValue(secret, salt_iv)
>>> ov2 = ObfuscatedValue(secret, salt_iv)
>>> assert ov1 == ov2
True
```

```python
# Compare an ObfuscatedValue object with an obfuscated value
>>> from ls_keystore_utils.crypto import obfuscate_value
>>> secret = "my_secret_value"
>>> salt_iv = b'\\x00' * 32
>>> ov1 = ObfuscatedValue(secret, salt_iv)
>>> obfuscated = obfuscate_value(secret, salt_iv)
>>> assert ov1 == obfuscated
True
```

```python
# Extract the unobfuscated value from the cache
# Assume that we've already instantiated ks as a LogstashKeystore object and that
# it has extracted MY_KEY with secret from an existing keystore file, and that the
# salt_iv value used to obfuscate it is 32 empty bytes.
>>> key = "MY_KEY"
>>> secret = "my_secret_value"
>>> salt_iv = b'\\x00' * 32
>>> assert secret == ks._current[key].obfuscated_value.reveal(salt_iv)
True
```

## Features

- **Create/Update/Delete**: Full CRUD operations via `logstash-keystore` binary with batch support for efficiency.
- **Read Operations**: Retrieve individual or all key values using cached, parsed data.
- **List Keys**: Get a list of all key names in the keystore.
- **Validation**: Verify keystore integrity and detect external modifications.
- **Backups**: Create file backups of the keystore.
- **Metadata**: Track timestamps for change detection and age monitoring.
- **Change Tracking**: Automatically detects modifications and flags when Logstash restart is required.
- **Caching**: In-memory cache reduces redundant binary calls and ensures consistency.

### Change Tracking

If a key has been updated with a new value, Logstash might need to be restarted to get that value since it is converted to an environment variable. If this is the case, then the `LogstashKeystore` class will set property `needs_restart` to `True`.

```python
if ks.needs_restart is True:
    # do stuff here
```
