Metadata-Version: 2.5
Name: mind-castle
Version: 0.7.1
Summary: A universal secrets manager
License-File: LICENSE
Requires-Python: >=3.10
Requires-Dist: aws-encryption-sdk==3.3.0
Requires-Dist: boto3
Requires-Dist: hvac
Requires-Dist: hypothesis-jsonschema>=0.23.1
Requires-Dist: pydantic>=2.11.4
Requires-Dist: rich>=13.9.4
Requires-Dist: sqlalchemy<3.0.0,>=2.0.0
Requires-Dist: typer>=0.12.5
Requires-Dist: wrapt>=1.17.2
Provides-Extra: django
Requires-Dist: django>=5.2; extra == 'django'
Provides-Extra: migrate
Requires-Dist: iterfzf==1.8.0.62.0; extra == 'migrate'
Description-Content-Type: text/markdown

# Mind Castle - Build a wall around your secrets

A universal store for your secret data. Don't delay securing you or your customer's data by deliberating over cloud secret stores. Mind Castle makes it easy to get started, and easy to switch between cloud secret stores.

Mind Castle currently supports:
- AWS KMS
- Local Encryption
- None (passthrough - Mind Castle doesn't modify your data)

## Architecture

Mind Castle comes in three parts:
- A unified interface for several secret stores.
- An SQLAlchemy column type, and a Django model field, that transparently store and retrieve secrets for you.
- A migration tool to convert your existing DB column data into secrets.

Some other notes:
- Mind Castle is configured and secret stores are initialised at import time. That means env-vars used for configuration need to be defined when Mind Castle is imported.
- Mind Castle makes no attempt to manage secrets in memory. [Memory management](https://stackoverflow.com/questions/728164/securely-erasing-password-in-memory-python) in [Python](https://discuss.python.org/t/how-to-hide-or-remove-sensitive-data-from-getting-exposed-in-memory-dump/44526) is [futile](https://stackoverflow.com/questions/41509771/python-remove-password-from-memory), and if you need that level of control it's best to use another language.


## Install

`pip install mind-castle`


## Configure

You can configure Mind Castle by setting environment variables for your chosen secret store. To see what configuration options are required for each store:

```bash
$ python -m mind_castle
╭────────────────────────────────────────── MIND CASTLE ───────────────────────────────────────────╮
│                                                                                                  │
│                                          Secret Stores                                           │
│ ┏━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ │
│ ┃ Store Type      ┃ Required env var                    ┃ Optional env var                     ┃ │
│ ┡━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ │
│ │ localencryption │ MIND_CASTLE_LOCAL_ENCRYPTION_SECRET │                                      │ │
│ │                 │                                     │                                      │ │
│ │ awskms          │ MIND_CASTLE_AWS_REGION              │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE    │ │
│ │                 │ MIND_CASTLE_AWS_ACCESS_KEY_ID       │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_T… │ │
│ │                 │ MIND_CASTLE_AWS_SECRET_ACCESS_KEY   │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_C… │ │
│ │                 │ MIND_CASTLE_AWS_KMS_KEY_ARN         │                                      │ │
│ │                 │ --- OR ---                          │                                      │ │
│ │                 │ MIND_CASTLE_AWS_REGION              │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE    │ │
│ │                 │ MIND_CASTLE_AWS_USE_ENV_AUTH        │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_T… │ │
│ │                 │ MIND_CASTLE_AWS_KMS_KEY_ARN         │ MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_C… │ │
│ │                 │                                     │                                      │ │
│ └─────────────────┴─────────────────────────────────────┴──────────────────────────────────────┘ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
```

`"none"` is a fourth store type you can pass to a column or field. It is a
passthrough handled by the column type itself rather than a store, so it takes
no configuration and does not appear in that table. Anything else raises
`ValueError: Trying to put secret to unknown/unconfigured store '<name>'` the
first time you save.

### AWS KMS data-key cache (optional)

The `awskms` store uses envelope encryption: each secret is sealed with its own
data key, which KMS must decrypt on every read. When the same secret is read
repeatedly (e.g. frequent full re-fetches), this drives repeated KMS `Decrypt`
calls. You can opt in to caching the decrypted data keys in-process:

| Env var | Description | Default |
| --- | --- | --- |
| `MIND_CASTLE_AWS_KMS_DATAKEY_CACHE` | Enable the cache (`true`/`1`/`yes`) | off |
| `MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_TTL` | Cache entry TTL, seconds | `60` |
| `MIND_CASTLE_AWS_KMS_DATAKEY_CACHE_CAPACITY` | Max cached data keys | `1000` |

Notes:
- Caching applies to decryption only; `create_secret` still generates a fresh
  data key per secret. Legacy non-envelope secrets are never cached.
- A revoked/rotated KMS key keeps working from cache until the TTL expires — keep
  the TTL short. Invalid or non-positive TTL/capacity disables the cache.
- The cache is per-process (each worker has its own) and holds decrypted data
  keys in memory (up to `CAPACITY`, until TTL). Config is read at startup;
  changing it requires a restart.

## Use

### Migrating Existing Data
A migration tool is provided to move your existing data into a secret store. The tool assumes you have a database that can be connected to using SQLAlchemy and `create_engine(<db_uri>)`.

You will need to configure your selected secret store using environment variables as described above. The migration tool depends on `iterfzf`, which isn't installed by the base package - install it with `pip install "mind-castle[migrate]"`.

You can see what other options the migration tool accepts with:

```bash
$ python migration_tool.py --help

 Usage: migration_tool.py [OPTIONS] DB_URI

╭─ Arguments ─────────────────────────────────────────────────────────────────────────╮
│ *    db_uri      TEXT  [default: None] [required]                                   │
╰─────────────────────────────────────────────────────────────────────────────────────╯
╭─ Options ───────────────────────────────────────────────────────────────────────────╮
│ --target-table                        TEXT  [default: None]                         │
│ --target-column                       TEXT  [default: None]                         │
│ --dry-run           --no-dry-run            [default: dry-run]                      │
│ --demigrate         --no-demigrate          [default: no-demigrate]                 │
│ --to-secret-type                      TEXT  [default: json]                         │
│ --help                                      Show this message and exit.             │
╰─────────────────────────────────────────────────────────────────────────────────────╯
```

The easiest thing to do is specify `--to-secret-type` and let the tool guide you through the rest. e.g.:

```bash
$ MIND_CASTLE_LOCAL_ENCRYPTION_SECRET="<your_base64_key>" python migration_tool.py "postgresql://<user>:<pass>@<host>:5432/<database>" --to-secret-type localencryption
```

The tool will list tables and columns in the selected DB for you, and ask you to select each. Once a dry run is complete and **you also have a backup of your database**, you can add `--no-dry-run` to migrate the data for real.


### SQLAlchemy Model
Once any existing data is migrated, you can update your SQLAlchemy model to include a `SecretData` column:

```python
from mind_castle.sqlalchemy_type import SecretData

class MyDBModel(Base):
    name = Column(String, nullable=False)
    created_at = Column(DateTime, default=datetime.datetime.now)
    secret_data = Column(SecretData("localencryption"))
```

Your secrets will then be safely stored in AWS (or encrypted locally, or anywhere else you like)! The storage and retrieval of secrets will be completely transparent to your application.

### Django Model
For a Django model, use `mind_castle.django_type.SecretData` instead - it's a `JSONField` subclass with the same encrypt/decrypt behavior. Install the `django` extra (`pip install "mind-castle[django]"`) first.

```python
from mind_castle.django_type import SecretData

class MyDBModel(models.Model):
    name = models.CharField(max_length=255)
    created_at = models.DateTimeField(auto_now_add=True)
    secret_data = SecretData("localencryption", null=True)
```

Notes specific to the Django field:
- Filtering or looking up by value (`MyDBModel.objects.filter(secret_data=...)`, `secret_data__somekey=...`) raises `FieldError` on any store except `"none"` - the stored value is ciphertext with a fresh nonce per encryption, so a value-equality comparison could otherwise silently match nothing. Null checks still work, written either way: `filter(secret_data=None)` and `filter(secret_data__isnull=True)`.
- Admin and ModelForm rendering show the decrypted value, matching a normal `JSONField`'s editable UX. `dumpdata`/`serializers.serialize()` always export the encrypted envelope, and `loaddata` reads that envelope back without re-encrypting it, so a fixture round-trips.

## How values are compared, changed and moved around

These three rules apply to both the SQLAlchemy and the Django field types.

### Changing a value

Reassigning the field and editing it in place both work:

```python
obj.secret_data = {**obj.secret_data, "key": "new"}   # reassign
obj.secret_data["key"] = "new"                        # edit in place
```

A decrypted value is handed back inside a wrapper that keeps a copy of what it decrypted, so an edit to the contents is still noticed and the value is re-encrypted on save.

One caveat on SQLAlchemy: editing in place does not mark the attribute dirty, which is a general SQLAlchemy behaviour and not specific to this type. Nothing is written unless something else on that row changes, or you call `flag_modified(obj, "secret_data")`. Django writes every field on `save()`, so it needs nothing extra.

### Pickling

Pickling a decrypted value, or a whole model instance holding one, writes the **encrypted envelope** into the pickle and decrypts it again on load. That is what lets an instance cross a process or network boundary (Django's cache framework, a Celery task argument, `multiprocessing`) without the plaintext travelling. The loading process needs the matching store configured or the load raises. Pickling a value that was edited since it was decrypted also raises, because the envelope would still hold the old one.

### What counts as an encrypted value

On any store except `"none"`, a stored `dict` carrying both a `mind_castle_secret_type` and an `encrypted_value` key is treated as ciphertext, and one that does not is passed through as plaintext. That single rule is used everywhere a value crosses a boundary: reading a column, loading a fixture, unpickling. A dict that claims to be an envelope but does not parse as one is a corrupt row and raises, rather than being handed back as if it were your data.


## TODO

- Make migration script work for non-json columns
- Support deleting secrets when row is deleted
- Implement prefixes/folders for secrets
- Explain how secrets are stored
- Enforce tests on PR / branch protections
- Wire `secret_key_prefix` through to the stores, or drop it from the signature
