Metadata-Version: 2.4
Name: scalio-backup
Version: 0.1.0
Summary: Encrypted export and import (backup, restore, move) shared by the Scalio apps
License: Proprietary
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Requires-Dist: cryptography>=42
Requires-Dist: pymongo>=4.6
Requires-Dist: packaging>=23
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: mongomock-motor>=0.0.29; extra == "dev"
Requires-Dist: typing-extensions>=4.8; extra == "dev"
Requires-Dist: pyrage>=1.1; extra == "dev"
Requires-Dist: motor>=3.3; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"

# scalio-backup

Encrypted export and import of a whole Scalio app instance, shared by the Scalio apps (Nexus first, then
Foundry, Rosetta, Citadel). Used for backup/restore and for moving an instance between self-hosted and SaaS.
Concept and decisions E1-E12: https://claude.ai/artifact/Ej8DFrgi4hcFDQ69dvKWZ7

Nothing in this package is secret. The password is the customer's; without it nobody (Scalio included) can
open a bundle.

## Bundle format (`.scaliobak`)

- Outside: standard [age v1](https://age-encryption.org/v1) with a passphrase (scrypt, work factor 2^18),
  ChaCha20-Poly1305 in 64 KiB chunks. `age -d backup.scaliobak > backup.tar` opens it without Scalio.
- Inside: a tar stream. `manifest.json` first (format `scalio-backup/1`, product, app and schema version,
  source, options, SHA-256 per part), then `collections/<name>.jsonl` (MongoDB Extended JSON, canonical, so
  dates, Int64 and ObjectIds stay exact), `meta.jsonl`, and for file folders `files/<dir>.index.jsonl` plus the
  files. The import checks every hash; a changed or incomplete bundle is refused before anything is written.

## Classes

Every collection (and every document of a mixed `meta` collection) has one class: `config` (always exported),
`data` (exported unless "configuration only"), `cache`, `ephemeral`, `instance` (never exported). Each app keeps
its classes in its own `backup_spec.py` and a test that fails on an unclassified collection.

## Use in an app

```python
from scalio_backup import AppSpec, Preserve, apply_import, export_instance, open_bundle, recover, rollback, purge

SPEC = AppSpec(product="scalio-nexus", app_version=VERSION, collections=..., meta_ids=...,
               secret_fields={"settings": ("entra.clientSecret", ...)}, file_dirs={"hr": HR_UPLOAD_DIR},
               schema_version=current_schema, export_hook=decrypt_instance_secrets, import_hook=encrypt_again)

# export (background job): writes the encrypted bundle to a file object
manifest = await export_instance(db, SPEC, out, password, include_data=True, include_secrets=True,
                                 created_by=user_email, source={"mode": "selfhosted", "host": host, "tenant": tid})

# import, step 1: decrypt and verify into a work folder, nothing changes yet (WrongPassword, BundleError,
# IncompatibleBundle when the product differs or the bundle is newer than this instance)
bundle = open_bundle(upload, password, SPEC, target_schema=await current_schema(db))
show(bundle.preview())          # source, counts, warnings; the app adds its own (other tenant, license limits)

# import, step 2: inside the app's maintenance window (no writes, background jobs paused)
await apply_import(db, SPEC, bundle, preserve=(Preserve("app_users", {"email": me}),), imported_by=me)
bundle.cleanup()
# then: migrations, end all sessions, re-sync caches, show the checklist

await rollback(db, SPEC)        # back to the state before the import (until purge)
await recover(db, SPEC)         # on startup: cleans up an import that stopped half way
await purge(db, SPEC)           # daily: drops the rollback copy after 7 days
```

How the swap works: documents go into `_import_<name>`, then each collection is renamed to
`_preimport_<name>` and the staging copy into its place. A journal (`_backup_journal`) records each step, so an
interrupted import is rolled back on the next start. File folders keep their old content in `.preimport/`.

## Development

`pip install -e ".[dev]"`, then `pytest -q` and `ruff check .`. Engine tests use mongomock; with `MONGO_URL` set
they run against a real MongoDB (CI does both). `tests/vector.scaliobak` is the shared test bundle
(`tests/make_vector.py`); the age layer is checked against rage (`pyrage`) in both directions.
