Metadata-Version: 2.4
Name: postern
Version: 0.4.0
Summary: Run untrusted Python in an OS-isolated sandbox whose only exit is a host-defined hatch.
Author: Centre for Population Genomics
License: MIT
Project-URL: Repository, https://github.com/populationgenomics/postern
Project-URL: Issues, https://github.com/populationgenomics/postern/issues
Keywords: sandbox,bubblewrap,isolation,untrusted-code,seccomp,rpc,agent,tool-use
Classifier: Development Status :: 3 - Alpha
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Interpreters
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: grpc
Requires-Dist: grpcio>=1.60; extra == "grpc"
Dynamic: license-file

# postern

Run untrusted Python in an OS-isolated sandbox whose **only** exit is what the
host opens: a set of typed gRPC methods, or one raw stream per resource.

A postern is the small guarded gate through an otherwise sealed wall. That is the
model: guest code runs with no network, no filesystem beyond a workspace, no
capabilities — and reaches the outside world only through the hatch the host
binds in. The security boundary is what that hatch exposes, not a coarse
permission flag.

```python
from postern import Sandbox, SandboxProfile
from postern.grpc import GrpcHatch
import greeter_pb2_grpc

hatch = GrpcHatch(allowlist={'/greeter.Greeter/SayHello'})
hatch.add_servicer(greeter_pb2_grpc.add_GreeterServicer_to_server, MyGreeter())

profile = SandboxProfile.with_venv('/opt/analysis-env')   # pandas, grpcio, stubs
result = Sandbox(profile, hatch=hatch).run_python(guest_code)
# guest dials unix:$POSTERN_HATCH with the generated stub; a non-allowlisted
# method → PERMISSION_DENIED; there is no network.
```

## Why

Coarse sandbox permissions (`--allow-net`, `--allow-read`) are the wrong grain
for untrusted agent/tool code: you rarely want "the network", you want "this one
method that fetches this one resource". postern inverts the default — the guest
gets **nothing** except the host methods you allowlist, each a typed proto shape.
Whatever a method can reach (a database, a credentialed API, a compute backend)
the guest reaches only through that shape, never directly.

This is the design [enclave](https://github.com/populationgenomics/enclave-py)
prototyped over WebAssembly (WASI-compiled CPython). postern delivers the same
"fine-grained function injection is the boundary" promise over a different
substrate — OS isolation (bubblewrap) plus a hatch over a Unix socket — so the
guest is real CPython with arbitrary third-party packages, no custom toolchain,
and the arguments and results are typed and language-neutral.

## Isolation

`Sandbox` launches the guest under [bubblewrap](https://github.com/containers/bubblewrap):

- **empty network namespace** — no egress of any kind (a socket can be created
  but has no route). The user and cgroup namespaces are unshared **strictly**
  (`--unshare-user`/`--unshare-cgroup`, not `--unshare-all`'s best-effort `-try`
  variants), so a host that can't provide a user namespace is a hard launch
  failure rather than a silent fall-through to a real-root guest;
- **surgical filesystem** — read-only base system dirs + one writable
  `/workspace`; no `/etc`, `/home`, `/root`, or host application code. bwrap's
  fresh `--proc` re-exposes the procfs sysctl surface writable and discards the
  runtime's mask, so `/proc/sys` (and `/proc/sysrq-trigger`, `/proc/irq`,
  `/proc/kcore`, …) are re-masked read-only — without it a guest whose mapped
  kernel uid is root can write `core_pattern`/`modprobe` and gain init-namespace
  root (a full host escape);
- **`--cap-drop ALL`**, **`--new-session`** (anti terminal-injection),
  **`--die-with-parent`**, **`--clearenv`**;
- **`--as-pid-1`** — the guest entrypoint *is* PID 1 of the guest's PID
  namespace. A resident bwrap there would share the guest uid, putting its
  `/proc/1` (cmdline, maps, read/write mem, env) within reach from inside. Who
  plays init in its place depends on the entrypoint: see
  [PID 1 and the resource backstops](#pid-1-and-the-resource-backstops) below;
- **non-root guest** — the guest runs as uid/gid `65534` (`nobody`), so it holds
  no capabilities inside its user namespace, and if the userns fails to
  materialise on a root host it still drops to a non-root real uid
  (`SandboxProfile(guest_uid=None)` restores the legacy uid-0-in-userns). bwrap
  maps `--uid` to its *own* real uid, so a root bwrap still gives the guest
  kernel uid 0; `SandboxProfile(host_uid=…)` opts into running bwrap itself
  non-root (defense in depth beyond the `/proc/sys` mask) — off by default
  because the deploy must then make every bind source reachable by that uid;
- a **seccomp denylist** blocking escape-enabling syscalls (`unshare`, `setns`,
  `mount`, `ptrace`, `bpf`, `keyctl`, `io_uring_setup`, …). `socket` is
  deliberately *not* blocked — network isolation is the netns's job, and the guest
  needs `socket(AF_UNIX)` for the hatch.

The hatch UDS is bind-mounted in as the single controlled opening. Because the
RPC rides that socket, the guest's own stdin/stdout/stderr stay free.

### PID 1 and the resource backstops

Everything above is a bwrap flag, so every entrypoint gets it. These two are not,
and they follow the entrypoint instead:

- **`Sandbox.run_python`** binds in a shim (`_guest.py`) and makes *that* the
  entrypoint. As PID 1 it is a real init: it forks the guest, reaps orphaned
  descendants that reparent to it, propagates the guest's exit status, and marks
  *itself* `PR_SET_DUMPABLE=0` so a co-uid process the guest spawns cannot read
  the init. Before `exec`ing the guest code it applies **`RLIMIT_NPROC`**
  (`SandboxProfile(rlimit_nproc=1024)`) as a fork-bomb backstop and, when set,
  **`RLIMIT_AS`** (`SandboxProfile(rlimit_as=...)`, off by default; a cgroup
  `memory.max` at the deploy layer is the real memory isolation).
- **`Sandbox.run`** runs the caller's argv directly, with no shim. That argv is
  PID 1 and must tolerate being it — nothing reaps orphans for it, nothing hides
  its `/proc/1` from a same-uid child it spawns — and **neither rlimit is set**;
  the entrypoint manages its own. `rlimit_nproc`/`rlimit_as` on the profile are
  inert on this path. The stream hatch's `git_url` entrypoint is an example: git
  is the PID 1 there.

**Fail-closed boot check.** Every control is enforced on the launch path: the
strict `--unshare-*` flags make bwrap abort if it can't create the namespaces,
apply `--uid`, or drop capabilities, and the seccomp loader refuses an uncovered
architecture — so a successful launch *is* the proof, and there is no runtime
probe. `Sandbox(profile).verify()` triggers one trivial launch at startup so a
broken platform (no user namespace, gVisor, uncovered arch) raises
`IsolationError` there rather than on the first request. Call it at worker startup
and refuse to serve if it raises, as `examples/worker.py` does.

## The environment (getting pandas etc. in)

The sandbox has no egress, so packages are provisioned **ahead of time** and
mounted read-only — never `pip install`ed at run time.

- `SandboxProfile.with_venv('/opt/env')` binds a venv read-only (at its own path,
  so its `site.py` resolution works) and runs its interpreter. The venv holds the
  guest's libraries **and** its hatch client (grpcio + the generated stubs).
- `SandboxProfile(rootfs='/opt/guest-root')` binds a curated base directory as the
  guest's system dirs *instead of the host's* — hiding the host userland
  entirely. Build it at image-build time (build-time Docker is fine; only
  *runtime* container engines are excluded): `docker export` a container into a
  dir, or ship a single squashfs/erofs image file mounted read-only via FUSE
  (`squashfuse`, unprivileged, Cloud-Run-compatible) and point `rootfs` at the
  mountpoint. `bwrap --ro-overlay` can stack OCI layer dirs without flattening.

## Requirements

Linux with **bubblewrap** and unprivileged user namespaces (a Cloud Run gen2 Job,
or any such host). `postern.available()` reports whether a sandbox can launch.
The seccomp filter is a prebuilt multi-arch BPF blob (x86_64, x86, x32, aarch64,
arm); on any other architecture it would be a default-allow no-op, so `Sandbox`
**refuses to launch** there (fail-closed) rather than run with an unenforced
filter — set `SandboxProfile(seccomp=False)` to override deliberately. Not
runnable on macOS except against a Linux target — `import postern` works
anywhere, `Sandbox.run*` needs the OS.

**Ubuntu 23.10+ / 24.04** restrict unprivileged user namespaces by default
(`kernel.apparmor_restrict_unprivileged_userns=1`), which bubblewrap needs —
the symptom is `bwrap: setting up uid map: Permission denied` or `loopback:
Failed RTM_NEWADDR`. Lift it with `sudo sysctl -w
kernel.apparmor_restrict_unprivileged_userns=0`, or install an AppArmor profile
that grants bwrap `userns`. Cloud Run gen2 does not have this restriction.

The bare `Sandbox` has **no third-party dependencies and no cloud dependency** —
it is a Linux primitive. The gRPC hatch pulls `grpcio` via the `grpc` extra; the
stream hatch (below) is stdlib-only.

## Stream hatch

Some protocols are neither typed RPC nor request/response. `StreamHatch` gives
the guest **one socket** and nothing else; per accepted connection a handler
decides what its bytes are spliced to — a host-side subprocess's stdio, or
nothing. The motivating case is git, whose native wire protocol is pkt-line over
a raw bidirectional stream and whose `ext::` transport carries that over a
command's stdin/stdout.

```python
from postern import Sandbox, SandboxProfile
from postern.stream import StreamHatch, git_url, splice_subprocess

hatch = StreamHatch(splice_subprocess(['git', 'upload-pack', '/srv/repo.git']), name='repo')
profile = SandboxProfile()
sandbox = Sandbox(profile, hatch=hatch)
sandbox.run(['git', '-c', 'protocol.ext.allow=always', 'clone',
             git_url('repo', profile=profile), 'work'])
```

Pass `profile=` to `git_url`: the in-guest interpreter comes from
`profile.python`, the same place `run_python` gets it, so the URL cannot disagree
with the sandbox it runs in. Without it the default is a bare `python3` off the
guest `PATH`, which is wrong for `with_venv` or a curated `rootfs`.

**The socket is the capability.** The same access could be brokered through an
HTTP forward proxy with a handler policing each request; a bound stream socket
differs in four ways.

- *Capability by descriptor, not policy by parser.* Through a proxy the guest
  names a URL, so "only this one repository" means validating request targets in
  a handler — a parser in the policy path, fed attacker-controlled input. One
  socket per resource makes the wrong resource unrepresentable. The service is
  fixed too: a hatch bound to `git upload-pack` cannot be talked into
  `receive-pack`.
- *No body buffering, and no copying.* A proxy that inspects request bodies has to
  buffer them, and so has to cap them. Here the socket **is** the command's stdin
  and stdout, so the kernel moves every byte and postern is not on the data path.
  The kernel also propagates the command's disposition: a command that consumed
  its input and exited gives the guest end-of-stream, one that died mid-request
  gives it a reset — the only failure signal a stream with no framing of its own
  has.
- *No protocol translation.* The host side runs a subprocess and splices its
  stdio, rather than decoding framing and re-emitting headers to reach the same
  subprocess.
- *It works with `Sandbox.run`.* A hatch needs nothing in-guest, so a bare `git`
  entrypoint reaches it, not only a `run_python` guest. Both entrypoints bind and
  serve every configured hatch.

Stream hatches are **named**, so a sandbox carries as many as it has resources:
each binds at `/run/postern/<name>.sock` and is exported as
`$POSTERN_HATCH_<NAME>`, while the unnamed `GrpcHatch` keeps `$POSTERN_HATCH`.
`hatch=` takes one hatch or a sequence. The in-guest connector that bridges a
command's stdio to the socket is bound in at `$POSTERN_CONNECT` — stdlib-only,
one blocking thread per direction, leaving with `os._exit` rather than finalising,
because finalising the interpreter around a reader still parked on a descriptor
git has torn down aborts the connector (`python3 died of signal 6`). git gates `ext::` behind `protocol.ext.allow` because an `ext::` URL is
command execution; inside the sandbox that gate protects nothing, since the guest
is already running untrusted code, so enable it per invocation with `-c` and leave
the host's git config alone.

Guest bytes reach a host-side process only as its **stdin** — never its argv,
env, cwd, or a dial's destination, all fixed when the hatch is constructed.
`splice_subprocess` gives the process a fixed minimal `PATH` and a fixed `cwd`
rather than the worker's, so ambient host state cannot decide what the capability
is, and discards its stderr, because a command's diagnostics quote host paths.
`max_conns` gates *accepting* rather than dispatch: a stream connection is
long-lived, so a queue of accepted-but-unserved connections would be a queue of
host file descriptors. Past the cap, dials wait in the kernel backlog.

**Teardown has two known holes.** Per connection the hatch waits for the command
and then signals its whole process *group*, because a child the command left
behind inherits the guest's socket and would otherwise hold the connection open
for ever. Two things bound that:

- *Another reaper in your process voids it.* If the embedding process reaps
  arbitrary children — a supervisor loop calling `waitpid(-1)`, `multiprocessing`,
  an asyncio child watcher, `SIGCHLD` set to `SIG_IGN` — then `Popen` synthesises
  an exit status of `0` on `ECHILD`, which is indistinguishable from a clean exit,
  and the group signal is skipped. Declining is the answer to that ambiguity: the
  foreign reap freed the pid, so signalling anyway could land on a recycled one.
  What leaks is the command's *children*; the worker thread and the slot come
  back. If your host process has its own reaper, keep the outer
  `Sandbox.run(timeout=...)` short.
- *A grandchild that calls `setsid()` escapes it*, because it is no longer in the
  group. A shell's `&` child stays and is collected; a daemonising sidecar does
  not. Prefer a command that does not daemonise.

`postern.stream`'s module docstring covers the mechanism under "Teardown and its
caveats", including which platforms can observe a command's exit without reaping
it (`waitid` on Linux, and on darwin from CPython 3.13; `kqueue` on macOS and the
BSDs) and what a platform with neither loses.

**The command's stdin grammar is part of the capability.** A fixed argv means
guest bytes never become *that* process's argv. It does not stop them becoming a
*downstream* process's argv or a shell command, and `splice_subprocess` cannot
check a grammar for you. `git upload-pack` grants nothing beyond the repository.
`sqlite3` — even `-readonly` — has `.shell`, so splicing it is host command
execution; so are `psql` (`\!`, `COPY … FROM PROGRAM`), `mysql` (`system`),
`redis-cli`, `ftp`, `gdb` and `ed`.

`stderr=DEVNULL` covers fd 2 and nothing else: a command that multiplexes
diagnostics onto *stdout* routes around it (`git upload-archive` reports
`fatal: '<path>' does not appear to be a git repository` on its pkt-line
sideband), so pass `cwd` and a bare basename rather than an absolute host path.

## Install

```bash
pip install postern              # the bare sandbox + the stream hatch (no deps)
pip install 'postern[grpc]'      # + the gRPC hatch
```

## Public API

- `Sandbox(profile=None, *, hatch=None)` — `.run(argv)`, `.run_python(code)` → `ProcResult(returncode, stdout, stderr, ok)`; `.verify()` (fail-closed boot check, raises `IsolationError`). Both entrypoints bind and serve every configured hatch and get the identical bwrap profile; they differ in PID 1 and the rlimits ([above](#pid-1-and-the-resource-backstops)). `hatch` takes one hatch or a sequence, with at most one *unnamed* hatch since that one owns a fixed guest env var.
- `SandboxProfile(workspace=None, rootfs=None, python='python3', ro_binds=[], stubs=None, env=..., seccomp=True, rlimit_nproc=1024, rlimit_as=None, guest_uid=65534, guest_gid=65534, host_uid=None, host_gid=None)` and `SandboxProfile.with_venv(venv, **kw)`. `host_uid=` runs bwrap itself at a non-root real uid; the deploy must then make every bind source reachable by it. `stubs=` injects a dir or list of files at `/run/postern/stubs`, prepended to `PYTHONPATH`. `rlimit_nproc=`/`rlimit_as=` are applied by `run_python`'s shim, so they are inert under `run()`.
- `postern.grpc.GrpcHatch(allowlist, *, socket_path=None)` — `.add_servicer(register_fn, servicer)`; `with hatch.accepting(): ...`. (`grpc` extra.)
- `postern.stream.StreamHatch(handler, *, name='stream', socket_path=None, max_conns=8, backlog=64, grace=5.0)` — a raw bidirectional byte stream over the sandbox UDS, reached as a plain file at `$POSTERN_HATCH_<NAME>`, so `run()` works and not only `run_python()`. Named, so several coexist: one socket per resource. Stdlib-only. `with hatch.accepting(): ...`, and `close()` is terminal as `GrpcHatch`'s is.
  - `handler(stream) -> Process | None`: return `Process(argv, cwd=None, env=None, stderr=DEVNULL)` to hand the connection to a subprocess as its stdio, or `None` to refuse. The hatch spawns it, so the descriptor is in ordinary-stdio shape (blocking, no signal-driven I/O, no socket timeouts) before there is a child to race.
  - `Process.from_popen(popen)` adopts a subprocess you spawned yourself, for what the declarative form does not cover (`pass_fds`, `user=`, an rlimit). An already-spawned verdict can only be validated, and it has none of the declarative path's `stderr`/`env`/`cwd` defaults.
  - `stderr=PIPE` is refused because nothing drains it and the command would deadlock; `stderr=STDOUT` is refused because stdout is the guest socket, so it would relay host-path diagnostics to the guest. Checked at construction for `Process(argv)`, and by detection for an adopted `Popen` where the platform allows, since `subprocess` keeps no record of the `stderr` it was passed.
  - `Stream.read_preamble(max_bytes, timeout)` bounds a preamble read without touching socket flags. Do not use `settimeout` for that: the command shares the descriptor.
  - `splice_subprocess(argv, *, cwd=None, env=None, stderr=DEVNULL)` is the stock handler; `git_url(name, *, profile=None, python=None)` builds the `ext::` URL for the in-guest connector at `$POSTERN_CONNECT`.
- `Sandbox.accessor()` / `postern.Workspace(dir)` — a reference-closed handle to a workspace; `WorkspacePath` is its `pathlib`-like facade. `.pack_tar(f, *, exclude=…)` and `.restore_tar(f, *, max_entries=…, max_bytes=…)` → `WorkspaceReport`; `ws / 'a/b'`, `.iterdir()`, `.walk()`, `.open()`, `.read_bytes()`. `reference_closed_filter` plugs into `tarfile.extractall(filter=...)`, member-vetting only — see below.
- `available()` — whether bubblewrap is on the PATH.

## Reading the workspace safely

The workspace is the one writable surface the guest and host share, and it
outlives the sandbox. Nothing stops the guest planting a reference that points
*outside* it — `ln -s /proc/self/environ doc`, `ln -s / root`, a FIFO. Inside the
jail these are inert; the danger is when the **host** later reads, tars, or
restores the tree in its own namespace and privileges and becomes a confused
deputy (exfiltrating its own secrets, or writing through the link to a host path).

postern guarantees the workspace is **reference-closed**: read, pack or restore it
through the host-side accessor and no guest-planted symlink, `..`, or special file
is ever followed out of the tree.

```python
with sandbox.accessor() as ws:                 # or Workspace(some_dir)
    # only regular files + dirs; symlinks/FIFOs and escaping hardlinks are
    # neutralized, and exclude drops paths a checkpoint persists elsewhere
    report = ws.pack_tar(open('snap.tar', 'wb'), exclude=lambda p: p == 'document.md')
    # report.skipped is the audit trail of what was neutralized (never silent)
    ws.restore_tar(open('snap.tar', 'rb'), max_entries=20_000, max_bytes=512 << 20)
    data = (ws / 'out' / 'result.json').read_bytes()
```

Every path resolves one component at a time with `O_NOFOLLOW` relative to a
directory fd (the model is Go's `os.Root` / Rust's `cap-std::Dir`); the accessor
never hands back a dereferenceable host path. Pure stdlib, no mount privilege, so
it runs on an unprivileged host such as a Cloud Run container. A sticky
world-writable workspace (`0o1777`) additionally stops the guest unlinking
host-written files to swap in escaping symlinks. `restore_tar`'s
`max_entries`/`max_bytes` bound a decompression bomb from an untrusted store.

`reference_closed_filter` plugs into stock `TarFile.extractall(filter=...)` for
consumers that keep `tarfile`, but it only **vets members** — stock extraction
still follows a symlink that already exists in the destination, so it is safe
only into a fresh host-controlled directory. To extract into a workspace a guest
may have touched, use `restore_tar`: it writes *through the confined root*, never
through an in-tree symlink, and reports what it neutralized.

`examples/e2e_greeter.py` is a runnable end-to-end example: a typed hatch call
plus pandas, on a Linux host.

## Deploy: bundle the rootfs into the Job image

For a Cloud Run Job you build an image anyway, so bundle the guest rootfs into it
and let postern bind it, with no runtime container engine. `examples/Dockerfile`
is the recipe: a three-stage build that generates the stubs, builds a minimal
guest rootfs (`python:3.12-slim` + grpcio + your data libs + the client stubs),
and assembles the worker (bubblewrap + `postern[grpc]` + your servicer) with the
guest rootfs copied to `/opt/guest-root`. `examples/worker.py` binds it with
`SandboxProfile(rootfs='/opt/guest-root')`, so the guest sees only that curated
image. Cloud Run gen2 provides the unprivileged user namespaces bubblewrap needs.

## Roadmap

Not implemented:

- **Checkpoint/restore** — a `Store` protocol and durable-glob workspace
  snapshots, sitting on the reference-closed `Workspace` accessor
  (`pack_tar`/`restore_tar`).
- **`overlay=` profile mode** — emit `bwrap --ro-overlay` to stack layers with a
  tmpfs upper, instead of a single `--ro-bind` rootfs.
- **Agent-runtime adapters** — drive the same sandbox from Anthropic Managed
  Agents, Google ADK, or MCP.

## License

MIT.
