Metadata-Version: 2.4
Name: adx-environment
Version: 0.1.0rc1
Summary: Python SDK for Agent DX environments with EXECD HTTP and reverse-tunnel transport
License-Expression: Apache-2.0
Project-URL: Repository, https://gitcode.com/openJiuwen/agent-dx
Project-URL: Documentation, https://gitcode.com/openJiuwen/agent-dx/blob/refactor/platform/sdk/environment/python/README.md
Classifier: Development Status :: 3 - Alpha
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.24
Requires-Dist: idna<4,>=3
Requires-Dist: websockets>=13.0
Dynamic: license-file

# Agent DX Environment Python SDK

`adx-environment` is the high-level Python client for the frontend
**environment v1** API. It creates and manages environments, including reusable
Snapshots and a paused-environment lifecycle. It also provides the gateway routes
used for reverse tunnels.

```python
from adx_environment import Environment

with Environment(image="python:3.12-slim", cpu=2000, memory=4096) as environment:
    environment.files.write("/tmp/hello.txt", "hello world")
    print(environment.commands.run("cat /tmp/hello.txt").stdout)
```

`close()` releases local SDK resources and leaves the remote environment alive.
`kill()` deletes a non-detached remote environment as well.
The context manager explicitly calls `kill()` on exit; garbage collection does
not make a remote delete request. If the block raises, its original exception
remains visible even when cleanup also fails.

## Server compatibility

The new Environment backend supports basic lifecycle, on-demand pause/resume, reusable snapshots, grouped placement, idle deletion/pause and restart policy. It requires an image containing the release EXECD at the configured command path; generic `python:3.12-slim` below is only an illustrative image name.

The Environment backend supports S3 rootfs, S3/image mounts, entrypoint inheritance,
creation and runtime network policy, `extra_config`, `failover=True`, independent
resource limits, and per-environment data-plane security. Declared user ports use
Ingress and Relay routing. Public local rootfs paths and host mounts are not a
tenant-facing contract. `upstream` reverse tunnel uses the published
`/tunnel/{environment}` route; the legacy `/invoke` fallback remains unavailable. See the
[current HTTP contract](https://gitcode.com/openJiuwen/agent-dx/blob/refactor/gateway/apiserver/docs/environment-lifecycle-api.md).

## OOM diagnostics

Command results and background-command info preserve `exit_signal`, such as
SIGKILL=9. EXECD does not observe OOM. A user process can be killed while the
Environment stays Running; a signal alone is not an OOM classification. The current
sandboxd API does not expose a separate user-process OOM reason.

If the execution backend itself is OOM-killed, `environment.get_info().failure`
contains an `EnvironmentFailure(reason="OomKilled", exit_code=..., message=...)`
after the terminal state is committed. A failed direct invocation can raise
`EnvironmentError` with `code="ENVIRONMENT_OOM"`, `retry="never"`, `outcome="terminal"`
and `failure` details. Before publication, the caller may receive
`OUTCOME_UNKNOWN`; query `get_info()` instead of interpreting route errors as
OOM. Successful calls do not perform an additional catalog query.

## Working directory

`Environment(cwd="/workspace/task")` sends the constructor directory to the server.
Execd creates the directory and any missing parents before reporting Ready;
the SDK does not issue a separate filesystem initialization call. An existing
directory and its contents are preserved. Initialization failure makes creation
fail through the normal lifecycle cleanup path.

Commands without an explicit `cwd` use the constructor directory. A per-command
`commands.run(..., cwd="/other")` override must already exist; create it explicitly
with `files.make_dir()` when needed. A failed command does not stop Execd. The
constructor setting does not change Execd's own working directory or an inherited
OCI entrypoint's `WORKDIR`.

## Image startup process

Set `inherit_entrypoint=True` on a fresh image-backed environment to start the
image's effective `Entrypoint` and `Cmd` as its managed workload:

```python
environment = Environment(image="example/worker:latest", inherit_entrypoint=True)
exit_code = environment.wait_entrypoint()
print(exit_code, environment.entrypoint_exit_info)
```

Creation fails when the image has no effective startup command or cannot start
it. A managed process may exit successfully; its terminal status remains
available through bounded polling while the environment and its other APIs remain
usable. Configure workload probes when creation must wait for a service.

## Workload probes

```python
from adx_environment import Environment, ReadinessProbe, HealthcheckProbe, EnvironmentError

try:
    environment = Environment(
        image="example/server:latest",
        inherit_entrypoint=True,
        init_timeout=120,
        readiness_probe=ReadinessProbe.http_get(port=8080, path="/ready"),
        healthcheck_probe=HealthcheckProbe.http_get(
            port=8080, path="/health", failure_threshold=3,
        ),
    )
except EnvironmentError as error:
    print(error.probe_failure)
    raise
```

Both probe types provide `http_get(port=..., path="/")`,
`tcp_socket(port=...)`, and `exec(command=[...])` factories. HTTP and TCP check
environment loopback; the port need not be publicly exposed. HTTP requires a 2xx
response and does not follow redirects. Exec uses argv, the image workload's
user, working directory and effective environment; exit code zero succeeds.
Use an explicit shell argv when shell syntax is needed.

| Setting | ReadinessProbe | HealthcheckProbe |
| --- | --- | --- |
| `initial_delay_seconds` | 0 | 0, measured after readiness |
| `period_seconds` | 2 | 10 |
| `timeout_seconds` | 1 | 2 |
| `failure_threshold` | retries until initialization deadline | 3 consecutive failures |

Execd runs readiness once per startup/resume/restore and only then reports
Running. Adxlet activates routes and publishes Running after that report.
`init_timeout` bounds runtime startup plus readiness; by default the SDK's total
create budget is scheduling plus initialization plus a 30-second buffer.
Readiness failure raises `EnvironmentError` with `probe_failure` and is terminal
for that create attempt. Initialization is transient runtime state, not a
separate persisted Environment lifecycle state.

Healthcheck starts after readiness (or immediately after runtime startup when
readiness is absent). A success resets the failure count. At the threshold,
Execd reports a sticky failure; Adxlet retires the execution and applies the
existing restart/failover policy. `environment.get_info().failure` is a
`EnvironmentFailure(reason="ProbeFailed", ...)`; its `probe_failure` dictionary
contains the probe kind, message, consecutive failures and available exit/HTTP
status. Backend OOM and exit failures retain their existing reason values.
Internal probes do not extend the environment's user-activity idle window.
Pause/resume and restore cancel in-flight checks and reset healthcheck counts.
The SDK checks server capability before a probe-dependent create; unsupported
API servers return an error before submission. Use the matching release of
API Server, Adxlet and Execd.

## Lifecycle overview

There are three deliberately different checkpoint paths:

| Path | Public SDK API | Artifact and placement | When to use it |
| --- | --- | --- | --- |
| Reusable Snapshot | `create_snapshot()` then `Environment.create()` | New Environment identity; shared storage allows fresh placement, local-only pins the source node. | Independent clones from a prepared source. |
| Pause / resume | `pause()` then `resume()` | Same Environment ID; Coordinator first tries the owner, then schedules a compatible node after definitive resource rejection or owner loss. | Stop and resume one logical environment. |
| Failure recovery | `failover=True` | Same Environment and node; restores the latest unexpired checkpoint after unexpected backend exit. | Workloads that must recover execution state rather than cold-start. |
| Explicit reload | `reload()` | Same Environment; replaces a Running backend from its latest unexpired checkpoint. | Operator-requested reset to a known recovery point. |

The SDK is a client-side validation, request-ID, attempt, and result-shaping
layer. Adxlet owns lifecycle and checkpoint bytes; Coordinator owns placement, committed state and the snapshot catalog.

## Reusable Snapshots

Create a reusable Snapshot from an open environment:

```python
from adx_environment import Environment

source = Environment(image="python:3.12-slim", name="source")
snapshot = source.create_snapshot(name="python-ready")

# The source stays running. `snapshot` is a SnapshotInfo value.
clone = Environment.create(snapshot, name="worker-1")

clone.kill()
source.kill()
```

The public signature is:

```python
Environment.create_snapshot(*, name=None, timeout_seconds=300) -> SnapshotInfo
```

`name` is optional, but if supplied it must be a nonblank string.
`timeout_seconds` must be an integral, non-boolean value from 1 through 3600.
The result is a frozen `SnapshotInfo(snapshot_id: str, names: tuple[str, ...])`.
The Snapshot has no TTL: delete it explicitly when it is no longer needed.

```python
same_snapshot = Environment.get_snapshot(snapshot.snapshot_id)
snapshots, next_page_token = Environment.list_snapshots(
    name="python-ready", page_size=20,
)
Environment.delete_snapshot(same_snapshot.snapshot_id)
```

The source must have a valid lifecycle identity. In particular, an Environment with
an active reverse tunnel cannot create a reusable Snapshot. A successful
Snapshot leaves the source running and publishes an immutable artifact; the
server records READY metadata only after that artifact is available.
The SDK does not expose the reusable-Snapshot request ID and generates a fresh
identity for each call. Raw HTTP clients that retry an uncertain request must
reuse it only for the same source and name; they must not reuse one request ID
for different catalog content.
The SDK keeps its local tunnel client active while the checkpoint request is
in flight; expected reconnect noise during that scope is logged at debug level
without changing the checkpoint result.

### Create from Snapshot

Use `Environment.create(snapshot_id, **kwargs)` with either a `SnapshotInfo` or a
Snapshot ID string. It performs the normal environment create request with a
`snapshotId`, so the result is a new `Environment` handle that reaches RUNNING in
the usual way.

```python
clone = Environment.create(
    snapshot,
    name="worker-2",
    # Omitted resources inherit the snapshot; explicit values must match it.
)
```

The new Coordinator inherits omitted image/runtime/scalar resources and validates explicit values against the source geometry. It does not resize a restored VM. Environment overrides and placement constraints are carried into the new Environment; local-only snapshots require the source node. Shared snapshots use normal scheduling. The source briefly pauses during snapshot creation and resumes before success. A reusable snapshot is not consumed by cloning; deletion blocks new references and waits for existing references to be released. See [storage and cloning](https://gitcode.com/openJiuwen/agent-dx/blob/refactor/docs/testing/snapshot-storage.md).

Later dual-clone FC runs exposed a network failure, tracked in the [investigation](https://gitcode.com/openJiuwen/agent-dx/blob/refactor/docs/testing/2026-09-16-fc-clone-network.md); successful earlier batches do not close that issue.

## Pause and resume

Pause one open environment and later resume the same logical environment:

```python
from adx_environment import RetentionPolicy, Environment

environment = Environment(image="python:3.12-slim")

paused = environment.pause(retention=RetentionPolicy.for_seconds(90_000), timeout_seconds=300)
print(paused.snapshot_id, paused.expires_at)

running = environment.resume()
print(running.route_address, running.port_mappings)
```

The public signatures are:

```python
Environment.pause(retention=None, *, timeout_seconds=300) -> PauseResult
Environment.resume() -> ResumeResult
```

`retention=None` inherits `pause_policy.retention`; the default policy retains the
restore point for 90,000 seconds. Use `RetentionPolicy.for_seconds(seconds)` for
a positive integer TTL or `RetentionPolicy.forever()` to retain the point until
explicit deletion or replacement. Finite retention begins after publication.
Permanent pause returns `expires_at=None`; the stored record and RPC use an absent
expiry, rather than a large timestamp. `PauseResult` also contains `environment_id`,
`snapshot_id`, byte `size`, and `state="paused"`.

`timeout_seconds` is keyword-only, integral and non-boolean, from 1 through 3600.
It bounds the sandboxd capture RPC and sets the SDK's per-attempt HTTP wait with
an extra 30-second transport allowance. It is not a total pause deadline: OBS
publication follows capture and uses its own storage request timeout. Losing the
HTTP reply does not cancel an accepted lifecycle operation; its outcome can remain
unknown and the SDK retries the same operation identity.

`resume()` returns a frozen `ResumeResult` with `environment_id`, `state`,
`route_address`, `function_proxy_id`, `node_id`, and `port_mappings`. The SDK
requires the response to identify this environment, report `"running"`, and
include a route address and function-proxy ID. Resume uses Coordinator EnsureRunning, first admitting on the owning node. A shared checkpoint allows another compatible node after definitive NoCapacity or owner invalidation; an uncertain RPC result retains ownership and retries there. The selected node restores the checkpoint, rearms the runtime listener and commits Running. Route-cache convergence after
that result is outside the resume success boundary.

Pausing validates the configured store's artifact and commits Paused after removing the old execution. Local-only files are immediately readable on the owning node; adxlet does not force filesystem writeback, so publication does not guarantee survival of a host crash or power loss. S3 publication waits for upload completion. A SQLite-only result is not cluster success. Public resume uses authoritative ownership and a valid checkpoint. Local-only points remain tied to the original node; shared points preserve the Environment ID across a new ownership generation. Node loss preserves Paused until demand, expiry or deletion.

### Idle pause and traffic resume

```python
from adx_environment import CheckpointStorage, IdleAction, PausePolicy, RetentionPolicy, Environment

environment = Environment(
    image="example/execd:latest",
    idle_timeout=60,
    idle_action=IdleAction.PAUSE,
    pause_policy=PausePolicy(retention=RetentionPolicy.forever(), timeout_seconds=300, storage=CheckpointStorage.SHARED),
)
```

`idle_action` is an `IdleAction` enum and defaults to `IdleAction.DELETE`; `IdleAction.PAUSE` requires a positive idle timeout
and a checkpoint-capable runtime. `CheckpointStorage.CONFIGURED` uses the node backend,
`CheckpointStorage.LOCAL` retains files on that node, and `CheckpointStorage.SHARED` requires complete publication
to its configured object store. Configure an OBS S3-compatible endpoint on every
eligible node, with the same alias/bucket/prefix. The shared setting also limits
creation placement to nodes advertising shared checkpoint storage.

An automatically paused environment also supports `environment.resume()` explicitly.
It uses the same restore intent as traffic-triggered wake-up, so concurrent
manual and automatic requests do not start two runtimes. The checkpoint must
still be valid and unexpired; local-only restore requires the owning node.
The idle policy stays enabled after resume.

The existing idle window observes Execd client operations and Relay streams;
health, metrics and command observation do not keep the environment busy. A detached
background command can be checkpointed. Automatic pause atomically closes new
admission and rechecks activity. New demand before capture cancels pause; demand
once capture starts waits for publication and restores afterward.

Authorized business traffic through Ingress shares the manual resume entry for
both automatic and manual pauses; no auto-resume opt-in is required. Concurrent first requests coalesce into one durable intent.
Ingress waits for the committed Running route before sending application bytes;
passive route subscriptions and health/metrics do not wake a paused environment.
Requests already sent upstream are not replayed. Transient owner RPC failures
retry on the same owner within the resume budget. A temporary wake failure before
forwarding returns HTTP 503. A finite SDK HTTP request sends
`X-ADX-Processing-Timeout-Ms`: authentication, route lookup, wake, route delivery,
connection admission, forwarding and response streaming share one Ingress deadline.
The SDK defaults to 300 seconds for a data request and adds a configurable
communication margin (30 seconds by default) to its HTTP timeout. Without the
header, wake and route delivery share a 300-second default; existing forwarding
and streaming lifetimes remain unchanged. Negative invoke timeouts omit the
header and retain unbounded application HTTP waits.

If readiness is still pending when the deadline expires before application
forwarding, Ingress returns HTTP 503 with `X-ADX-Error-Code: ENV_RESUMING`,
`Retry-After: 1`, and a JSON `error` containing `code: ENV_RESUMING`,
`retry: same_operation`, `outcome: not_started`, `requestForwarded: false`,
`requestId`, and `environmentId`. The durable restore continues independently.
SDK retries retain the request ID and body; **each allowed retry gets the same
configured timeout plus communication margin**. Retry count is configurable via
`ConnectionConfig.request_max_attempts` / `ADX_REQUEST_MAX_ATTEMPTS` (default 3,
including the first data-path attempt). Known-unsent connection failures and absent
direct routes retain their existing single control-plane fallback; `ENV_RESUMING`
never takes that fallback. Retry-After and jitter apply between demand retries.
Exhaustion raises `EnvironmentHTTPError` with the same `ENV_RESUMING` contract; callers
can choose to try again. No sticky Ingress routing is required: replicas share the
Coordinator intent and independently subscribe to the committed route.

HTTP 504 after forwarding carries `PROCESSING_DEADLINE_EXCEEDED`, `outcome: unknown`,
`requestForwarded: true`, and `retry: never`; it is not a safe demand retry. If
response headers have already been delivered, deadline expiry ends the body stream
instead. Existing idempotent invoke transport retries keep the same EXECD operation
ID and never cross into a control-plane fallback after an unknown outcome.
File bytes and seekable file bodies support safe demand retry. Consumed iterators
are not replayed automatically and surface the structured error instead.
Expired or unavailable checkpoints return HTTP 409 `ENV_UNAVAILABLE`, `retry: never`.

A timeout may leave the durable operation pending. If every node rejects admission,
rejected nodes are excluded for a 30-second capacity round before reevaluation.

## Recovery methods

`failover=True` restores from the latest unexpired checkpoint after an
unexpected backend exit on the owning node. If no valid checkpoint exists, the
Environment becomes Failed and is not recreated from the original image.
`Environment.reload() -> bool` explicitly replaces a Running backend from that same
recovery point. It returns `True` only after the replacement reaches Running and
the authoritative result is committed. Automatic restart policy is a separate cold
restart mechanism. Shared-checkpoint node-failure recovery is coordinated by
Coordinator and is not selected by either public call.

## Timeouts, attempts, and errors

For `create_snapshot()` and `pause()`, `timeout_seconds` defaults to the
public `ADX_GET_DEFAULT_TIMEOUT` value of 300 seconds. It is the logical server
checkpoint timeout sent as `timeoutSeconds`; each SDK HTTP attempt uses that
logical value plus a 30-second transport buffer. `resume()` and `reload()`
have no public logical-timeout body field and use the SDK default plus that
buffer for each transport attempt. SDK argument errors are raised
before any request; closed `create_snapshot`, `pause`, and `resume` handles
raise `RuntimeError`.

The SDK generates request IDs; callers of `Environment` cannot set them. Its
attempt rules intentionally differ by operation:

- Reusable Snapshot creation makes one SDK transport attempt only. A connection problem or
  gateway 502/503/504 produces `EnvironmentError` with an **uncertain** outcome,
  because the immutable Snapshot might already have committed. Reconcile it
  explicitly instead of assuming that another attempt is safe.
- Pause, resume, and reload make up to three transport attempts for transient transport
  or gateway failures, all with one internal request ID. If the final result is
  still uncertain, reconcile the instance state instead of changing the request
  under that identity.
- Create from Snapshot uses the normal create policy with up to three attempts
  and one `create-*` identity. When the caller omits `name`, the SDK derives a
  stable name from that identity, so another API Server replica receives the
  same Environment ID. An uncertain result must still be queried or retried with
  the same identity.
- `Environment.delete()` treats HTTP 404 as an idempotent success. HTTP 403 raises
  `PermissionDenied` with the request ID and is not retried; transient transport
  or gateway failures retain one request ID across bounded retries.

Structured server failures raise `EnvironmentHTTPError`, whose `code`, `retry`,
`outcome`, `request_id`, `operation_id`, and `environment_id` fields implement the
[management error contract](https://gitcode.com/openJiuwen/agent-dx/blob/refactor/platform/api/http/error-contract.md). Transport
failures with an uncertain write result surface the same fields on
`EnvironmentError`. Other malformed typed-result
shapes can instead surface `ValueError` or `TypeError` while values are
converted, or `RuntimeError` when resume `portMappings` is not an object.
`reload()` translates `EnvironmentError` into `False`; it does not normalize those
other programming/shape exceptions.

### SDK versus raw HTTP

These are SDK semantics, not a substitute for the [frontend REST contract](https://gitcode.com/openJiuwen/agent-dx/blob/refactor/gateway/apiserver/docs/environment-lifecycle-api.md).
The SDK uses these paths internally:

```text
POST /api/environment/v1/environments/{id}/snapshots
POST /api/environment/v1/environments                 # create from Snapshot: snapshotId
POST /api/environment/v1/environments/{id}/pause
POST /api/environment/v1/environments/{id}/resume
POST /api/environment/v1/environments/{id}/reload
PUT  /api/environment/v1/environments/{id}/network
```

Raw HTTP has intentionally different validation in a few places. A raw
Snapshot request has `name` as its handler field: an omitted or empty name is
accepted, while a supplied whitespace-only name is rejected. Its
`timeoutSeconds` defaults to 300, is validated in the range 1 through 3600,
converted to milliseconds, and forwarded as the checkpoint RPC logical timeout. Raw Pause accepts optional `retention` as `{"kind":"forever"}` or
`{"kind":"ttl","ttlSeconds":3600}`; omission inherits the creation policy.
Its `timeoutSeconds` is validated from 1 through 3600 and defaults to 300. Only Snapshot and Pause currently accept this caller-provided logical
timeout body field. The SDK sends `{}` for resume and reload, and their
handlers define no body fields. Raw HTTP requires a pattern-valid
`X-ADX-Request-ID` header;
the SDK generates that header internally.

Internal signal/POSIX/runtime SDK APIs are not part of this package or the new Environment protocol.

## Other create options

Request one kind of whole device with `type:model:count`:

```python
Environment(xpu="gpu:l20:1")
Environment(xpu="gpu:h100:2")
Environment(xpu="gpu::1")  # any GPU model
Environment(xpu="npu:ascend910b4:1")
```

The SDK accepts one `gpu` or `npu` request with a positive whole-device count.
An empty model leaves the scheduler to select a model. Real device execution
requires a suitable worker and is a separate pending validation gate; see the
[whole-device acceptance](../../../../build/e2e/device/README.md).

Temporary writable storage is specified in MiB:

```python
Environment(storage_mb=153600, storage_limit_mb=153600)
```

`storage_limit_mb=0` uses `storage_mb`, or the cluster default when
`storage_mb` is omitted. A nonzero limit cannot be below `storage_mb`. The
request controls scheduling while the limit is passed separately to sandboxd's
writable-layer enforcement.

## Rootfs and mounts

`rootfs=S3Config(...)` starts from an S3-compatible EROFS root filesystem.
`Mount` supports image-backed read-only bind mounts and S3-backed bind or EROFS
mounts. Credentials are sent to the control plane and sandboxd; callers should
use scoped object-store credentials. EXECD is still supplied by the deployment's
local runtime environment, so a custom rootfs does not need to bake in ADX.

The deployment Runtime Environment is the rootfs baseline. `runtime=` overrides
only its isolation runtime. `rootfs_readonly=True` or `False` overrides only its
read-only setting; leaving it as `None` inherits the deployment default. Passing
`image=` or `rootfs=S3Config(...)` replaces the source atomically while omitted
runtime/read-only fields continue to inherit. Runtime-only and read-only-only
overrides keep the baseline source and do not add the bootstrap mount.

## Network policy models

`NetworkPolicy`, `NetworkRule`, and `PortRange` are accepted at creation.

Deny-all policies retain platform rules for Execd and declared public TCP
ports. Stateful mode uses connection tracking for replies; stateless mode
matches those local service ports in both directions. When runtime control is
configured, a separate rule preserves the node's exact control IPv4/TCP
endpoint. Ordinary traffic remains subject to the user policy; see the
[node network contract](../../../../docs/deployment/runtime-control-stream.md).
`update_network_policy(policy)` atomically replaces the complete runtime policy;
passing `None` clears it. Adxlet reserves the EXECD control port and declared
published ports with the highest rule priority so a user default-deny policy
cannot cut the control route. User priorities must be in `1..UINT32_MAX-1`.
Enforcement is provided by sandboxd's network policy implementation.

## Connection configuration

Pass an immutable `ConnectionConfig` to avoid process-global environment
configuration:

```python
from adx_environment import ConnectionConfig, Environment, resources

connection = ConnectionConfig(
    server_address="frontend.example.com:443",
    token="<token>",
    use_tls=True,
    gateway_address="gateway.example.com:443",
    gateway_use_tls=True,
    communication_margin_seconds=30,
    request_max_attempts=3,
)

with Environment(image="python:3.12-slim", connection=connection) as environment:
    print(environment.id)

nodes = resources(connection=connection)
Environment.delete("environment-id", connection=connection)
```

`resources()` reports scheduling capacity and remaining allocatable resources.
Online nodes that are not accepting allocations remain in the result with
status 1, zero CPU/Memory/Disk resources and no GPU/NPU resource entries.
Existing allocations remain held; this view does not report actual utilization
or the rated capacity of admission-disabled nodes.

Without a `ConnectionConfig`, the SDK reads `ADX_SERVER_ADDRESS`, `ADX_TOKEN`,
`ADX_TLS`, `ADX_GATEWAY_ADDRESS`, `ADX_GATEWAY_TLS`,
`ADX_COMMUNICATION_MARGIN_SECONDS`, and `ADX_REQUEST_MAX_ATTEMPTS`.
The communication margin must be finite and between 0 and 3600 seconds;
request_max_attempts must be a positive integer. The gateway address
defaults to the frontend address for reverse-tunnel and user-port routes. PTY
uses the Ingress data-plane route when a gateway address is configured, and falls
back to the server address for combined deployments.

## Build and test

From this directory:

```bash
PYTHON=python3 bash build.sh /tmp/adx-environment-dist
PYTHONPATH=. python3 -m unittest discover -s tests/unit
```

The SDK version is defined once in `pyproject.toml`, matching the AKernel SDK
packaging model. Change `[project].version` when publishing a new SDK release;
the build does not infer or override it from repository tags or environment
variables.

From the monorepo root, the SDK wrapper does the same:

```bash
PYTHON=python3 bash platform/sdk/environment/build.sh /tmp/adx-environment-dist
```

Live K8S/frontend checks also need `ADX_SERVER_ADDRESS`,
`ADX_GATEWAY_ADDRESS`, and a valid token:

```bash
PYTHONPATH=. python3 tests/e2e_execd_direct.py
PYTHONPATH=. python3 examples/reverse_tunnel.py
```

## Runnable examples

Examples are client demonstrations. Basic commands/files are supported; tunnel and user-port examples require matching published routes and are not part of the new basic K8s acceptance:

- `examples/basic_usage.py`
- `examples/command_stdin.py`
- `examples/persistent_shell.py`
- `examples/tunnel_large_response.py`
- `examples/port_forwarding.py`
- `examples/reverse_tunnel.py`
- `examples/named_environment.py`
- `examples/bench_cp.py`
- `examples/recoverable_command.py`

Infra-specific demos should be documented separately instead of being shipped as
runnable SDK examples.

## Architecture

- **Control plane** — environment v1 create, delete, lifecycle, and invoke routes.
- **Direct data plane** — frontend/gateway `/direct/{environment}/...` routes for
  command invoke and binary file I/O.
- **Reverse tunnel** — gateway `/tunnel/{environment}` WebSocket back to a local
  upstream (`adx_environment/tunnel_client.py`).

See [`TODO.md`](TODO.md) for remaining SDK work.

## Recover a background command

Background commands run without an execution deadline by default. Pass
`timeout` to set one explicitly; foreground commands default to 60 seconds.

The SDK process does not persist command handles. Generate a command ID and
persist the `(environment_id, command_id)` pair in the caller's own database before
submission, then bind fresh handles after a restart:

```python
from adx_environment import Environment

environment = Environment.from_id(saved_environment_id)
command = environment.commands.run(
    "python train.py",
    background=True,
    command_id=saved_command_id,  # generate and persist before submission
)

# In a later SDK process:
environment = Environment.from_id(saved_environment_id)
command = environment.commands.get(saved_command_id)
result = command.wait()
```

`wait()` and `wait_async()` use one hidden multiplexed WebSocket per connection
context as a notification channel. The final result is always read back from
EXECD's authoritative command registry. When the wait deadline expires, they
return `CommandResult(status=RUNNING, error_code="WAIT_TIMEOUT")`; the remote
command remains active and the handle can wait again. A client restart also
does not terminate the remote command. `CommandHandle.kill()` and
`environment.commands.kill()` return `False` when the command is absent or already
finished, and still raise on other execution or transport failures.
Long-running command polling retries connection failures and errors marked
retryable by the server. Terminal errors keep their original error and request ID.
Temporary downstream subscription failures (including a checkpoint freeze)
reconnect the watch for the same command, without starting it again. The SDK
uses a 30-second continuous-outage budget; an Ingress WebSocket handshake alone
does not reset it. Receipt of an actual PENDING/RUNNING or terminal command state
resets the budget. Authorization and permanent subscription errors fail directly.


### Placement

`Environment(..., labels={"app": "worker"}, schedule_affinities=[...])` exposes the
platform's grouped node and same-tenant instance placement rules. Conditions
use the public `scheduleAffinities` shape with `kind`, `affinity`, `labelOps`,
optional `weight` and `preferredPriority`. `node_id` remains a hard requirement
when combined with alternatives. See [placement rules](https://gitcode.com/openJiuwen/agent-dx/blob/refactor/docs/testing/http-node-placement.md)
for the matching, ranking and label contracts. Caller-provided option lists are
copied before the SDK adds a node constraint.
