Metadata-Version: 2.5
Name: runtime-labs-sdk
Version: 0.1.1
Summary: OpenTelemetry tracing with eval payloads and verified source identities
Project-URL: Repository, https://github.com/arpandesai0/runtime-labs
Project-URL: Documentation, https://github.com/arpandesai0/runtime-labs/blob/main/packages/sdk/README.md
Requires-Python: <3.14,>=3.11
Requires-Dist: httpx<1,>=0.28
Requires-Dist: opentelemetry-api==1.45.1
Requires-Dist: opentelemetry-exporter-otlp-proto-http==1.45.1
Requires-Dist: opentelemetry-sdk==1.45.1
Requires-Dist: python-dotenv<2,>=1.1
Requires-Dist: runtime-labs-contracts
Provides-Extra: google
Requires-Dist: google-genai==2.27.0; extra == 'google'
Provides-Extra: langgraph
Requires-Dist: langgraph==1.2.18; extra == 'langgraph'
Provides-Extra: otlp
Requires-Dist: opentelemetry-exporter-otlp-proto-http==1.45.1; extra == 'otlp'
Description-Content-Type: text/markdown

# Runtime Labs local Python SDK

The SDK emits OpenTelemetry spans with full eval payloads for LangGraph execution
and Google Gen AI non-streaming model calls. The scanner's strict `DeploymentManifest`
adds `runtime_labs.node_id` when source proof succeeds. Tracing also works with
missing or stale manifests: those operations remain unattributed and the handle
reports fixed diagnostic codes.

## Install and initialize

From this workspace:

```bash
uv sync
uv run python examples/sdk_offline/demo.py
```

For an application with its own environment, install both local distributions
(substitute the checkout directory):

```bash
uv pip install /path/to/runtime-labs/packages/contracts \
  '/path/to/runtime-labs/packages/sdk[google,langgraph]'
```

`runtime-labs-sdk` is independently packaged and has no scanner dependency. The
Google and LangGraph adapters are optional and loaded at initialization; importing
`runtime_labs` does not import either integration. Unsupported dependency versions
are diagnosed and their adapters are skipped.

Connect and scan the application checkout in the web UI first. Run the application
inside its Git checkout with an `origin` remote. No manifest file is downloaded or
configured by the customer. The SDK reads that remote and asks the backend for the
successful scan's manifest at startup. SSH and HTTPS remotes normalize to the same
host/owner/repository identity.

Optional application `.env` (the default is shown):

```dotenv
RUNTIME_LABS_API_URL=http://127.0.0.1:8000
```

At startup, before constructing provider clients or building/executing graphs:

```python
import runtime_labs

sdk = runtime_labs.init(api_key="runtime-labs-local-dev-key")

try:
    # Normal application startup and execution.
    ...
finally:
    sdk.flush()
    sdk.shutdown()
```

The API key is sent to the backend as an Authorization bearer header for manifest
lookup and automatic telemetry delivery. Authentication and workspace lookup are
currently deferred: the backend uses the default tenant/workspace. The key is
hidden from configuration representations and is never included in span payloads
or provider credentials.

Configuration rules:

- `api_url=` overrides process `RUNTIME_LABS_API_URL`, then the nearest `.env` value,
  then the local default above. SDK settings are read without interpolation or
  mutating the application's environment.
- By default the SDK resolves the checkout's `origin` and HEAD, requests
  `POST /api/sdk/manifest`, validates the strict deployment manifest, and holds it
  in memory. No manifest cache file or customer download step is required.
- The backend resolves the complete canonical Git identity, reads the successful
  scan's saved artifact, and returns the manifest and repository ingestion route.
  Duplicate matching checkouts, unavailable artifacts, and revision mismatches
  are rejected rather than guessed.
- Existing source-path, source-digest, and loaded-code proofs still apply.
- For explicit offline/test use only, `RUNTIME_LABS_MANIFEST_PATH` remains a legacy
  override. Paths retain their existing `~` and nearest-`.env` resolution rules.
  An empty override enables normal runtime fetching instead of disabling attribution.
- A failed lookup never breaks application execution. Inspect `sdk.diagnostics`
  for `repository_identity_unavailable`, `manifest_fetch_failed`, or
  `manifest_revision_mismatch`. Unsupported or unverified spans stay unattributed.
- Deployed environments must retain Git metadata for automatic remote discovery.
  Initialization fetches once with a five-second HTTP timeout; retry after fixing
  backend/scan availability by shutting down and initializing again.

## Public API and local output

`init(*, api_key, exporter=None, tracer_provider=None, capture_payloads=True, api_url=None)`
returns an SDK handle with:

- `diagnostics`: a sorted tuple of sanitized, deduplicated diagnostic codes.
- `sources.manifest`: a detached strict-contract snapshot; editing it cannot change
  the immutable attribution indexes. Repository/revision metadata stay here.
- `flush(timeout_millis=30000) -> bool`: flush only this SDK's exporter.
- `shutdown()`: flush, disable instrumentation, restore owned hooks, and release
  SDK-owned resources. It is safe to call repeatedly.

Identical initialization returns the same handle. A different key, resolved path,
exporter, explicit provider or payload setting returns that handle with
`initialization_conflict`,
without installing more hooks. Shut down and initialize again to change settings;
construct new graphs after reinitialization. Shutdown should follow completion or
cancellation of application work; it does not await running requests.

After successful runtime bootstrap, the SDK automatically exports OTLP/HTTP spans
to the repository route supplied by the backend. The OTLP exporter is included in
the base SDK dependency set. Explicit exporters and standard `OTEL_*` settings
remain supported; `OTEL_TRACES_EXPORTER=none` disables delivery. Offline manifest
mode retains the console default. Console records contain trace/span/parent IDs,
start/end timestamps, duration, status, and allowed attributes including application
payloads. HTTP authentication, environment dumps, and source bodies are not captured.
Errors use only `error.type` values `cancelled`, `timeout` or `_OTHER`; automatic
exception events and status descriptions are disabled. Application code and other
instrumentation retain their own logging/export policies.

## Payloads for evals

Full payload capture is enabled by default. The MVP performs **no redaction or SDK
size limiting**. Each payload attribute is a JSON string, supported by standard
OTel exporters. Decode it with `json.loads(span.attributes[attribute_name])`.

| Span | Payload attributes |
| --- | --- |
| Workflow / agent | `runtime_labs.input`, `runtime_labs.output`: input state and returned value |
| Google model call | `runtime_labs.request`: original model, contents and supplied generation config; `runtime_labs.response`: returned response body, candidates, usage, parsed output and automatic tool-calling history |
| Google model call | `gen_ai.input.messages`, `gen_ai.output.messages`: standard message/part representation; `gen_ai.system_instructions` when supplied |
| Tool execution | `gen_ai.tool.call.arguments`, `gen_ai.tool.call.result`: bound arguments and returned value |

The `runtime_labs.*` payload attributes carry application data, not additional
source identity fields. `runtime_labs.node_id` remains the only custom source ID.
Prompts, completions, all returned candidates and tool payloads are stored in full.
Raw provider request/response attributes retain provider-specific body fields;
HTTP options and `sdk_http_response` transport envelopes are not body payloads.
The SDK does not inspect client credentials or automatically export its API key.

Request/input snapshots are taken before execution, so in-place application or
Google automatic function-calling mutations cannot rewrite the original input.
The returned automatic function-calling history retains its intermediate tool
calls/results. Workflow outputs contain the final graph state, including the
application's answer, enabling complete input/output evaluation examples. Runtime
trace and span IDs join these payloads to the manifest's source node IDs.

JSON-compatible values, Pydantic models, dataclasses, enums, dates and bytes are
supported; bytes are encoded as base64. Callable tool definitions store callable
names and qualified names rather than executable code. Tool arguments are serialized
independently: unsupported or cyclic argument values (such as database sessions
and ORM models) are omitted while the other arguments, including defaults, remain
captured. `runtime_labs.tool.arguments.omitted` lists the omitted parameter names,
and the handle reports `tool_arguments_omitted`. No object internals or string
representations are inspected as a fallback. Unsupported or cyclic values in other
payloads produce `payload_capture_failed` and omit that payload attribute while
preserving application results/exceptions and other span data. A failed or
cancelled operation keeps its input; it does not invent a response.

To select metadata-only output explicitly:

```python
sdk = runtime_labs.init(api_key="runtime-labs-local-dev-key", capture_payloads=False)
```

OTel/application processors and downstream collectors can have their own limits;
the Runtime Labs SDK does not introduce payload truncation. Local JSON and OTLP
export receive the same payload attributes. Evaluation execution/dataset storage
are separate later components; this change supplies their runtime examples.

For assertions, supply the standard OTel in-memory exporter:

```python
from runtime_labs.export import InMemorySpanExporter

exporter = InMemorySpanExporter()
sdk = runtime_labs.init(api_key="runtime-labs-local-dev-key", exporter=exporter)
# Execute work; then inspect exporter.get_finished_spans().
```

The explicitly selected in-memory exporter retains spans until `clear()` or
`shutdown()`; applications should bound its use. Custom exporters use the standard
OTel `SpanExporter` interface. Export/flush/shutdown exceptions become diagnostic
codes and do not replace application results or exceptions. Local export is
synchronous, so exporter work contributes latency.

An application-owned global `TracerProvider` is respected. An optional explicit
`tracer_provider` also works. The SDK adds one reusable processor scoped to
`runtime_labs`, never shuts down an application provider, and does not export its
other instrumentation scopes through the local exporter. If no application provider
exists, the SDK creates a private OTel provider without taking the global slot.
Active OTel context still establishes parentage across both providers. Application
processors on a shared provider can also observe SDK spans.

Remote OTLP/HTTP export requires **both** explicit standard settings:

```bash
uv pip install '/path/to/runtime-labs/packages/sdk[otlp]'
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4318/v1/traces
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
```

`OTEL_EXPORTER_OTLP_ENDPOINT`, generic protocol/timeout/header settings and the
trace-specific settings are handled by the standard exporter. HTTP/protobuf is the
supported protocol. An endpoint alone never enables remote export; missing extras,
missing endpoint or unsupported protocol produce diagnostics and local fallback.
The pinned exporter's per-instance transport logger is silenced so server reasons
and transport exceptions cannot echo private text; failures use sanitized SDK
diagnostics. Its optional internal metrics use a no-op provider.
`OTEL_TRACES_EXPORTER=none` discards SDK output. No collector/server is included.

## Correlation identifiers and trace context

The SDK records the identifiers a framework uses to tie runs together, verbatim,
and never decides what they mean. On every LangGraph `invoke`/`ainvoke` it copies
allowlisted keys from the run config onto the workflow span as
`runtime_labs.ctx.<framework>.<key>`, for example
`runtime_labs.ctx.langgraph.thread_id`. The defaults are LangGraph's `thread_id`,
`checkpoint_id`, `checkpoint_ns` and `run_id`. Only allowlisted keys leave the
process, because applications often keep user IDs and secrets in the same config;
only non-empty strings, integers and UUIDs are copied, truncated to 256 characters.

Extend the allowlist without code changes:

```bash
export RUNTIME_LABS_CORRELATION_KEYS=langgraph.tenant,langgraph.session
```

or with `runtime_labs.init(api_key=..., correlation_keys={"langgraph": ["tenant"]})`.
Runtime Labs pipelines map these raw identifiers to concepts such as conversations
with rules that are data, so new frameworks need a rule, not an SDK release.

Within one process, span parents follow OpenTelemetry context, which LangGraph and
asyncio carry into their own threads and tasks. Work that crosses a process or
service boundary (an HTTP call, a queue message, an outbox row) must carry the
context itself, as a standard W3C `traceparent`:

```python
headers = runtime_labs.inject_context()        # producer
with runtime_labs.continue_trace(headers):     # consumer
    handle(message)
```

Code that starts raw threads should run work with
`contextvars.copy_context().run(...)` to keep the parent.

## Supported interfaces and conventions

| Component | Pinned/tested version | Coverage |
| --- | --- | --- |
| CPython | 3.11–3.13 | Code identity and UTF-8 call positions |
| Google Gen AI | 2.27.0 | `Models.generate_content`, `AsyncModels.generate_content` |
| LangGraph | 1.2.18 | `StateGraph` construction, `add_node`, `compile`; compiled `invoke`/`ainvoke` |
| LangChain Core | 1.7.0 (workspace lock) | LangGraph callback signature and context propagation |
| OTel API/SDK | 1.45.1 | Context, tracing, processors and standard exporters |
| OTel OTLP/HTTP exporter | 1.45.1 (optional) | Explicit HTTP/protobuf export |
| OTel Python semantic-conventions package | 0.66b1 (SDK dependency) | SDK resource conventions |
| GenAI conventions | `51a0f580f8a6ad804ca4de5e033683c6fbbe5906` | Development conventions, mapping in `semconv.py` |

Verified official references: [OTel Python instrumentation](https://opentelemetry.io/docs/languages/python/instrumentation/),
[GenAI inference conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/51a0f580f8a6ad804ca4de5e033683c6fbbe5906/docs/gen-ai/client-inference.md),
[agent/workflow/tool conventions](https://github.com/open-telemetry/semantic-conventions-genai/blob/51a0f580f8a6ad804ca4de5e033683c6fbbe5906/docs/gen-ai/gen-ai-agent-spans.md),
[Google SDK](https://github.com/googleapis/python-genai/tree/v2.27.0),
[LangGraph](https://docs.langchain.com/oss/python/langgraph/overview), and
[streaming boundaries](https://docs.langchain.com/oss/python/langgraph/streaming).
GenAI conventions remain in development; their revision is explicit rather than
silently adopting future changes.

Provider spans use `generate_content`, `CLIENT`, and `gcp.vertex_ai`, `gcp.gemini`
or `gcp.gen_ai` according to runtime backend metadata. Request/response models come
from actual request/response metadata. Tokens come only from provider usage:
input = prompt count, output = candidate count plus reasoning count when reported;
reasoning and cache-read counts are separately reported when available. Missing
counts remain absent. For automatic function calling, the final SDK response's
reported usage is recorded; counts from hidden HTTP rounds are not estimated or
aggregated. One span covers the whole public logical call, including these rounds.
Return identity, arguments, exceptions and async cancellation are preserved.

Workflow, agent and tool spans use `INTERNAL` and standard `invoke_workflow`,
`invoke_agent` and `execute_tool` operations and names. The only custom source
attribute is `runtime_labs.node_id`. No extra source IDs or static parent IDs are
attached. The Google adapter honors and sets standard OTel instrumentation
suppression, and deduplicates reentry. Uncooperative third-party instrumentors that
ignore suppression cannot be deduplicated reliably.

## Source-binding policy

The SDK consumes `DeploymentManifest` directly and validates all references using
the shared strict contract. It neither scans nor executes repository source, calls
an LLM, generates IDs, or imports modules selected by manifest data.

Attribution requires a unique kind and qualified symbol match, normalized for
Python `<locals>` names, the complete canonical manifest-relative path suffix of
the actual module/frame file, the verified full-file SHA-256, and agreement between
the executing code and code compiled from those bytes. Compilation is never
execution. Checkout filenames alone are normalized out of the code comparison;
bytecode, constants, nested code, closure layout, flags and position tables remain.
Different absolute checkout directories therefore work without scanner paths or
basename-only guessing.
Direct scripts use their module spec or a canonical module name derived from the
manifest path; source proof is still required. Verification suppresses compiler
warnings because those warnings can otherwise echo source lines.

For model calls, the executed CALL instruction must match the exact manifest
range, including UTF-8 byte columns; same-line sync/async calls remain distinct.
Workflow construction must fall within its unique verified constructor range.
Registered callback objects identify nested agents. Known module-level Python
tools and their already-loaded aliases receive wrappers; registration by itself
never emits a tool span. Ordinary functions such as `finalize_response` receive
no span. Stale, missing, ambiguous or unprovable code produces generic spans and
diagnostics, with no guessed source identity.

Files are checked again at each attribution, including after construction or a
previous successful match. Preloaded code is accepted only if its executable code
and positions agree with the current verified file. This rejects a matching disk
file paired with different loaded bytecode/constants. It does **not** reconstruct
historical import-time bytes: comment-only changes with identical code/positions,
changes outside the executing code unit, mutable globals/closures, or deliberate
code-object forgery are not a deployment attestation. The current whole-file digest
must still match the manifest. IDs alone do not validate a deployed revision or
promise identity reconciliation across commits.

Runtime parents always come from OTel context. A direct shared-tool call is under
the executing agent. A Google automatic tool call is under that logical provider
span, itself under the executing agent. Static registration/containment edges do
not override these parents. LangGraph's public runtime events primarily describe
streaming and do not expose this example's exact Python callback identities, so
this adapter uses narrowly tested construction/registration/execution wrappers
without callback arguments or customer decorators.

## Offline acceptance harness

`examples/sdk_offline` contains an unchanged source copy of the sibling simple
example and its saved verified manifest. The sibling application's source and
configuration are untouched. The demo denies socket connections and uses real
pinned clients/graphs with HTTPX mock transports and fake Vertex credentials.

Each of billing, technical and general, through both sync and async graph entry
points, asserts this six-span tree (the selected agent varies):

```text
invoke_workflow build_workflow
  invoke_agent triage_agent
    generate_content offline-request-model
  invoke_agent billing_agent | technical_agent | general_agent
    execute_tool lookup_policy
    generate_content offline-request-model
```

All six spans in each run must have IDs in the loaded manifest. Across the three
branches, the harness observes all ten distinct nodes, including all four model
sites. It checks the actual tool parent on every run and prints node IDs with the
tree. This is an observation of the offline runs, not a source coverage estimate.
Tests additionally exercise true async callbacks/provider calls using a newly
compiled scanner manifest, automatic tool execution, parallel requests, stale
loaded code and failure/privacy/lifecycle cases. Tests deny network globally.

## Limitations

Streaming graph execution has no workflow-lifetime adapter; registered callbacks
may still emit leaf spans. Google streaming, chats and embeddings are not adapted.
Provider-driven tools are covered when they are already-loaded, manifest-matched
Python functions passed to Google; arbitrary callable objects, local tool closures,
cached pre-init callable references, decorators with replaced code, packaged/zip or
unavailable source, and other framework/provider versions are not guaranteed.
Graphs must be constructed after initialization. No whole-program tracing or
instrumentation of ordinary application functions is performed. Hosted lookup,
authentication, collectors, other providers, evaluation execution and cross-commit
reconciliation are outside this milestone.
