Metadata-Version: 2.4
Name: beaker-sdk
Version: 0.6.5
Summary: Beaker prompt-optimization SDK and CLI by BeakerAI.
Author: BeakerAI
License-Expression: MIT
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: gh-bin==2.97.0 ; sys_platform != 'win32'
Requires-Dist: packaging>=24
Requires-Dist: pydantic>=2,<3
Requires-Dist: pyyaml>=6
Requires-Dist: claude-agent-sdk>=0.2,<1 ; extra == 'claude-agent-sdk'
Requires-Dist: opentelemetry-api>=1.30,<2 ; extra == 'claude-agent-sdk'
Requires-Dist: opentelemetry-sdk>=1.30,<2 ; extra == 'claude-agent-sdk'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.30,<2 ; extra == 'claude-agent-sdk'
Requires-Dist: opentelemetry-proto>=1.30,<2 ; extra == 'claude-agent-sdk'
Requires-Dist: langchain-core>=0.3,<2 ; extra == 'langchain'
Requires-Dist: opentelemetry-api>=1.30,<2 ; extra == 'langchain'
Requires-Dist: opentelemetry-sdk>=1.30,<2 ; extra == 'langchain'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.30,<2 ; extra == 'langchain'
Requires-Dist: litellm>=1.92,!=1.107.0,<2 ; extra == 'litellm'
Requires-Dist: opentelemetry-api>=1.30,<2 ; extra == 'litellm'
Requires-Dist: opentelemetry-sdk>=1.30,<2 ; extra == 'litellm'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.30,<2 ; extra == 'litellm'
Requires-Dist: openai-agents>=0.20,<1 ; extra == 'openai-agents'
Requires-Dist: opentelemetry-api>=1.30,<2 ; extra == 'openai-agents'
Requires-Dist: opentelemetry-sdk>=1.30,<2 ; extra == 'openai-agents'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.30,<2 ; extra == 'openai-agents'
Requires-Dist: pydantic-ai>=1.74 ; extra == 'pydantic-ai'
Requires-Dist: opentelemetry-api>=1.30,<2 ; extra == 'pydantic-ai'
Requires-Dist: opentelemetry-sdk>=1.30,<2 ; extra == 'pydantic-ai'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.30,<2 ; extra == 'pydantic-ai'
Requires-Dist: opentelemetry-api>=1.30,<2 ; extra == 'tracing'
Requires-Dist: opentelemetry-sdk>=1.30,<2 ; extra == 'tracing'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.30,<2 ; extra == 'tracing'
Requires-Dist: opentelemetry-proto>=1.30,<2 ; extra == 'tracing'
Requires-Dist: verifiers>=0.3.1,<0.4 ; extra == 'verifiers'
Requires-Dist: opentelemetry-api>=1.30,<2 ; extra == 'verifiers'
Requires-Dist: opentelemetry-sdk>=1.30,<2 ; extra == 'verifiers'
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.30,<2 ; extra == 'verifiers'
Requires-Python: >=3.12
Project-URL: Cookbook, https://github.com/rilixai/beaker-cookbook
Project-URL: Documentation, https://www.withbeaker.ai/docs
Project-URL: Homepage, https://www.withbeaker.ai
Provides-Extra: claude-agent-sdk
Provides-Extra: langchain
Provides-Extra: litellm
Provides-Extra: openai-agents
Provides-Extra: pydantic-ai
Provides-Extra: tracing
Provides-Extra: verifiers
Description-Content-Type: text/markdown

# beaker

[Website](https://www.withbeaker.ai) · [Documentation](https://www.withbeaker.ai/docs)

Beaker SDK 0.6.5 defines a customer `Integration`, validates it locally, uploads
datasets and operates hosted optimization runs. Import contract types from
`beaker` and declare a module-level `integration = Integration(...)` value.

```python
from beaker import (
    Integration,
    Case,
    CaseResult,
    CaseScore,
    RolloutRuntime,
    RunSetup,
    SetupRuntime,
    repository,
)

integration = Integration(
    targets=repository(),
    run_setup=MySetup,
    run_case=run_case,
    score_case=score_case,
)
```

`MySetup` subclasses `RunSetup`, declares a Pydantic-compatible
`row_model`, and implements async-generator `load_cases(row, *, runtime)`.
It yields `Case(id=..., input=..., expected=...)` values with unique IDs.
Beaker constructs one setup instance per attempt and enters/closes its
`prepare_run()` async context manager, which yields one result per declared
target kind; the default yields `{"repository": RepositoryRunSetupResult()}`.

`run_case(*, case_input, runtime)` returns `CaseResult(output=...)`. Its JSON
application result is what the scorer reads; telemetry belongs in `runtime.trace`.
`score_case(*, case, result, case_files_dir)` returns `CaseScore` with the
agreed objective, stable field metrics and optional per-case checks.

For documents, declare `documents(groups=(...))` and override `prepare_run()`. Declare it
alone to optimize only documents, or beside the repository to let one candidate change code and
documents together: `targets=(repository(...), documents(groups=(...)))`. Declare one `documents()`
target per document store (two S3 buckets of wikis are two targets); group names must be unique
across them, and there is at most one `repository()`. `prepare_run()` yields one result per target
kind, such as `{"repository": RepositoryRunSetupResult(), "documents": DocumentRunSetupResult(...)}`:
a single documents result holds the `TargetDocument` seeds of every document target, and each
document's group says which target it belongs to. `run_case` reads the candidate's documents from
`runtime.targets_dir`. To build state every case of a candidate shares, such as an index of its
documents, pass `Integration(open_candidate=...)`, an async context manager taking
`(*, targets_dir, scratch_dir)`: it is entered once per rollout process before any case, its value
reaches `run_case` as `runtime.candidate`, and it is exited after the last case. Each document
target's result is its own `DocumentChangeSet` of conditional
operations, naming its `groups`; the customer decides when to apply them.

## Configuration and validation

`beaker init` preserves existing configuration. For a new setup, it creates
`.beaker/beaker_integration.py` and:

```yaml
default_integration: invoices
integrations:
  invoices:
    entrypoint: beaker_integration:integration
    source_dir: .
    package_import_root: .beaker
    required_env: []
```

You can add `targets` to an Integration entry so playbook generation can start
while its image builds. For example, a repository Integration can declare:

```yaml
    targets:
      kind: repository
      paths: ["src"]
```

The declaration must match `Integration.targets`. At run startup, Beaker imports
the Integration with its runtime environment and secrets, then compares its targets
with the declaration saved for that build. A mismatch stops the run before
`prepare_run()` or baseline evaluation and reports both target declarations.
The image build does not import the Integration for this check. Playbook generation
can still start during the build; a mismatch can therefore waste generation work,
but the run cannot use the mismatched declaration for optimization.
The configured `entrypoint` selects the module and attribute; the Integration name
does not change how the module is resolved.

`source_dir` is relative to the Git root; `package_import_root` is inside it.
Use `--integration-id` when several entries exist. Config selection is explicit
ID, configured default, or the sole entry; ambiguous selection fails.
To register an Integration elsewhere, pass
`beaker init --integration-id <id> --target <path.py-or-module:attribute>`.
Init does not scan the repository for Integrations.

`beaker agent setup "<Agent Name>" --integration-id invoices` records the hosted
agent key at `integrations.invoices.agent_key`. Agent selection uses `--agent` /
`--agent-key`, then `BEAKER_AGENT_KEY`, then the selected integration's key.
Top-level `agent_key` and `project_key` fields are no longer used.

```bash
beaker onboarding status --integration-id invoices
beaker agent env check --integration-id invoices
beaker run smoke --strict --integration-id invoices --dataset invoices@revision
beaker run trigger --integration-id invoices --dataset invoices@revision
beaker run status RUN_ID --watch
beaker run pull RUN_ID
```

Smoke validates all rows before entering setup, then loads cases, validates
input files, and opens a document seed candidate if applicable. It closes all
resources without invoking the agent or scorer. Setup hooks may use external
services. Init verification only imports and validates the Integration.

## Models

Integrations support explicitly selected comparison
models using repeated `--optimization-model provider:model` flags. When
`runtime.model` is present, `inference_target(runtime)` provides the hosted
gateway configuration. Otherwise preserve the application's production defaults.
If scoring requires a hosted LLM judge, declare `requires_scorer_model: true`
and `scorer_model` on the selected integration in `.beaker/beaker.yaml`:

```yaml
integrations:
  my-agent:
    # ...existing integration settings...
    requires_scorer_model: true
    scorer_model: openrouter:deepseek/deepseek-v4-flash
```

Call `scoring_inference_target()` inside scoring. Judge selection is owned by
the integration, not launch settings or `config_defaults`; CLI/API overrides
are rejected. Missing or invalid required judges fail before creating a build
or run. Provider credentials are checked before launch. Integrations without
LLM scoring can still launch without a judge.

Read [the CLI reference](../../fern/pages/cli-reference.md) for commands.

The SDK and skills are released together with matching versions. Deploy the
compatible hosted runtime before publishing the SDK and skills.
