## Connect to LangSmith

We need your API key to access your datasets and evaluators.

LangChain API Key
Get your key at smith.langchain.com/settings

## Select a Dataset

Choose the dataset to evaluate your target function against.

## Dataset Preview

Sample rows from your dataset.

## Source Files & Target

Select the files Weco will optimize, then choose the target function.

Source Files
Select the Python files that Weco will modify during optimization.
Target Function
Function (module:function)
e.g. `agent:run_chain` imports `run_chain` from `agent.py`

## Target Adapter

How does your target function accept inputs?

### Raw default

Pass the full inputs dict directly.

`def target(inputs: dict) -> dict`

### LangChain

Your target is a LangChain Runnable. We call `.invoke(inputs)`.

### Single Input

Extracts a single string from common keys (`input`, `question`, `text`, `query`).

## Evaluators

Evaluators score your target function's outputs on each dataset example.

Evaluator Files
Select Python files containing your evaluator functions.

Code Evaluators
Space-separated. Use `module:function` for custom evaluators, or built-in names like `exact_match`. Leave empty to default to your metric name ().
Dashboard Evaluators
Dashboard evaluators run asynchronously on the LangSmith server. Enter the Feedback Key of each evaluator as configured in your LangSmith dashboard.
Metric Function (optional)
A custom function that receives all aggregated evaluator scores as a dict and returns a single float. Use this to combine multiple evaluators into one composite metric (e.g. a weighted average).

## Run Settings

Configure optimization parameters. All fields have sensible defaults.

Optimization
Metric
The metric name your evaluator outputs
Goal
Steps
Optimization iterations (default: 100)
Model
LLM model. Leave blank for auto.
Additional Instructions
LangSmith
Summary Aggregation
Experiment Prefix
Max Examples
Limit examples per eval
Max Concurrency
Parallel eval threads
Dashboard Evaluator Timeout
Seconds to poll for async scores (default: 900)
Execution
Eval Timeout
Seconds per evaluation
Log Directory
Save execution logs and code snapshots
Auto-apply best solution
Require manual review of each change

## Review & Start

Confirm your configuration and begin optimization.

Next time, skip the wizard with:

✓

## Configuration Sent

Return to your terminal — optimization is starting.