Metadata-Version: 2.4
Name: transformerlab-internal
Version: 0.1.61
Summary: Python SDK for Transformer Lab
Author-email: Transformer Lab <developers@lab.cloud>
License: Apache-2.0
Project-URL: Homepage, https://github.com/transformerlab/transformerlab-platform
Project-URL: Bug Tracker, https://github.com/transformerlab/transformerlab-platform/issues
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: werkzeug
Requires-Dist: pytest
Requires-Dist: pytest-asyncio
Requires-Dist: wandb
Requires-Dist: fsspec
Requires-Dist: s3fs>=2023.12.0
Requires-Dist: aiofiles
Requires-Dist: gcsfs
Requires-Dist: adlfs
Requires-Dist: requests
Requires-Dist: trackio
Dynamic: license-file

# Transformer Lab SDK

The Transformer Lab Python SDK provides a way for ML scripts to integrate with Transformer Lab.

## Install

```bash
pip install transformerlab-internal
```

## Usage

```
from lab import lab

# Initialize with experiment ID
lab.init("my-experiment")
lab.log("Job initiated")

config_artifact_path = lab.save_artifact(<config_file>, "training_config.json")
lab.log(f"Saved training config: {config_artifact_path}")
lab.update_progress(1)

...
lab.update_progress(99)

model_path = lab.save_model(<training_output_dir>, name="trained_model")
lab.log("Saved model file to {model_path}")

lab.finish("Training completed successfully")
```

## Shared storage

Each user has a personal area of the team workspace for large files reused across
experiments and jobs — a base model, a tokenized corpus, a reference checkpoint.
Upload once from the terminal with `lab storage upload`, then read it from any job:

```python
from lab import lab

corpus = lab.storage_download("tokenized-corpus")                 # directory → local dir
weights = lab.storage_download("models/base-model.safetensors")   # file → local path

for entry in lab.storage_list("models"):
    print(entry["relpath"], entry["size"])

lab.storage_upload("./derived-index.faiss", "indexes/wiki.faiss")
```

Downloads always land on the node's own disk — the returned path is a file you can
open. By default they go to a cache under `~/.transformerlab/cache/user_storage/`
(pass `dest=` to choose a location) and read straight from object storage, so a
multi-GB file moves at full bandwidth without routing through the API server. A
local copy whose size already matches is reused, so re-running a job does not
re-fetch it; pass `force=True` to override.

Every path is relative to your own storage root. Async variants exist for all three
(`async_storage_download`, `async_storage_upload`, `async_storage_list`).

## Reporting metrics

Pass a **dict** of named metrics to `lab.finish(score=...)`. The dict shape is required —
metrics are stored under `job_data.score` and surfaced by `lab job list` (Score column)
and `lab job info`, and read by sweep / autoresearch flows for optimization.

```python
lab.finish(message="Done!")                              # success, no score
lab.finish(message="Done!", score={"accuracy": 0.78})    # success with one metric
lab.finish(score={"accuracy": 0.78, "f1": 0.83})         # multiple metrics
```

Do not pass a scalar (e.g. `lab.finish(score=0.78)`) — wrap it in a dict instead:
`lab.finish(score={"score": 0.78})`.

Sample scripts can be found at
https://github.com/transformerlab/transformerlab-platform/tree/main/lab-sdk/scripts/examples

## Development

The code for this can be found in the `lab-sdk` directory of
https://github.com/transformerlab/transformerlab-platform

To develop locally in editable mode and run automated tests:

```bash
cd lab-sdk
uv venv
uv pip install -e .
uv run pytest  # Run tests
```
