Metadata-Version: 2.4
Name: learnml-sdk
Version: 0.1.10
Summary: Python SDK for the LearnML training data management platform
Home-page: https://github.com/milindjain0/learnml
Author: LearnML
Author-email: milindjain0@gmail.com
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: grpcio>=1.68.1
Requires-Dist: requests<3,>=2.32.0
Requires-Dist: requests-toolbelt<2,>=1.0.0
Requires-Dist: protobuf<6.0.0,>=5.28.1
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# LearnML Python SDK

The complete [Python SDK reference](../docs/sdk-reference.md) documents every public client method, its parameters, return values, and usage examples. Open **SDK Documentation** in the app, or visit `/docs/sdk` (no sign-in required).

HTTP(S) URLs select the website API; bare host:port addresses retain direct gRPC.

```python
from getpass import getpass
from learnml import LearnMLClient

with LearnMLClient("https://your-learnml-server.example") as client:
    client.login("you@example.com", getpass("LearnML password: "))
    universe = next(u for u in client.list_universes() if u["name"] == "My experiments")
    uid = universe["id"]
    run = client.create_training_run(uid, "my-experiment", model_name="my-model")
    client.log_metrics(uid, run["id"], step=1, loss=0.42, accuracy=0.91)
    client.upload_data(uid, "experiment-data", "CUSTOM", "experiment.csv")
    client.end_training_run(uid, run["id"])
```

Install with `python -m pip install --upgrade learnml-sdk`, or install a local checkout with `python -m pip install ./sdk`. Restart a notebook kernel after upgrading an already-imported package. HTTP support requires version 0.1.3 or later.

HTTP mode preserves the existing public methods and camelCase response dictionaries. `token` and refresh state work in both modes. Requests have connection/read timeouts; API and network errors use the SDK exception classes. HTTP multipart uploads stream from the client file; the current gateway still buffers uploads in server memory. `chunk_size` controls gRPC chunks; the HTTP transport controls its own multipart read sizes. HTTP collection batching uses paginated API calls.

HTTP `save_checkpoint(metadata=...)` supports nonempty metadata when the server advertises `directUploads`; otherwise it raises `LearnMLError` with code `UNIMPLEMENTED`. Empty metadata works with ordinary multipart uploads. Files above 8 MiB use direct uploads when supported; otherwise gateway request limits apply.

Run regression tests with `python -m unittest discover -s sdk/tests` from the repository root after installing the SDK.

## API tokens for Colab and long-running jobs

Sign in to the website, open **API Tokens**, and select **Generate token**. Give the token a name and choose **No expiry — until revoked**, or a fixed expiry. Copy the full token immediately: it is shown only once. Token hashes, names, prefixes and timestamps are stored on the server; full secrets cannot be retrieved later.

Save it in Colab Secrets as `LEARNML_API_TOKEN`, enable notebook access, and use:

```python
from google.colab import userdata
from learnml import LearnMLClient

client = LearnMLClient(
    "your-learnml-server.example:50051",
    token=userdata.get("LEARNML_API_TOKEN"),
)
# No login or token-refresh loop is required.
print(client.list_universes())
```

HTTP clients also accept the same token. The `token=` argument works in older SDK versions; SDK 0.1.4 adds the explicit `api_token=` alias. Pass one of these arguments, not both.

To manage tokens from SDK 0.1.4, first sign in using `client.login(...)`, then call:

- `create_api_token(name, expires_in_days=0)` returns `{ "apiToken": {...}, "token": "lml_..." }` once. Zero days means no automatic expiry; 1–3650 days sets an expiry.
- `list_api_tokens()` returns metadata and token prefixes, never full secrets.
- `revoke_api_token(token_id)` disables an owned token on its next request.

API tokens inherit your current workspace permissions; removing membership removes access. They cannot create, list, or revoke credentials. Use a password-based login session to manage tokens. Store tokens like passwords; their presence does not add encryption to a plaintext HTTP/gRPC connection.

### Create and edit individual rows

For bulk creation, `batch_create_data_points(universe_id, rows,
collection_id=None)` accepts up to 10,000 dictionaries with the same keyword
names as `create_data_point`: `name`, `data_type`, `labels`, `metadata`,
`raw_content` (bytes), and `llm_data` (dict). It returns `dataPointIds` in input
order and `createdCount`. Rows and optional collection membership commit together.
Requests are limited to 32 MiB total and 1 MiB per row. Split larger datasets into
multiple calls; there are no automatic retries or deduplication after ambiguous
network failures. See the repository's `docs/row-api.md` for batch examples.

Row content can live directly in PostgreSQL, without a bucket file. Inline content
is limited to 1 MiB per create/update request. File uploads remain available for
larger data. These methods require a server with the row-content update enabled.

```python
row = client.create_data_point(
    universe_id, "sample-001", "LLM_SFT",
    llm_data={"instruction": "Tag names", "input": "Hello Alice",
              "output": "Hello <NAME>Alice</NAME>",
              "metadata": {"spans": '[{"start": 6, "end": 11}]'}},
    labels={"split": "train"},
    metadata={"source": "manual"},
)
row = client.update_data_point(
    universe_id, row["id"],
    llm_data={"input": "Hello Bob", "output": "Hello <NAME>Bob</NAME>"},
)
collection = client.create_collection(universe_id, "Examples")
client.add_to_collection(universe_id, collection["id"], [row["id"]])
row = client.get_data_point(universe_id, row["id"])
for batch in client.stream_data_batch(universe_id, collection["id"], include_content=True):
    print(batch)
```

For text or arbitrary JSON rows, supply `raw_content=text.encode("utf-8")`.
For JSON, set `metadata={"content_type": "application/json"}`. Read responses
represent `rawContent` as base64; decode with `base64.b64decode(row["rawContent"])`.

Updates replace only supplied fields. Passing `metadata={}` or `labels={}` clears
that map; `raw_content=b""` saves an intentionally empty row. Replacing `llm_data`
replaces the whole structured example, so include every LLM field you want to keep.
Labels and both metadata maps use string values; encode nested annotations as JSON
strings. `GetDataPoint` returns inline content; list RPCs omit it to keep pages small.

RPCs: `DataService.CreateDataPoint`, `GetDataPoint`, `UpdateDataPoint` (with
`google.protobuf.FieldMask`), and `StreamDataBatch(include_content=true)`.
Collections use `CollectionService.AddDataToCollection`, `ListCollectionData`, and
`RemoveDataFromCollection`. Removing membership keeps the row itself. HTTP clients
use POST/GET/PATCH `/api/universes/{id}/data[/{row_id}]` and the collection endpoints.


### Create or reuse a collection while writing rows (SDK 0.1.6)

`CreateDataPoint`, `UpdateDataPoint`, and `BatchCreateDataPoints` accept either
`collection_name` or `collection_id`. A name reuses the exact, case-sensitive name
in the current workspace, creating it if absent. Names must be nonblank and at
most 200 UTF-8 bytes. An ID must already exist in that workspace. Do not supply
both. Omit both to retain the existing behavior.

```python
row = client.create_data_point(universe_id, "example", raw_content=b"hello",
                               collection_name="Training examples")
client.update_data_point(universe_id, row["id"], raw_content=b"edited",
                         collection_name="Reviewed examples")
# Collection-only updates preserve content and other row fields.
client.update_data_point(universe_id, row["id"], collection_name="Training examples")
batch = client.batch_create_data_points(universe_id,
    ({"name": f"row-{i}", "raw_content": f"example {i}".encode()} for i in range(10000)),
    collection_name="Training examples")
collection_id = batch["collectionId"]
```

Each operation commits the row writes, optional collection creation, and membership
in one PostgreSQL transaction. Assignment adds membership without removing existing
memberships; repeat assignment does not duplicate it. Batch collection selection is
at the request level and applies to every row. Responses include `collection_id`
(`collectionId` in JSON and SDK results) when assigned. HTTP request fields are
`collectionName` / `collectionId`; the RPC fields use snake_case. Direct RPC updates
should use `update_mask` for explicit field replacement/clearing; a collection-only
request needs no mask.

The batch supports up to 10,000 rows, subject to 1 MiB per row and 32 MiB serialized
request size. Inline content is stored in PostgreSQL `data_points`; membership is
stored in `collection_data_points`. Uploaded files continue to use object storage.

## Evaluation results and version comparison (0.1.7)

Save immutable collection versions with `create_collection_version`, iterate source
examples with `iter_version_examples`, upload externally computed model results
with `upload_evaluation`, and compare with `compare_collection_versions`.
See [the complete evaluation guide](https://github.com/milindjain0/learnml/blob/main/docs/evaluation-comparison.md) for Colab examples and limits.

## Maintaining the complete reference

After changing the public client API, update the reviewed descriptions and examples in
`scripts/build_sdk_reference.py` from the repository root, then run:

```bash
python3 scripts/build_sdk_reference.py
python3 scripts/build_sdk_reference.py --check
```

The generator reads signatures directly from the client source, checks that every public
method has an entry, and produces both the Markdown reference and the app page data.
It needs Python 3.9+ and does not connect to a server.

## Reading stored content (0.1.10)

`open_data(uid, data_point_id)` returns a readable binary stream of a row's stored content:
bucket files stream as they download, so reading the first records of a large file transfers
only about that much. `download_data(uid, data_point_id, dest_path)` saves the content to a
file, writing `dest_path + ".part"` first and checking the stored SHA-256 before renaming.

```python
import gzip, json
with client.open_data(uid, row_id) as stream, gzip.open(stream, "rt") as lines:
    first = [json.loads(next(lines)) for _ in range(5)]
client.download_data(uid, row_id, "part-00000.jsonl.gz")
```

Bucket content needs the HTTP(S) API URL; gRPC mode reads inline content only.
