Metadata-Version: 2.4
Name: agnipod
Version: 0.5.0
Summary: Official Python SDK for the AgniPod LLM platform
Home-page: https://agnipod.com
Author: AgniPod
Author-email: support@agnipod.com
Project-URL: Documentation, https://docs.agnipod.com
Project-URL: Source, https://github.com/agnipod/agnipod-python
Project-URL: Issues, https://github.com/agnipod/agnipod-python/issues
Keywords: agnipod llm ai inference api sdk gpu operations
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Systems Administration
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: requests>=2.28.0
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: project-url
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# AgniPod Python SDK

AgniPod is a fully managed internal LLM inference platform. You name a model;
the platform provisions GPUs, schedules the work, routes it, retries on
failure, records metrics and reclaims idle capacity. There is no "pick a
provider", no "start an instance", no infrastructure surface.

```bash
pip install agnipod
```

```python
import agnipod

client = agnipod.AgniPod()          # reads AGNIPOD_API_KEY

r = client.generate.create(
    model="qwen3:4b",
    prompt="Name three primary colours.",
    max_tokens=512,
    enable_thinking=False,          # fast, direct answer
)
print(r.content)                    # the answer
print(r.reasoning)                  # thinking (empty when disabled)
print(r.finish_reason, r.truncated) # "stop" / "length", bool
print(r.metrics["tokens_per_second"])
```

---

## Credentials

Two, and they are not interchangeable.

| Credential | Grants | Used by |
|---|---|---|
| API token (the fixed internal token) | inference, models, batches | every program |
| Admin token (a console JWT) | everything under `client.admin` | operations tooling |

There is no username and password. The token is issued out of band; `agnipod
login` verifies it against the platform and stores it at
`~/.agnipod/credentials.json` with mode `0600`, so it never has to live in
shell history or a dotfile:

```console
$ agnipod login                 # prompts (no echo), verifies, stores
$ agnipod login --admin         # the console JWT, for client.admin
$ agnipod status                # what is configured, and where it came from
$ agnipod logout                # remove it again
```

Resolution order, for both credentials:

```
AgniPod(api_key=…, admin_token=…)      explicit, always wins
AGNIPOD_API_KEY / AGNIPOD_ADMIN_TOKEN  environment
~/.agnipod/credentials.json            agnipod login
```

The environment deliberately outranks the stored file — otherwise an
`agnipod login` a developer ran months ago would silently override what CI
exports, which is exactly the "works here, 401 there" failure the ordering
exists to prevent. `agnipod status` names the winning source for that reason.

```python
client = agnipod.AgniPod(api_key="…", admin_token="…")

# Rotating an admin token mid-process, without rebuilding the client:
client.admin.use_token("<new jwt>")
```

The separation is the safety property: an API-token holder can run inference and
submit batches, and only an admin token can terminate instances, delete models
or disable a provider. `AGNIPOD_BASE_URL` overrides the endpoint.

---

## Inference

```python
# Single prompt
r = client.generate.create(model="qwen3:4b", prompt="…", max_tokens=1024)

# Multi-turn. History is managed for you: if the transcript would exceed the
# instance's context window, the oldest user/assistant pairs are dropped
# (system messages and the latest turn are always kept) so the request fits.
c = client.chat.create(model="qwen3:4b", messages=[
    {"role": "system", "content": "You are helpful."},
    {"role": "user", "content": "Hi"},
])
```

Sampling parameters (`temperature`, `top_p`, `top_k`, `min_p`, `repeat_penalty`,
`presence_penalty`, `frequency_penalty`, `seed`, `stop`) are ordinary keyword
arguments.

**`enable_thinking`** — reasoning models (Qwen3, DeepSeek-R1) think before
answering. `False` gives a fast, direct answer; omit it for the model's trained
default. On a short `max_tokens` budget the monologue can consume the whole
response, so it is worth setting deliberately.

**Recoverable conditions arrive as HTTP 200** with an `error` object, so a
caller can retry rather than treat a transient state as a failure:

```python
r = client.generate.create(model="qwen3:4b", prompt="…")
if r.error:
    print(r.error["type"])   # service_unavailable | context_expanding | …
```

Streaming is not offered, because the platform does not implement it. The
parameter used to exist and was silently ignored, which is worse than its
absence.

---

## Batches

```python
batch = client.batches.create(
    items=[{"custom_id": "1", "model": "qwen3:4b", "prompt": "…"}],
    batch_name="nightly-scoring",
    priority=5,                      # 1 urgent … 9 whenever
    deadline="2026-08-01 02:00:00",  # escalates priority as it nears
    not_before="2026-07-31 22:00:00" # park for an off-peak window
)

client.batches.wait(batch.batch_id, on_progress=lambda p: print(p.status))

raw = client.batches.results(batch.batch_id)          # bytes
client.batches.results(batch.batch_id, save_to="out.jsonl")   # or stream to disk
```

Also accepts `file_path=` (read locally) or `url=` (fetched server-side). Max
10,000 items inline.

**Durability.** A batch's input file *is* its work queue: as each result comes
back the item is removed from the input and appended to the output. If the GPU
serving it dies, the platform re-provisions and continues from the remaining
items — completed work is never re-run, and the delivered file has exactly one
result per `custom_id`.

---

## Administration

`client.admin` covers every endpoint the operations console uses — the same
routes, so the two can never drift.

```python
admin = client.admin

# Is anything wrong right now? One database round trip; no provider API is
# called, so this stays fast when a provider is not.
o = admin.overview()
for a in o.critical:
    print(a["message"])

# Can the platform actually serve a request? Distinct from a liveness check:
# a revoked provider credential leaves /health green while provisioning is
# impossible.
for check in admin.diagnostics().failures:
    print(check["name"], check["message"])
```

| Namespace | What it reaches |
|---|---|
| `admin.overview()` `.diagnostics()` `.metrics()` `.analytics()` `.flush_metrics()` | aggregate views |
| `admin.instances` | list, detail, metrics, compare, provider info, capabilities, directives, events, model change, destroy, stuck, sweep, blacklist |
| `admin.models` | full CRUD plus upload |
| `admin.providers` | list, configure, enable/disable, test credential, reset circuit breaker |
| `admin.benchmarks` | learned GPU profiles, facets, reset |
| `admin.requests` | request history, facets, terminations |
| `admin.events` | operational event log and summary |
| `admin.batches` | reschedule, requeue, operational detail |
| `admin.config` | effective configuration |

A few behaviours worth knowing:

```python
# Serving, billed, and excluded from routing — the state no status column
# reveals and the one most worth acting on.
for i in admin.instances.list():
    if i.stranded:
        admin.instances.destroy(i.id)

# Enabled is not the same as usable: a row with no driver, an open circuit
# breaker or an exhausted balance is enabled and still never rented from.
for p in admin.providers.list():
    if not p.usable:
        print(p.name, p.health)

# Batch traffic is sampled, so a row count is not a request count. Both are
# reported rather than one being passed off as the other.
page = admin.requests.list(success=False, window_hours=24)
print(page.total, "rows /", page.executions, "executions")

# Only what you pass is written — an absent key means "leave alone", so this
# cannot clobber a concurrent edit.
admin.models.update(7, context_length=16384, has_template=False)
```

Runnable scripts are in [`examples/`](examples/): inference, batches,
operations, and a model lifecycle walkthrough.

---

## CLI

```bash
agnipod status         # which credentials are configured
agnipod models         # what can be named as `model`
agnipod health         # fleet, spend and anomalies      (admin token)
agnipod diagnostics    # every dependency, with remedies (admin token)
```

---

## Errors

Everything derives from `AgniPodError`, so one `except` catches the lot.

| Exception | When |
|---|---|
| `AuthenticationError` | 401 — credential missing, wrong, or the wrong *kind* |
| `PermissionDeniedError` | 403 — signed in without `llm.manage` |
| `NotFoundError` | 404 |
| `ConflictError` | 409 — e.g. a model whose upload never finished |
| `RateLimitError` | 429 |
| `ServiceUnavailableError` | 503 — no worker available yet, retry |
| `ServerError` | 5xx |
| `APIConnectionError` / `APITimeoutError` | network |
| `ValidationError` | rejected client-side, never sent |
| `ConfigurationError` | missing credential or bad base URL |

Transient statuses (429, 5xx) are retried automatically with exponential
back-off; `max_retries` controls how many times.

---

## What the platform does for you

- **Provisioning & selection.** GPUs are chosen by measured throughput-per-dollar
  (memory bandwidth is the decode bottleneck), preferring cheap community-cloud
  capacity, tuned by each GPU's historical efficiency and startup reliability —
  so scheduling improves over time.
- **Context sizing.** Instances are provisioned at the model's context and each
  request uses only what it needs; the instance grows its window on demand.
- **Concurrency.** Each instance runs exactly its GPU's decode slots; interactive
  requests always keep a reserved slot so a large batch never blocks them.
- **Failure handling.** Stuck or dead instances are detected and replaced;
  in-flight batches resume; nothing is left billing.
- **Cost control.** Idle instances are reclaimed lowest-performer-first; no
  persistent storage is rented; a per-hour price ceiling is enforced.

See `ARCHITECTURE.md` in the platform repository for the internals.
