Metadata-Version: 2.4
Name: llmlatency
Version: 0.1.0
Summary: Measured latency and uptime for AI inference APIs — client for the open llmlatency.dev dataset
Author-email: "llmlatency.dev" <bot@mail.llmlatency.dev>
License: MIT
Project-URL: Homepage, https://llmlatency.dev
Project-URL: Methodology, https://llmlatency.dev/methodology
Project-URL: Machine-readable API, https://llmlatency.dev/api/rankings.json
Project-URL: Source, https://github.com/mazamaka/llm-latency-tracker
Project-URL: Citable dataset (DOI), https://doi.org/10.5281/zenodo.21954788
Keywords: llm,latency,benchmark,ai,inference,observability,time-to-first-token,api-monitoring
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: System :: Monitoring
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# llmlatency

Measured latency and uptime for AI inference APIs, as a small Python client for
the open dataset published at **[llmlatency.dev](https://llmlatency.dev)**.

Probes run every five minutes from four regions and are never routed through a
gateway or an aggregator, so the numbers describe the providers themselves
rather than a proxy in front of them.

- **Network probe** — DNS → TCP → TLS → time to first byte.
- **Inference probe** — time to first token on a real completion request.

These are different quantities, an order of magnitude apart, and this library
never mixes them in one ranking.

## Install

```bash
pip install llmlatency
```

No dependencies — standard library only.

## Use it from Python

```python
import llmlatency

llmlatency.regions()
# ['ap-tokyo', 'eu-hetzner', 'sa-east', 'us-central']

llmlatency.fastest("eu-hetzner")
# {'rank': 1, 'provider': 'nscale', 'p50_ms': 99, 'p95_ms': 203,
#  'uptime_pct': 100, 'samples': 289}

for row in llmlatency.ranking("us-central")[:3]:
    print(row["rank"], row["provider"], row["p50_ms"], "ms")

llmlatency.provider("anthropic")          # one provider across every region
llmlatency.generated_at()                 # snapshot timestamp, ISO-8601 UTC
```

Fetch once and pass the snapshot around if you make several queries:

```python
data = llmlatency.fetch()
fast = {r: llmlatency.fastest(r, data=data) for r in llmlatency.regions(data)}
```

## Use it from the shell

```bash
llmlatency fastest                        # fastest provider in every region
llmlatency ranking --region eu-hetzner --top 5
llmlatency provider openai
llmlatency fastest --probe inference --json
```

## Data, methodology, licence

- Live rankings and full methodology: <https://llmlatency.dev>
- Machine-readable snapshot: <https://llmlatency.dev/api/rankings.json>
- Citable archive with a DOI: <https://doi.org/10.5281/zenodo.21954788>
- Daily snapshots in git: <https://github.com/mazamaka/llm-latency-tracker>

The dataset is **CC-BY-4.0**; this client library is MIT.

**Limitations, stated up front.** Vantage points are cloud data centres, not
consumer networks, so absolute values are lower than an end user would see — the
comparison between providers is the meaningful part. Provider coverage changes
over time as APIs appear and shut down.
