Metadata-Version: 2.5
Name: ekte
Version: 0.33.1
Summary: LLM relay reality check: audit whether an API relay actually serves the model it claims, from the command line, a local web UI, an HTTP API or CI.
Project-URL: Homepage, https://github.com/mn567250/ekte
Project-URL: Repository, https://github.com/mn567250/ekte
Project-URL: Issues, https://github.com/mn567250/ekte/issues
Author: ekte contributors
License-Expression: Apache-2.0
License-File: LICENSE
License-File: NOTICE
Keywords: api,audit,context-window,llm,model-fingerprinting,openai,proxy,relay
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Quality Assurance
Requires-Python: >=3.11
Requires-Dist: httpx>=0.27
Requires-Dist: pydantic>=2.6
Requires-Dist: pyyaml>=6.0
Requires-Dist: reportlab>=4.0
Requires-Dist: rich>=13.7
Requires-Dist: typer>=0.12
Requires-Dist: tzdata>=2024.1; sys_platform == 'win32'
Provides-Extra: a11y
Requires-Dist: playwright>=1.47; extra == 'a11y'
Provides-Extra: dev
Requires-Dist: httpx>=0.27; extra == 'dev'
Requires-Dist: jsonschema>=4.20; extra == 'dev'
Requires-Dist: mypy>=1.10; extra == 'dev'
Requires-Dist: pypdf>=4.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-xdist>=3.5; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Provides-Extra: pdf
Provides-Extra: tokenizers
Provides-Extra: web
Requires-Dist: cryptography>=42; extra == 'web'
Requires-Dist: fastapi>=0.110; extra == 'web'
Requires-Dist: uvicorn>=0.29; extra == 'web'
Description-Content-Type: text/markdown

# ekte — LLM relay reality check

> **🇬🇧 English** · [🇨🇳 中文](https://github.com/mn567250/ekte/blob/v0.33.1/docs/i18n/zh-CN/README.md) · [🇫🇷 Français](https://github.com/mn567250/ekte/blob/v0.33.1/docs/i18n/fr/README.md) · [🇪🇸 Español](https://github.com/mn567250/ekte/blob/v0.33.1/docs/i18n/es/README.md) · [🇵🇹 Português](https://github.com/mn567250/ekte/blob/v0.33.1/docs/i18n/pt/README.md) · [🇮🇹 Italiano](https://github.com/mn567250/ekte/blob/v0.33.1/docs/i18n/it/README.md) · [🇩🇪 Deutsch](https://github.com/mn567250/ekte/blob/v0.33.1/docs/i18n/de/README.md) · [🇳🇴 Norsk](https://github.com/mn567250/ekte/blob/v0.33.1/docs/i18n/nb/README.md)

[![CI](https://github.com/mn567250/ekte/actions/workflows/ci.yml/badge.svg)](https://github.com/mn567250/ekte/actions/workflows/ci.yml)
[![License: Apache-2.0](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://github.com/mn567250/ekte/blob/v0.33.1/LICENSE)
[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://github.com/mn567250/ekte/blob/v0.33.1/pyproject.toml)

**ekte** is a local-first tool that audits whether an API relay (reseller /
proxy) actually serves the model it claims to — or quietly substitutes a cheaper
one, truncates your context window, fakes streaming, or inflates token billing.
In short: do you get what you paid for? It speaks the **OpenAI Chat Completions
API**, the **Anthropic Messages API** and the **OpenAI Responses API**
(`/v1/responses`) — auto-detected, or forced with `--api openai|anthropic|responses`.

You point it at a relay endpoint and the model it claims to serve; ekte runs a
battery of black-box probes, compares what it observes with a bundled knowledge
base of **130 model profiles from 7 providers**, and gives a clear, evidence-backed
verdict — on the command line, in a local web UI, or as JSON for another tool or
LLM.

> ekte reports **black-box evidence of divergence and risk, not cryptographic
> proof of fraud.** See [Responsible use](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/privacy.md#responsible-use).

## Why

The relay-key market is full of "GPT-4o for 1/10th the price" offers. Many are
honest. Some are not — and the dishonest ones are hard to catch by eye:

- You ask for `gpt-4o`; you're quietly served `gpt-4o-mini` or an open model.
- The relay advertises a 1M-token context but silently truncates to 32K.
- "Streaming" is the full response buffered and re-chunked, with no latency win.
- Reported `usage` tokens are inflated, so your balance burns faster than it should.
- A model that should support tool calling / JSON mode quietly doesn't.

ekte turns "this feels off" into a reproducible report.

## Install

Requires Python 3.11+ on Linux, macOS or Windows.

```bash
pip install 'ekte[web]'          # or: uv tool install 'ekte[web]'
```

The `web` extra adds the local web UI; leave it out for the command line only.
From source, with uv or in Docker: [Install](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/install.md).

## Quick start

```bash
export EKTE_API_KEY=sk-your-relay-key

# audit a relay against the model it claims to serve
ekte check --base-url https://relay.example.com/v1 --api-key env:EKTE_API_KEY \
  --model gpt-4o --suite standard

# the strongest check: compare against a trusted baseline of the same model
export OPENAI_API_KEY=sk-your-openai-key
ekte compare \
  --target-base-url https://relay.example.com/v1 --target-api-key env:EKTE_API_KEY --target-model gpt-4o \
  --baseline-base-url https://api.openai.com/v1 --baseline-api-key env:OPENAI_API_KEY --baseline-model gpt-4o

# or point and click: the local web UI
ekte serve
```

More examples, config files and use from scripts or LLM agents:
[Quick start](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/quickstart.md). The web UI: [Web UI](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/web-ui.md).

## What it checks

ekte scores ten dimensions. The four **core dimensions** — model identity,
context window, capability claims and billing & usage — most directly reveal a
bait-and-switch. The names are the ones the web UI and reports use.

| Dimension | Id | What it catches |
|---|---|---|
| **Model identity** | `model_identity` | Silent model downgrade or substitution |
| **Context window** | `context_window` | Silent context truncation and lost-in-the-middle |
| **Capability claims** | `capability` | Tool calling, JSON mode, max output or vision that isn't delivered |
| **Protocol compliance** | `protocol` | Request parameters and response attributes that are dropped or ignored |
| **Billing & usage** | `billing` | Inflated or missing token usage |
| **Connectivity** | `connectivity` | Endpoint reachability and the advertised model list |
| **Streaming authenticity** | `streaming` | Fake streaming (buffer-then-chunk) |
| **Concurrency reliability** | `reliability` | Failures and slowdowns under concurrent load |
| **Transport security** | `security` | Plain HTTP, secret echo, injected system prompts, tampering |
| **Performance** | `performance` | How consistent latency, time to first token and throughput are |

How every check works and how the verdict is reached:
[Methodology](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/methodology.md).

## Documentation

| User guide | |
|---|---|
| [Install](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/install.md) | pip, uv, from source, optional extras |
| [Quick start](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/quickstart.md) | First audits, config files, use by scripts and LLM agents |
| [Web UI](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/web-ui.md) | Pages, background audits, analytics, languages |
| [Suites and compare mode](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/suites.md) | Suites, custom suite, trusted baseline, LLM judge |
| [Performance](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/performance.md) | The performance section, the probe and request timeouts |
| [Monitoring](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/monitoring.md) | Scheduled re-audits, webhook alerts, API key storage |
| [Embedding, rerank, image and audio](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/other-endpoints.md) | Audits of endpoints that are not chat |
| [Knowledge base](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/knowledge-base.md) | Model profiles and adding your own |
| [Reports](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/reports.md) | Report formats and contents |
| [Privacy and local data](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/privacy.md) | What stays on your machine; responsible use |
| [GitHub Action](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/github-action.md) | Gate a CI workflow on a relay audit |
| [Docker](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/docker.md) | Run the web UI in a container |
| [Methodology](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/methodology.md) | Every check, its scoring scale and its caveats |
| [Coming from zing](https://github.com/mn567250/ekte/blob/v0.33.1/docs/guide/migrating-from-zing.md) | What changes for an existing zing install |

| Reference (English) | |
|---|---|
| [Command line](https://github.com/mn567250/ekte/blob/v0.33.1/docs/reference/cli.md) | Every command and option, and the exit codes |
| [Report and webhook schemas](https://github.com/mn567250/ekte/blob/v0.33.1/docs/reference/schemas.md) | The JSON formats and their `schema_version` |
| [Stability and compatibility](https://github.com/mn567250/ekte/blob/v0.33.1/docs/reference/stability.md) | What semver protects, deprecation, supported Python and systems |
| [Changelog](https://github.com/mn567250/ekte/blob/v0.33.1/CHANGELOG.md) | What changed in each release |

Contributing: [Contributing guide](https://github.com/mn567250/ekte/blob/v0.33.1/docs/CONTRIBUTING.md) and
[Developer guide](https://github.com/mn567250/ekte/blob/v0.33.1/docs/dev/developer-guide.md) (English).
Security issues: [Security policy](https://github.com/mn567250/ekte/blob/v0.33.1/docs/SECURITY.md).

## License

[Apache-2.0](https://github.com/mn567250/ekte/blob/v0.33.1/LICENSE)
