Metadata-Version: 2.4
Name: oragon
Version: 0.1.3
Summary: Local-first LLM serving for software that has to work: a resident engine, typed errors, validated extraction and an OpenAI-compatible API — on hardware you don't control, with data that can't leave.
License: Proprietary
License-File: LICENSE
Keywords: llm,local,inference,offline,openai-compatible,rag,on-premise
Author: Oragon Labs
Author-email: hello@oragonlabs.com
Requires-Python: >=3.9,<4.0
Classifier: Development Status :: 4 - Beta
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Intended Audience :: Developers
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Provides-Extra: gpu
Provides-Extra: langchain
Requires-Dist: httpx (>=0.27,<1.0)
Requires-Dist: langchain-core (>=0.2,<0.3) ; extra == "langchain"
Requires-Dist: pydantic-settings (>=2.0,<3.0)
Requires-Dist: pynacl (>=1.5) ; extra == "gpu"
Requires-Dist: requests (>=2.31,<3.0) ; extra == "langchain"
Requires-Dist: vllm (>=0.8) ; (sys_platform == "linux" and python_version >= "3.10") and (extra == "gpu")
Project-URL: Homepage, https://www.oragonlabs.com
Description-Content-Type: text/markdown

# Oragon

**Run AI models inside your own software** — on a laptop, your own server or
your own datacenter, CPU or GPU. Your data never leaves the machine.

```sh
pip install oragon
```

That is the whole install. The engine is inside the package: no compiler, no
separate download, no API key. No account either, until a computer has used
100,000 chat tokens (see [Pricing](#pricing)).

```python
from oragon import Oragon, Engine, LocalModel, user

client = Oragon(engine=Engine.EMBEDDED, model=LocalModel.QWEN2_5_0_5B)
reply = client.chat.generate(messages=[user("Name one primary color.")])
print(reply.choices[0].message.content)
```

`model=` takes any open-weight model, not just the catalog: a `.gguf` file on
disk, an `"org/repo"` name, or your own fine-tune. See
[Use your own model](#use-your-own-model).

It does speech too. The same client turns a recording into text, and the
audio never leaves the machine:

```python
transcript = client.audio.transcribe("voice-note.opus")
print(transcript.text)
```

The model loads once and stays ready, so the second request does not pay for
the first one's startup. Requests run side by side: up to four at once for free,
and more than four on a paid plan. When more arrive than can run, they wait
in a queue instead of failing.
[How that works](#many-requests-at-once).

## Why Oragon

- **It tells you when it doesn't know.** You get valid data or a clear error,
  the exact label you asked for or "no match", a cited answer or an honest
  refusal. Never a guess dressed up as an answer.
- **It fails like production software.** Every error is a typed exception that
  says whether it is safe to retry. Memory is checked before a model loads,
  and context length before a word is generated.
- **It handles many requests at once.** Requests are batched together, shared
  prompts are cached, extra requests wait in a bounded queue, and a request
  turned away is retried for you.
- **It listens, too.** Speech to text from recordings or a live microphone,
  in English, on the same engine. The audio never leaves the machine.
  [Transcribe speech](#transcribe-speech).
- **It runs anywhere.** The same code works on a laptop, a server or a GPU.
  Moving between them is a setting, not a rewrite.
- **Your data stays yours.** No telemetry, ever. The full list of what touches
  the network is [below](#your-data).

## Quick start

```sh
oragon try                # download a small model and chat with it
oragon try --tunnel       # the same chat at a public HTTPS link — open it on your phone
oragon try extract        # turn text into validated JSON
oragon try rag            # ask questions about your documents
oragon serve              # run an OpenAI-compatible API, with a browser playground
oragon doctor             # check what this machine can run
```

The chat runs against a live OpenAI-compatible API the whole time, so any
OpenAI client in any language can join in while it is up.

The first run downloads the model. Disk space is checked first, progress is
shown, and the file is verified. Every later run finds it on disk. To fetch
ahead of time (CI images, air-gapped machines), run
`oragon download qwen2.5-0.5b`, and `oragon download speech-english-v2` for
speech to text. Set `ORAGON_AUTO_DOWNLOAD=0` to stop the SDK
from ever going online; a missing model then becomes a clear error that names
the download command.

If anything goes wrong, `oragon doctor` reports the platform, memory, engine,
models and settings in one paste. `ORAGON_DEBUG=1` turns on detailed logging.

## The browser playground

`oragon serve` also serves a playground page from the engine itself. Open
`http://127.0.0.1:8080/` in a browser and you get:

- **a chat pane** to talk to the model,
- **a structured-output demo** that turns text into validated JSON,
- **a live console** showing what the engine is doing right now: tokens per
  second, lanes in use, requests waiting, requests answered, time to first
  word, how much of each prompt came from cache, and how full the cache and
  GPU memory are.

The page uses nothing from the internet, so it works the same on an
air-gapped machine. When the server needs a token, the link `oragon serve`
prints already carries it. With `--tunnel`, it also prints a public playground
link you can send to someone.

## Many requests at once

Oragon is built to serve traffic, not one chat window. This is what happens
when requests pile up, in order:

1. **Several run at the same time.** The engine has *lanes*, and each lane
   generates one request at a time. **Up to 4 lanes are free**, in every
   install. **More than 4 needs a paid plan.** Requests in different lanes are
   batched into the same passes over the model, so four requests cost far
   less than four times one. A long prompt is read in chunks, so it cannot
   freeze the others.
2. **Shared prompts are computed once.** Requests that start with the same
   text, such as a shared system prompt, share its cache across lanes. In one
   test, eight agents sharing an instruction went from 11 s to 2.2 s.
3. **The rest wait in a queue.** When every lane is busy, a new request waits
   and starts as soon as a lane frees up. Nothing is dropped, and nothing is
   cut off partway through an answer.
4. **The queue has limits, so it cannot sink the machine.** By default it
   holds 256 requests per lane, capped by a memory budget sized from free RAM.
   It also has a time limit: a request that would wait more than 60 seconds is
   turned away when it arrives, instead of being answered a minute late.
5. **A request that is turned away is never half-done.** It gets an immediate
   `overloaded` answer and nothing was generated, so sending it again is safe.
6. **The SDK retries it for you.** A turned-away request is retried
   automatically, up to three times, with short randomized pauses (about two
   seconds in total) so a thousand callers do not all retry at once. Streams
   are retried too: a refusal always comes before the first word, so a retry
   never repeats output. You only see `EngineBusyError` when the overload
   lasts. Before that, an `EngineBusyWarning` on requests that *succeeded*
   tells you pressure is building while you can still act on it.
7. **Over HTTP, it is a `429`.** The API returns the status OpenAI clients
   already understand, so the official OpenAI libraries retry it on their
   own. A server that is shutting down returns `503`, finishes what is
   running, and then exits.

On a GPU, the same rules cover memory. New work is turned away before the card
fills, while running requests finish, so there is no out-of-memory crash to
recover from. A request whose prompt and answer would not fit the cache waits
for room instead of being started and pushed out halfway.

You can change all of it. On the free tier, `slots` goes up to 4: asking
for more runs with 4 and says so. It is not an error, and nothing else
changes. A paid plan raises the limit.

```python
client = Oragon(
    engine=Engine.EMBEDDED,
    model=LocalModel.QWEN2_5_1_5B,
    slots=4,             # lanes: 1-4 free; more than 4 needs a paid plan
    max_queue=512,       # how many may wait for a lane
    max_queue_wait=10,   # seconds a caller will wait; 0 = always wait, never turn away
)

client.stats()   # lanes busy, requests waiting, queue limits, cache reuse
client.ping()    # health check; answers between words, even when busy
```

`AsyncOragon` has the same behaviour: `asyncio.gather` over many calls
really does overlap, and the engine batches them. Over HTTP, the same
figures are at `/metrics` (Prometheus) and `/v1/status` (JSON).

Measured on a 16 GB laptop with no GPU: 16 requests at once, each streaming
at 12+ tokens/s with its first word in about a second. In a flood of 6,000
requests arriving at the same instant, every one was either answered or told
`overloaded` right away. None were left hanging.

## What you can do

Everything in this section is free.

### Three engines, one API

```python
Oragon(engine=Engine.EMBEDDED, model=...)   # loads the model once, for this client
Oragon(engine=Engine.SHARED,   model=...)   # loads the model once, for the whole machine (paid)
Oragon(engine=Engine.HOSTED,   model=...)   # the Oragon API (not available yet)
```

All three have the same `generate()`, the same response type and the same
errors, so where the model runs is a deployment choice, not a rewrite.

### Get structured data

Pass a schema. You get data that matches it, or an `ExtractionError` —
nothing in between.

```python
rows = client.chat.generate(
    model=LocalModel.QWEN2_5_1_5B,
    messages=[user(invoice_text)],
    response_schema={"type": "array", "items": {
        "type": "object",
        "required": ["date", "amount"],
        "properties": {"date": {"type": "string"}, "amount": {"type": "number"}},
    }},
)
```

The schema is enforced while the model writes, so output that would break it
cannot be produced.

### Make a decision

You get back one of the labels you offered, or `matched=False`. It is never
stretched to fit the closest-looking label.

```python
decision = client.chat.generate(
    model=LocalModel.QWEN2_5_1_5B,
    messages=[user("Is 'J. Smith' the same person as 'John Smith'?")],
    response_format={"type": "choice", "choices": ["SAME", "DIFFERENT"]},
)
print(decision.choice, decision.matched)
```

For harder calls, `Settings(thinking=True)` shows the model its own answer and
asks it to check. Its first answer is the floor, so this cannot do worse than
leaving it off:

```python
decision = client.chat.generate(
    model=LocalModel.QWEN2_5_1_5B,
    messages=[user("Meeting started 9:45, ran 100 min — over before noon?")],
    response_format={"type": "choice", "choices": ["YES", "NO"]},
    settings=Settings(temperature=0.0, thinking=True),
)
print(decision.reason)   # the reasoning, for your audit log
print(decision.choice)   # the verdict, for your code
```

### Use tools

Give it plain Python functions and `act()` runs the whole loop for you. The
tool's description is read from its signature, type hints and docstring.

```python
def get_weather(city: str) -> dict:
    """Current weather for a city."""
    return {"city": city, "temp_c": 22}

reply = client.chat.act(
    model=LocalModel.QWEN2_5_1_5B,
    messages="What's the weather in Lagos?",
    tools=[get_weather],
)
print(reply.choices[0].message.content)
```

- If a tool fails, the model sees the error and tries again.
- Pass `messages` as a list and the conversation is kept, so you can answer a
  follow-up question and carry on.
- Several tool calls in one round run in parallel.
- `on_tool` and `on_round` let you watch what it does.

Prefer to run the loop yourself? `generate(tools=...)` uses the same format as
the OpenAI client.

### Ask questions about your documents

```python
from oragon.rag import DocumentQA, EngineRetriever, ingest

retriever = EngineRetriever()
ingest(["handbook.md", "policy.pdf"], retriever)

qa = DocumentQA(client, model=LocalModel.QWEN2_5_1_5B, retriever=retriever)
result = qa.ask("How many vacation days do new hires get?")

if result.refused:
    print("The documents don't say.")
else:
    print(result.text)               # with citations like [1]
    for hit in result.sources:       # the exact place in each file
        print(hit.chunk.source, hit.chunk.start, hit.chunk.end)
```

It searches by words and by meaning, entirely on your machine. A small
embedding model runs inside the same engine, so there is no second service to
run. If nothing relevant is found, it refuses instead of guessing, and a
citation the model made up is flagged rather than quietly fixed.

### Transcribe speech

```python
from oragon import SpeechModel

transcript = client.audio.transcribe("voice-note.opus", model=SpeechModel.ENGLISH_V2)
print(transcript.text)

with client.audio.live() as session:      # audio as it is captured
    for chunk in microphone():
        session.send(chunk)
        for event in session.poll():
            if event.type == "final":
                print(event.text)
```

English, written down as spoken. Recordings in WAV, MP3, M4A, FLAC, Ogg or
Opus (a WhatsApp voice note), or raw audio; live sessions give partial captions, an early transcript
the moment the speaker pauses, and each final in the order spoken. The audio
never leaves the machine.

- **No chat model needed.** `Oragon(engine="embedded")` with no `model` loads
  only the speech model, which is downloaded the first time you use it.
- **Voice agents.** `session.pause()` and `session.resume()` stop listening
  while your agent speaks. `AsyncOragon` has the same calls, with `await`.
- **Timed segments.** Every transcript has `segments` with start and end
  times, for captions and for finding a moment in a recording.
- **Its own lanes.** Speech runs beside chat, not in its queue: up to 4 lanes
  free; more than 4 needs a paid plan. 10 hours of speech are free on a
  computer without an account; your connected computers then share 10 hours a
  month.
  Only speech is counted, never silence, and every result says how much
  (`speech_seconds`). See [Pricing](#pricing).
- **Over HTTP too.** Once the speech model is on disk
  (`oragon download speech-english-v2`), `oragon serve` answers OpenAI's
  `/v1/audio/transcriptions`, and holds live sessions over a WebSocket at
  `/v1/audio/live`.

### Stream and run async

```python
for chunk in client.chat.generate(model=..., messages=[...], stream=True):
    if isinstance(chunk, str):
        print(chunk, end="", flush=True)
```

```python
from oragon import AsyncOragon

client = AsyncOragon(engine=Engine.EMBEDDED, model=LocalModel.QWEN2_5_1_5B)
reply = await client.chat.generate(model=..., messages=[...])
```

### Long conversations

A conversation too long for the model raises `ContextOverflowError` before
anything is generated. It is never silently cut. Pass `overflow="slide"` to
drop the oldest turns instead (the system prompt is kept):

```python
client.chat.generate(model=..., messages=history, overflow="slide")
```

### Fast without tuning

- **Tuned to your machine.** On first run the engine measures this machine
  and sizes its own work to match. The measurement is saved, so it happens
  once.
- **Prompt reuse.** A shared system prompt or a growing conversation is read
  from cache rather than recomputed. Watch `reply.usage.cached_tokens` climb.
- **Warm start.** `Oragon(warm_prompt=SYSTEM_PROMPT)` reads a long standing
  instruction at startup, so even the first request is fast: 10 s down to
  0.5 s in our test.
- **Conversations that survive a restart.** Pass `session="..."` and a
  conversation's computed state is kept on disk. The next request only
  processes what is new. The answer is the same with or without it.
- **Repeated questions answered instantly.** `oragon serve --cache auto` (or
  `Oragon(answer_cache="auto")`) answers a request it has already answered,
  word for word, without generating it again: under a millisecond instead of
  seconds. Only exact matches are served. `--cache measure` reports how much
  it would save on your traffic before you turn it on. Off by default.
- **Speculative decoding.** `draft_model=` lets a small model speed up a
  large one, up to 1.56× on a 3B model. It checks whether this helps on your
  machine and switches itself off when it would not.

`oragon doctor` tells you what this machine will do before you choose a model:
its measured memory speed, the generation speed to expect from each model, and
how much context each one has room for.

### Use your own model

```sh
oragon import ./my-finetune/       # a training checkpoint folder
oragon import my-model.gguf        # or a model file
```

It is converted and optimized for you, and your original file is never
changed. If it cannot be converted correctly, it is refused with the reason.

### Serve it over HTTP

```sh
oragon serve                               # this machine only: 127.0.0.1:8080
oragon serve --expose                      # other machines on your network
oragon serve --tunnel                      # anyone: a public HTTPS link
```

`--expose` and `--tunnel` create an access token for the run unless you pass
`--auth-token`. Any OpenAI client, in any language, can now use your model:

```python
from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8080/v1", api_key=TOKEN)
client.chat.completions.create(model="qwen2.5-1.5b", messages=[...])
```

| Route | What it is |
|---|---|
| `/` | the [browser playground](#the-browser-playground) |
| `/v1/chat/completions` | chat, with streaming, tools and `response_format` |
| `/v1/embeddings` | vectors, for a vector database you already run |
| `/v1/audio/transcriptions` | speech to text from a recording, as OpenAI's audio API takes it; `srt` and `vtt` captions too |
| `/v1/audio/live` | a live speech session over a WebSocket: audio in, transcripts out as the speaker talks |
| `/v1/models` | what is loaded |
| `/health` | liveness |
| `/v1/status`, `/metrics` | load, queue and cache figures (behind the token, when one is set) |

When the engine cannot do something, it says so instead of improvising.
Asking for a model it has not loaded is a `404`, not a quiet swap. Binding to
a network address without a token is refused at startup.

### Handle errors

All errors share a base class, `ApiError`, and say whether a retry is safe.

```python
from oragon import (
    ApiError, ContextOverflowError, EngineBusyError, EngineDiedError,
    EngineTimeoutError, ExtractionError, InsufficientMemoryError,
    AudioError, SpeechUnavailableError, SpeechAllowanceError,
    CheckinRequiredError, LiveSessionEndedError,
)
```

| Error | What it means |
|---|---|
| `EngineBusyError` | Still overloaded after the automatic retries. Nothing was started; retry later |
| `InsufficientMemoryError` | The model will not fit. Caught before loading, not a crash |
| `ContextOverflowError` | The conversation is too long; see `overflow="slide"` |
| `ExtractionError` | The output still did not match your schema after one corrected retry |
| `EngineDiedError` | The engine stopped mid-request; the next call starts a new one |
| `EngineTimeoutError` | The request took longer than your timeout |
| `SignupRequiredError` | This computer has used 100,000 chat tokens; connect it to a free account |
| `AudioError` | The audio could not be read, or the recording is too long |
| `SpeechUnavailableError` | No speech model on this computer and downloads are off; run `oragon download speech-english-v2` |
| `SpeechAllowanceError` | This month's free speech is used; says when it starts again |
| `CheckinRequiredError` | Free chat or speech needs its check-in and the computer is offline; connect and retry |
| `LiveSessionEndedError` | A live session closed after going quiet or running too long; open a new one |

### See what your app did — locally

Every request is summarized into a store on this machine: timings, token
counts, models, error codes, tool names. Prompts and answers are never
stored, and nothing is ever uploaded.

```sh
oragon stats             # last 7 days: latency, error rates, tool calls
oragon stats --json
oragon obs clear         # delete the store; always safe
```

`ORAGON_OBSERVABILITY=0` turns recording off entirely.

### More

- **Reproducible answers** — `Oragon(reproducible=True)`: an answer never
  depends on what was asked before it.
- **Audit export** — `oragon audit export --format jsonl` (or `cef`) for your
  audit system.
- **See a model's chat template** — `oragon template qwen2.5-1.5b`.
- **Free tier status** — `oragon status`: chat tokens counted and free speech
  left on this computer, read offline.
- **LangChain** — `pip install 'oragon[langchain]'`, then
  `from oragon.llm import OragonChat`.

## Servers and GPUs

The same engine runs on a server or in a datacenter.

```sh
pip install "oragon[gpu]"          # Linux, Python 3.10+, an NVIDIA driver
oragon serve Qwen/Qwen2.5-1.5B-Instruct --gpu auto --expose
```

The engine starts a GPU model server, waits for it to load, and serves the
API in front of it. Typed errors, the queue, licensing and document search all
stay the same.

- **Tuned to the card.** `--gpu-profile auto` (the default) reads the card's
  memory and sets itself up. `small`, `balanced` and `throughput` choose
  explicitly.
- **It does not thrash.** If the GPU starts running out of room, the engine
  admits fewer requests until it recovers, instead of slowing everyone down.
- **It reserves before it starts.** A request is only started when its prompt
  and answer fit. Otherwise it waits for a running one to finish.
- **It restarts itself.** If the model server crashes, it is replaced, and
  queued requests wait for it.

**On CPU servers**, `--kv-pool auto` sizes one shared cache from the free
memory. Requests that share a prompt store it once, so more lanes fit in the
same memory, and requests that do not fit wait instead of failing. None of
this changes an answer.

**Running several machines?** Each response carries an `X-Oragon-Affinity`
header. Route on it, and requests with the same system prompt land on the
machine that already has it cached. Several machines then behave like one
deployment instead of several cold starts.

No GPU is required. CPU is a first-class path, not a fallback. If a GPU is
there it is used automatically; pass `cpu=True` to force CPU and compare.

## Benchmarks

Measured, not estimated. CPU figures come from a 16 GB MacBook (Apple M4)
running `qwen2.5-1.5b` — the smallest machine Oragon is built for. Servers do
better.

### Oragon on its own

| Test | Result |
|---|---|
| 16 requests at once, CPU only | each gets 12+ tokens/s, first word in 1 s |
| 16 requests at once, one Tesla T4 GPU | 224 tokens/s total, slowest request still 59.9 tokens/s |
| 6,000 requests at the same instant | every one answered or told "busy" — none left hanging |
| 600,000 requests in three floods, never restarted | 0 lost, 0 unexplained errors, memory flat, health check in 0.02 ms |
| First request with a long system prompt | 10 s → 0.5 s when warmed at startup |
| 8 agents sharing one prompt | 11 s → 2.2 s with caching |
| The same question asked again, answer cache on (0.5B model) | 0.5 ms instead of 1.3 s |
| Document search over 10,000 passages | 7 ms per search; 150 of 150 citations pointed at the exact text |
| Same model, with Oragon's checks around it | 7/18 → 11/18 correct on graded tasks |
| Invoice extraction on documents that aren't invoices | 5/7 handled correctly, against 2/7 for the model alone |
| Startup, small model | first answer 0.8 s after launch |

### Oragon and Ollama, side by side

Same model file, same machine, one running at a time (Ollama 0.33.3).

| Test | Oragon | Ollama |
|---|---|---|
| 1,000 different requests, CPU, both at 16 parallel | **done in 810 s** | 1,244 s — **Oragon 1.54× faster** |
| 6,000 requests at the same instant, one CPU | **none left hanging**, 17 answers/s | 4,757 got no answer, 8 answers/s |
| 16 requests at once, one Tesla T4 GPU | **224 tokens/s**, slowest 60 tokens/s | 140 tokens/s, slowest 25 tokens/s |
| First read of a long instruction, CPU | **10.9 s** | 23.8 s — **2.2× faster** |
| 16 requests at once, CPU, default settings | **9.3 s** | 20.1 s |
| 16 requests at once, Apple M4, default settings | **10.9 s** | 15.7 s |
| Startup to first word | **0.96 s** | 1.61 s |
| Invoice extraction, correct outcome | **5/7** | 2/7 |
| One request, generation speed | 92 tokens/s | 94 tokens/s — a tie |
| Repeated long prompt (caching) | 0.17 s | 0.18 s — a tie |
| Accuracy on the same model | 14/36 | 14/36 — a tie |
| One request on the T4, time to first word | 228 ms | **24 ms** — Ollama ahead |

Ollama answers one request at a time by default; with its parallel setting
raised it matches us on the Apple M4. The 1,000-request, 6,000-request, T4 and
long-instruction results hold with Ollama tuned. Ollama spent about 14% more
work per request on the 1,000-request run, so not all of that gap is engine.

Speed per request is set by your hardware, and answer quality by the model —
what Oragon adds is everything around it.

## How it compares

| | Oragon | Ollama |
|---|---|---|
| Install | `pip install oragon` | a separate installer, then a client library |
| Runs on | laptop or server, CPU or GPU | laptop or server, CPU or GPU |
| Many requests at once | up to 4 lanes free (more on a paid plan), a bounded queue, automatic retry | one at a time by default |
| When it is full | answers `429` at once; the SDK retries for you | requests can wait with no answer |
| Asking for JSON | valid data, or a clear error | text you check yourself |
| Asking for a category | the exact label, or an honest "no match" | text you have to interpret |
| Questions over your documents | included, with citations, and it says when it doesn't know | build it yourself |
| When something breaks | typed errors that say whether to retry | a status code and a message |
| Model too big for the machine | you are told before it loads | it fails partway through loading |
| Your data | stays on the machine | stays on the machine |

Ollama is a good way to try models. Oragon is built for the next step:
software you hand to someone else, running on a machine you do not control,
where an odd answer from the model has to be something your code can catch.

## Platforms

Python 3.9 to 3.13.

| Platform | Supported | Speeds up with |
|---|---|---|
| macOS 11+, Apple silicon | ✅ | Apple GPU |
| Linux x86-64 (Ubuntu 24.04+, Debian 13+, Fedora 40+) | ✅ | CUDA, Vulkan (NVIDIA, AMD, Intel), or CPU |
| Linux arm64 (same versions) | ✅ | CPU |
| RHEL 9, Ubuntu 22.04, Debian 12 | coming soon | |
| Windows, Intel Macs | coming soon | |

Every feature, chat and speech, works the same on every platform; only the
speed-up differs. The speed-up is for chat: speech to text runs on the CPU
everywhere, so it needs no GPU.
No GPU needed. If one is there, it is used automatically. One install covers
every machine: there is no separate GPU package to choose. On an unsupported
platform, pip installs the older `0.0.1` placeholder instead — if you see
version 0.0.1, that is why.

## Pricing

Trying and building with Oragon are always free. Serving real users or
customers, including in a pilot or beta, is production: the free tier covers
it, commercial use included, for anyone with up to US$200K a year in revenue,
in money raised and, for public bodies and non-profits, in budget, with no
limit on users or servers. Past any of these, move to Starter (see the
LICENSE file in the package).

Chat needs no account for a computer's first 100,000 tokens; after that, a
free account connected to that computer keeps chat going, unlimited. Make
one at [sdk-app.oragonlabs.com](https://sdk-app.oragonlabs.com/) (or run
`python -m oragon signup`, which opens it), then run the
`python -m oragon activate <key>` command it gives you. On servers,
containers and fleets, set `ORAGON_LICENSE_KEY` to your key once in the
deployment: every machine that runs with it is connected, with nothing to run
per machine and no limit on how many.

| | Free | Paid |
|---|---|---|
| Up to 4 lanes (requests generating at once), plus the queue | ✅ | ✅ |
| Chat: 100,000 tokens per computer with no account, then unlimited with a free account | ✅ | ✅ |
| Every feature on this page, including the HTTP API and GPU serving | ✅ | ✅ |
| Many processes sharing one model (`Engine.SHARED`) | — | ✅ |
| More than 4 lanes | — | ✅ |
| Speech to text: up to 4 lanes; 10 hours per computer without an account, then 10 hours a month per account | ✅ | ✅ |
| More speech than that | — | ✅ |
| Oragon's own licensed models, encrypted on disk | — | ✅ |
| Production use above US$200K a year in revenue, funding or budget | — | ✅ |

If four at a time in one process is enough and you are within the free
limits, you never need to pay. That is
intended, not a loophole. Every computer that uses more than 100,000 chat
tokens needs a free account connected to it, including the computers of
people you ship an app to; `SignupRequiredError` says so, and the SDK warns
from about 80,000. Free chat and speech check in about once a week and work
offline between check-ins: about 10 days, or 37 once connected to a free
account. A container with no machine ID needs `ORAGON_LICENSE_KEY` from the
start. A paid licence never checks in for chat.

## Your data

The complete list of what crosses the network, so you can check it rather
than trust it:

- **Your prompts and answers** never leave the machine. There is no code path
  that could send them, and they are never used to train anyone's model.
- **No crash reports and no update checks.**
- **The free tier's check-in** — about once a week, in the background, and
  never at startup or in the way of a request. It sends a device ID (a one-way
  hash, not the machine's own identifier) and usage counts: chat requests and
  tokens, minutes of speech and the number of words transcribed — never the
  words themselves. It also says what kind of machine this is, so a server can be told from a
  laptop: whether it runs in a container, on a named cloud or a virtual
  machine, whether it has a battery, its processor cores, how many hours of
  the last week had chat, the most requests at once, and whether it served
  other machines. Never a hostname, an address or a serial number. Free chat
  and speech run on what this check-in collects, so they need the internet
  about once a week: they work offline for about 10 days, or 37 once connected
  to a free account. A paid licence does not check in for this.
- **Model downloads** — only the model you asked for, only incoming.
  `ORAGON_AUTO_DOWNLOAD=0` turns even that off. That includes the speech
  model; your audio is never sent anywhere.
- **Licence check-in** — only on a machine with an activated licence. It sends
  the licence key, usage counts (token totals, model names, timings — never
  content) and a device ID. The device ID is a short hash, not the machine's
  own identifier, and `ORAGON_DEVICE_ID` lets you set it yourself. No licence,
  no check-in.

Licences work offline for their whole validity period, so air-gapped
deployments are fully supported.

## Not yet supported

- Images and video, and text to speech. Speech to text is supported, in
  English.
- The `hosted` engine. Asking for it raises an error rather than pretending.
  Code written today will move to it by changing one word.
- Some platforms — see [Platforms](#platforms).

## Third-party licences

Oragon includes third-party material. Its notices are in
`THIRD-PARTY-NOTICES-speech.md`, in the installed package's `oragon/_bin/`
folder.

## Who builds this

Oragon Labs — an AI research lab building AI that runs where people and their
data already are: on their own machines, in their own country, at a price
local economies can sustain. No GPU required, no data leaving your control,
no single company or server as a point of failure.

## Learn more

- Documentation: <https://www.oragonlabs.com>
- Contact: [hello@oragonlabs.com](mailto:hello@oragonlabs.com)

Already have an Oragon account? The same package is on your private index, so
you can install it from your dashboard.

