Metadata-Version: 2.4
Name: outo-llms
Version: 0.1.0
Summary: Deploy local LLMs behind your own managed API server - vLLM or llama.cpp in isolated environments, with API keys, workspaces, and usage tracking.
Project-URL: Homepage, https://github.com/llaa33219/outo-llms
Project-URL: Repository, https://github.com/llaa33219/outo-llms
Project-URL: Issues, https://github.com/llaa33219/outo-llms/issues
Author: outo-llms contributors
License: Apache-2.0
License-File: LICENSE
Keywords: api-server,inference,llama.cpp,llm,self-hosted,vllm
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: cryptography>=42
Requires-Dist: fastapi>=0.110
Requires-Dist: httpx>=0.27
Requires-Dist: platformdirs>=4
Requires-Dist: pydantic>=2
Requires-Dist: rich>=13
Requires-Dist: typer>=0.12
Requires-Dist: uvicorn[standard]>=0.29
Description-Content-Type: text/markdown

# outo-llms

Deploy local LLMs behind your own managed, OpenAI-compatible API server.

vLLM or llama.cpp runs in an **isolated environment** managed by outo-llms,
kept apart from the Python environments you already have. Users sign up,
receive API keys, organize work into workspaces, and every request is
metered per workspace. Open the dashboard, point your OpenAI SDK at the
URL, and the rest is just an HTTP call.

Full documentation lives in [`docs/`](docs/index.md).

## Principles

1. **Fragmentation** - small, single-purpose modules.
2. **Fluidity** - features can be added or removed as needs change.
3. **Simplicity** - a handful of commands and a small API cover the common workflow.
4. **Automation** - one guided command gets you a working deployment.
5. **Explicitness** - every automated action is announced and logged; the
   system is never touched without your knowledge.

## Features

- **Isolated engine virtualenvs.** vLLM and llama.cpp install into their
  own virtual environments, never into the system interpreter.
- **Choice of engines.** Bring Hugging Face models through vLLM, or GGUF
  models through llama.cpp, and switch between them with one command.
- **Account, keys, and workspaces.** Open signup issues an `outo_sk_` API
  key, creates a `default` workspace, and lets each user create more.
- **Per-workspace usage metering.** Token usage is recorded by workspace
  and surfaced at `GET /v1/usage`.
- **OpenAI-compatible proxy.** `POST /v1/chat/completions`,
  `POST /v1/completions`, and `GET /v1/models` speak the same shapes
  clients already know.
- **Optional HTTPS.** A self-signed certificate is generated under
  `data/certs/` when you ask for it.
- **Explicit automation with an action log.** Every setup step is
  announced, confirmed when destructive, and appended to `actions.log`.
- **Dashboard and Swagger UI.** Visit the root URL for a tiny status
  page, or `/docs` for the full OpenAPI explorer.

## Install

```bash
pip install outo-llms
```

Prefer an isolated install? `pipx install outo-llms` works too.

## Quickstart

```bash
outo-llms setup                              # interactive, fully explicit setup
curl -s -X POST http://127.0.0.1:8611/v1/account/signup \
  -H 'Content-Type: application/json' \
  -d '{"username": "me"}'                    # returns an api_key + default workspace
outo-llms models add tinyllama               # register a Hugging Face model
curl -s http://127.0.0.1:8611/v1/chat/completions \
  -H "Authorization: Bearer outo_sk_..." \
  -H 'Content-Type: application/json' \
  -d '{"model": "tinyllama", "messages": [{"role": "user", "content": "hi"}]}'
curl -s http://127.0.0.1:8611/v1/usage \
  -H "Authorization: Bearer outo_sk_..."    # per-workspace token accounting
```

The setup wizard prints the exact base URL it configured (HTTP or HTTPS),
along with the path to the action log, and writes a short summary you can
copy into your shell history.

## Commands

| Command | Purpose |
| --- | --- |
| `outo-llms setup` | Automated, explicit server setup (engine, HTTPS, firewall, launch) |
| `outo-llms models add <name>` | Register a model in the registry |
| `outo-llms models list` | Show every registered model |
| `outo-llms models remove <name>` | Unregister a model (asks first) |
| `outo-llms engine list` | List known engines and their installed state |
| `outo-llms engine use <name>` | Switch the active engine (llamacpp or vllm) |
| `outo-llms engine install [name]` | Install an engine into its isolated virtualenv |
| `outo-llms engine status` | Show the active engine's runtime status |
| `outo-llms start` | Start the API server in the background |
| `outo-llms stop` | Stop the background API server |
| `outo-llms restart` | Restart the API server |
| `outo-llms status` | Show server, engine, and path status |
| `outo-llms reset` | Wipe everything back to factory state (asks twice) |
| `outo-llms version` | Print the version |

## How it works

The outo-llms server (`python -m outo_llms.server`) binds `127.0.0.1:8611`
by default and exposes an OpenAI-compatible HTTP API. Engines are
internal services that bind their own loopback ports (llama.cpp on 8612,
vLLM on 8613); clients never reach them directly.

When a request hits `POST /v1/chat/completions` or `POST /v1/completions`,
the server looks up the requested model in the registry, asks the engine
manager to ensure the right engine is running with that model, then
forwards the request. The active engine streams or returns its response
unchanged, and the server records prompt and completion tokens against
the calling workspace.

Every state change, install step, and external command goes through
`outo_llms.core.consent`, which announces what is about to happen, asks
for confirmation on destructive actions, and writes a timestamped line to
`logs/actions.log`. The dashboard at the root URL and Swagger UI at
`/docs` round out the picture.

## Documentation

- [`docs/index.md`](docs/index.md) — entry point for installation, API,
  configuration, operations, and testing guides.

## License

Apache License 2.0. See [`LICENSE`](LICENSE).
