Metadata-Version: 2.4
Name: mixle-mlops
Version: 0.7.0
Summary: All-in-one AI platform: host mixle models + open LLMs behind one OpenAI-compatible gateway, with accounts, API keys, multimodal, MCP, a chat UI, and a principled (mixle-powered) RLHF feedback loop.
Author: Grant Boquet
License-Expression: MIT
Project-URL: Homepage, https://github.com/gmboquet/mixle-mlops
Project-URL: Core, https://github.com/gmboquet/mixle
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mixle>=0.6.1
Requires-Dist: fastapi>=0.110
Requires-Dist: uvicorn[standard]>=0.29
Requires-Dist: pydantic>=2
Requires-Dist: pydantic-settings>=2
Requires-Dist: httpx>=0.27
Requires-Dist: sqlmodel>=0.0.16
Requires-Dist: python-multipart>=0.0.9
Requires-Dist: pyjwt[crypto]>=2.8
Requires-Dist: fsspec>=2023.1
Requires-Dist: numpy
Requires-Dist: scipy
Provides-Extra: dev
Requires-Dist: ruff==0.15.17; extra == "dev"
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: fakeredis>=2; extra == "dev"
Provides-Extra: documents
Requires-Dist: pypdf>=4; extra == "documents"
Requires-Dist: python-docx>=1; extra == "documents"
Requires-Dist: python-pptx>=0.6; extra == "documents"
Provides-Extra: scale
Requires-Dist: redis>=5; extra == "scale"
Provides-Extra: export
Requires-Dist: reportlab>=4; extra == "export"
Provides-Extra: datasets
Requires-Dist: pandas>=2; extra == "datasets"
Requires-Dist: pyarrow>=15; extra == "datasets"
Provides-Extra: structured
Requires-Dist: jsonschema>=4; extra == "structured"
Provides-Extra: local
Requires-Dist: torch>=2; extra == "local"
Requires-Dist: transformers>=4; extra == "local"
Requires-Dist: peft>=0.12; extra == "local"
Provides-Extra: cloud
Requires-Dist: psycopg[binary]>=3.1; extra == "cloud"
Requires-Dist: boto3>=1.34; extra == "cloud"
Requires-Dist: s3fs; extra == "cloud"
Requires-Dist: gcsfs; extra == "cloud"
Requires-Dist: adlfs; extra == "cloud"
Requires-Dist: ossfs; extra == "cloud"
Provides-Extra: mcp
Requires-Dist: mcp>=1.0; extra == "mcp"
Provides-Extra: image
Requires-Dist: diffusers>=0.27; extra == "image"
Requires-Dist: torch>=2; extra == "image"
Provides-Extra: docs
Requires-Dist: sphinx>=7; extra == "docs"
Requires-Dist: furo; extra == "docs"
Provides-Extra: all
Requires-Dist: mixle-mlops[cloud,datasets,documents,export,image,local,mcp,scale,structured]; extra == "all"
Dynamic: license-file

# mixle-mlops

![license](https://img.shields.io/badge/license-MIT-green)
![python](https://img.shields.io/badge/python-3.10%2B-blue)

An OpenAI-compatible gateway that serves [mixle](https://github.com/gmboquet/mixle) probabilistic models
alongside open and hosted LLMs (Llama, DeepSeek, Qwen, …) behind one API — accounts, API keys, a chat UI,
multimodal, RAG, an MCP server, and tool calling with a server-side agentic loop. Two things beyond that make
it more than an LLM proxy: a **probabilistic-bridge stack** that lifts a laptop-sized model toward frontier
quality on verifiable/structured tasks, and **self-evolution** that improves served models from their own
usage.

It runs end-to-end on a laptop (SQLite + filesystem + a local Ollama) and scales to the cloud (Postgres + object
store + Redis) by changing config — no code change. Local-first, cloud-optional.

## Quickstart

```sh
cp deploy/.env.example deploy/.env          # set MIXLE_SECRET_KEY
docker compose -f deploy/docker-compose.yml --env-file deploy/.env up -d gateway ollama
docker compose -f deploy/docker-compose.yml --env-file deploy/.env exec ollama ollama pull llama3.2
curl localhost:8000/v1/models               # OpenAI-compatible
```

## Installation

Python 3.10+ (developed and verified on 3.12), on PyPI as `mixle-mlops`.

```sh
pip install mixle-mlops                       # base: gateway, echo model, accounts, local storage
pip install "mixle-mlops[all]"                 # + documents, datasets, export, structured, local, cloud, mcp, image
mixle-mlops init-db && mixle-mlops create-user you@example.com pw --admin
mixle-serve                                    # the gateway on :8000
```

This installs the gateway package only. The chat UI (`frontend/`) is a separate Next.js app that
isn't shipped on PyPI — clone the repo for that (see Develop, below). See
[`docs/installation.rst`](docs/installation.rst) for the full extras list.

## What's in the box

| Capability | Surface |
|---|---|
| **OpenAI-compatible chat** (streaming) over any hosted model | `POST /v1/chat/completions`, `GET /v1/models` |
| **Host open LLMs** through their standard server (Ollama/vLLM/llama.cpp/hosted) | `MIXLE_LLM_BACKENDS` (local + cloud in one registry) |
| **Host mixle models** with a real probabilistic surface | `POST /v1/mixle/{predict,score,latent,decide}` |
| **Tool calling + server-side agent loop** (executes MCP tools, RAG, mixle decide/predict, exact compute) | `extra.agent`, `/mcp` |
| **Accounts, API keys, OAuth** (Sign in with Google/Apple) | `/auth/*`, `mk-…` keys |
| **Multimodal** (image inputs), **RAG** (PDF/DOCX/PPTX upload + retrieval), **image gen**, **dataset gen** | `/v1/files`, `/v1/documents`, `/v1/rag/search`, `/v1/images/generations`, `/v1/datasets` |
| **Conversations** (persisted threads + json/markdown/pdf export) | `/v1/conversations` |
| **LoRA/QLoRA fine-tuning** (SFT via chat template + prompt masking, or raw continuation; resume a prior adapter) and loading a trained adapter into the live gateway | `/v1/fine_tunes`, `/v1/models/load` |
| **Caching + rate limiting** (memory / Redis), **MCP server** | `extra` flags, `MIXLE_REDIS_URL`, `/mcp` |
| **Chat UI** (Next.js, assistant-style conversation surface) | `frontend/` |
| **Multi-cloud deploy** (AWS/Azure/GCP/Alicloud) | Helm chart + Terraform + `mixle-mlops init-cloud` |
| **Universal cloud compute** (GPU VMs, marketplaces, managed k8s, on-prem) | Any OpenAI-compatible endpoint + generic Docker/Kubernetes GPU recipes |

## Documentation

The Sphinx manual starts at [`docs/index.rst`](docs/index.rst). It includes installation notes, the
package map, validation guidance, generated API reference pages, and release-facing guides.

Build it with:

```sh
python -m pip install -e ".[docs]"
make -C docs html SPHINXOPTS="-W --keep-going"
```

Gateway surfaces are documented in
[`docs/gateway-capabilities.rst`](docs/gateway-capabilities.rst): substrate
serving, telemetry, pool jobs, creation verbs, Mixle model manifests, drift
retraining, and LoRA/QLoRA fine-tuning.

Release notes and the current changelog are in
[`docs/release-notes.rst`](docs/release-notes.rst) and
[CHANGELOG.md](CHANGELOG.md).

## The mixle bridge — frontier quality on a laptop

mixle is the calibrated **judge / combiner / router** around a small local generator. None of this puts 1T
parameters on your laptop; it trades inference-time compute + verification + routing for quality on
**verifiable / computational / structured / retrieval-grounded** tasks (and escalates the rest). All opt-in per
request via `extra`:

| Lever | Request | What it does |
|---|---|---|
| **Best-of-N self-consistency** | `extra.best_of_n: 5` | sample N, return the majority answer + a calibrated confidence (`X-Self-Consistency`) |
| **Cascade router** (FrugalGPT) | `extra.cascade: {frontier, threshold, n}` | answer locally when confident, escalate to a frontier model only on the hard tail (`X-Cascade-Escalated`) |
| **Mixture-of-Agents** | `extra.moa: {proposers, aggregator}` | several models propose, an aggregator synthesizes |
| **Program-offload** (PAL) | the `mixle_solve` tool (agent mode) | the model offloads exact arithmetic/probability/stats to a deterministic solver instead of doing mental math |

## Self-evolution — the system improves itself

Served mixle models improve from data and from the platform's own usage, **only ever promoting a challenger that
verifiably and non-regressively beats the champion** (built on `mixle.evolve`):

- `POST /v1/evolve/{model}` — run measure → propose → verify → promote (auto-select / online-update / recalibrate / refit), with rollback.
- `POST /v1/evolve/tick` — one autonomous pass over every hosted mixle model.
- `GET /v1/evolve/{model}/signals` — the cascade router **self-calibrates** its threshold from observed traffic.
- Every cascade decision + best-of-N vote is recorded as in-distribution feedback the loop consumes.

## Develop

```sh
pip install -e ".[dev]"                      # + extras: documents, scale, export, datasets, structured, local,
                                              #           cloud, mcp, image, all
pytest -q                                    # the full suite
ruff check mixle_mlops tests
mixle-mlops create-user you@example.com pw --admin   # a user + an API key
mixle-serve                                  # the gateway on :8000 (or: uvicorn mixle_mlops.gateway.app:app)
cd frontend && npm install && npm run dev    # the chat UI on :3000
```

Config is env-driven (prefix `MIXLE_`, see `deploy/.env.example` and `mixle_mlops/config.py`). Key knobs:
`MIXLE_LLM_BASE_URL` (Ollama by default), `MIXLE_LLM_BACKENDS` (per-model local+cloud), `MIXLE_DATABASE_URL`
(sqlite→postgres), `MIXLE_OBJECT_STORE_URL` (file→s3/gcs/azure/oss), `MIXLE_REDIS_URL`, `MIXLE_REQUIRE_AUTH`.

## Deploy

```sh
mixle-mlops init-cloud aws                    # scaffold a provider-correct .env
# Terraform provisions bucket + Postgres + Redis; Helm runs the same image on any k8s:
cd deploy/terraform/aws && terraform apply
helm install mixle deploy/helm/mixle-mlops -f deploy/helm/mixle-mlops/values-aws.yaml
```

The same image + chart run on EKS/AKS/GKE/ACK — only three URLs (database / object store / redis) change. Compute is
provider-neutral: point `MIXLE_LLM_BASE_URL` / `MIXLE_LLM_BACKENDS` at any OpenAI-compatible server running on a GPU
VM, marketplace instance, managed Kubernetes cluster, or on-prem host. See `deploy/cloud/README.md` and
`deploy/compute/README.md`.

## Design

The split mirrors mixle's: **mixle** owns the domain-neutral math (distributions, scoring, calibration, decision,
`mixle.evolve`); **mixle-mlops** owns serving/orchestration (gateway, accounts, RAG, the bridge components, the
evolution worker). The mixle math always upstreams to the core. See [`docs/overview.rst`](docs/overview.rst)
and [`docs/package-map.rst`](docs/package-map.rst) for the package boundaries.

> Requires `mixle.evolve` (published on PyPI since mixle 0.6.1; this package pins `mixle>=0.6.1`) for
> the self-evolution routes.

## Maintainers & contributors

Maintained by **Grant Boquet** ([@gmboquet](https://github.com/gmboquet) ·
grant.boquet@gmail.com).

Contributions, issues, and discussion are welcome — open a PR or an issue.

## License

MIT — see [LICENSE](https://github.com/gmboquet/mixle-mlops/blob/main/LICENSE).
