Metadata-Version: 2.4
Name: minix
Version: 0.3.4
Summary: A modular Python framework for backend, AI, and data projects
Author: AmirHossein Advari
Author-email: amiradvari@gmail.com
Requires-Python: >=3.11,<4
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Provides-Extra: ai
Provides-Extra: clickhouse
Provides-Extra: mysql
Provides-Extra: pgvector
Provides-Extra: postgresql
Provides-Extra: rag
Provides-Extra: vdb
Requires-Dist: Pillow (>=10.0.0) ; extra == "rag"
Requires-Dist: aiokafka (>=0.12.0,<0.13.0)
Requires-Dist: alembic (>=1.13) ; extra == "mysql"
Requires-Dist: alembic (>=1.13) ; extra == "postgresql"
Requires-Dist: alembic (>=1.13) ; extra == "rag"
Requires-Dist: authlib (>=1.3,<2.0)
Requires-Dist: boto3 (>=1.39.14,<2.0.0)
Requires-Dist: celery (>=5.5.3,<6.0.0)
Requires-Dist: celery-redbeat (>=2.3.2)
Requires-Dist: clickhouse-driver (>=0.2) ; extra == "clickhouse"
Requires-Dist: clickhouse-sqlalchemy (>=0.2) ; extra == "clickhouse"
Requires-Dist: dotenv (>=0.9.9,<0.10.0)
Requires-Dist: fastapi (>=0.116.1,<0.117.0)
Requires-Dist: httpx (>=0.27,<0.29)
Requires-Dist: langchain-core (>=0.3.0) ; extra == "rag"
Requires-Dist: langchain-litellm (>=0.6.0) ; extra == "rag"
Requires-Dist: langchain-qdrant (>=0.2.0) ; extra == "rag"
Requires-Dist: langchain-text-splitters (>=0.3.0) ; extra == "rag"
Requires-Dist: litellm (>=1.40.0) ; extra == "rag"
Requires-Dist: loguru (>=0.7) ; extra == "ai"
Requires-Dist: mlflow (>=2.0) ; extra == "ai"
Requires-Dist: numpy (>=1.24) ; extra == "ai"
Requires-Dist: numpy (>=1.24) ; extra == "vdb"
Requires-Dist: pdf2image (>=1.17.0) ; extra == "rag"
Requires-Dist: pgvector (>=0.4) ; extra == "pgvector"
Requires-Dist: psycopg[binary] (>=3.1) ; extra == "pgvector"
Requires-Dist: psycopg[binary] (>=3.1) ; extra == "postgresql"
Requires-Dist: psycopg[binary] (>=3.1) ; extra == "rag"
Requires-Dist: pydantic-settings (>=2.0,<3.0)
Requires-Dist: pymysql (>=1.1.1) ; extra == "mysql"
Requires-Dist: python-multipart (>=0.0.9) ; extra == "rag"
Requires-Dist: qdrant-client (>=1.0) ; extra == "vdb"
Requires-Dist: qdrant-client (>=1.12.0) ; extra == "rag"
Requires-Dist: redis (>=4.6.0,<5.0.0)
Requires-Dist: sqlalchemy (>=2.0.41,<3.0.0)
Requires-Dist: torch (>=2.0) ; extra == "ai"
Requires-Dist: typer (>=0.12,<1.0)
Requires-Dist: uvicorn (>=0.35.0,<0.36.0)
Description-Content-Type: text/markdown

# Minix


**Minix** is a modular Python framework for building backend, AI, and data-driven applications. It provides a clean, layered architecture with built-in support for REST APIs, task scheduling, message queues, and machine learning workflows.

![Python](https://img.shields.io/badge/Python-3.11+-blue.svg)
![Version](https://img.shields.io/badge/version-0.3.0-green.svg)
![License](https://img.shields.io/badge/license-MIT-lightgrey.svg)

---

## Table of Contents

- [Minix](#minix)
  - [Table of Contents](#table-of-contents)
  - [Key Features](#key-features)
  - [Architecture Overview](#architecture-overview)
  - [Installation](#installation)
    - [Requirements](#requirements)
    - [Install via pip](#install-via-pip)
    - [Install from source (development)](#install-from-source-development)
  - [Core Concepts](#core-concepts)
    - [Bootstrap](#bootstrap)
    - [Modules](#modules)
    - [Entities](#entities)
      - [SQL Entity](#sql-entity)
      - [Qdrant Entity (Vector DB)](#qdrant-entity-vector-db)
      - [PGVector Entity (PostgreSQL vectors)](#pgvector-entity-postgresql-vectors)
      - [Redis Entity](#redis-entity)
    - [Repositories](#repositories)
      - [SQL Repository](#sql-repository)
      - [Qdrant Repository](#qdrant-repository)
      - [PGVector Repository](#pgvector-repository)
    - [Services](#services)
    - [Controllers](#controllers)
    - [Connectors](#connectors)
      - [SQL Connector (PostgreSQL/MySQL/ClickHouse)](#sql-connector-postgresqlmysqlclickhouse)
      - [Qdrant Connector](#qdrant-connector)
      - [Object Storage Connector (S3-compatible)](#object-storage-connector-s3-compatible)
    - [Tasks \& Scheduling](#tasks--scheduling)
      - [Async Task](#async-task)
      - [Periodic Task](#periodic-task)
      - [Running Tasks Manually](#running-tasks-manually)
      - [Workflows (DAG Scheduling)](#workflows-dag-scheduling)
        - [Creating a Workflow](#creating-a-workflow)
        - [Dependency Result Passing](#dependency-result-passing)
        - [Running the Entire Workflow](#running-the-entire-workflow)
        - [Running a Specific Target Node](#running-a-specific-target-node)
        - [Workflow Execution Guarantees](#workflow-execution-guarantees)
    - [Kafka Consumers](#kafka-consumers)
    - [ML Models](#ml-models)
      - [Base Model](#base-model)
      - [Embedding Model](#embedding-model)
      - [MLflow Model (requires `ai` extra)](#mlflow-model-requires-ai-extra)
  - [RAG Module](#rag-module)
  - [OIDC Module](#oidc-module)
  - [Configuration](#configuration)
    - [Project config](#project-config)
    - [Connectors from config](#connectors-from-config)
    - [Environment overrides](#environment-overrides)
  - [Extras](#extras)
    - [SQL drivers](#sql-drivers)
    - [AI Capabilities](#ai-capabilities)
    - [Vector DB (Qdrant)](#vector-db-qdrant)
    - [Vector DB (pgvector)](#vector-db-pgvector)
    - [ClickHouse Support](#clickhouse-support)
    - [Install All Extras](#install-all-extras)
  - [Registry Usage](#registry-usage)
  - [CLI Commands](#cli-commands)
    - [Quick map](#quick-map)
    - [Global options](#global-options)
    - [`minix init`](#minix-init)
    - [`minix db`](#minix-db)
      - [Migration workflow](#migration-workflow)
      - [Migration commands](#migration-commands)
      - [Revision targets](#revision-targets)
      - [Migration recipes](#migration-recipes)
      - [How migrations work](#how-migrations-work)
      - [Migration troubleshooting](#migration-troubleshooting)
    - [`minix add module`](#minix-add-module)
      - [SQL binding (all or none)](#sql-binding-all-or-none)
      - [Flags](#flags)
      - [Naming rules](#naming-rules)
      - [Module layout](#module-layout)
    - [Adding to an existing module](#adding-to-an-existing-module)
      - [After scaffolding](#after-scaffolding)
  - [License](#license)
  - [Contributing](#contributing)
  - [Author](#author)

---

## Key Features

- **FastAPI Integration**: Build high-performance REST APIs with automatic OpenAPI documentation
- **Modular Architecture**: Organize code into self-contained modules with entities, repositories, services, and controllers
- **Multi-Database Support**: Built-in connectors for PostgreSQL (default), MySQL, ClickHouse, Redis, and Qdrant (vector DB)
- **Task Scheduling**: Celery-powered background tasks with RedBeat scheduler for periodic jobs
- **Kafka Consumers**: Async Kafka message processing with `aiokafka`
- **Object Storage**: S3-compatible storage support via `boto3`
- **ML Workflows**: Optional MLflow integration for model versioning and deployment
- **RAG Module**: Pluggable retrieval-augmented generation — collections, document ingestion, chunking, hybrid BM25 + vector retrieval, LLM reranking, and answers with citations
- **Dependency Registry**: Singleton-based service container for clean dependency injection
- **Environment Management**: `config.py` (pydantic-settings) with `.env` support

---

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────────┐
│                         Application                              │
├─────────────────────────────────────────────────────────────────┤
│  Bootstrap → Registers Connectors & Modules                      │
├─────────────────────────────────────────────────────────────────┤
│                          Modules                                 │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐            │
│  │Controller│ │ Service  │ │Repository│ │  Entity  │            │
│  │ (API)    │→│ (Logic)  │→│  (Data)  │→│ (Model)  │            │
│  └──────────┘ └──────────┘ └──────────┘ └──────────┘            │
├─────────────────────────────────────────────────────────────────┤
│  Tasks (Celery)  │  Consumers (Kafka)  │  Models (MLflow)       │
├─────────────────────────────────────────────────────────────────┤
│                        Connectors                                │
│  SQL (PostgreSQL/MySQL/ClickHouse) │ Redis │ Qdrant │ Object Storage (S3)  │
└─────────────────────────────────────────────────────────────────┘
```

---

## Installation

### Requirements

- Python 3.10 or higher

### Install via pip

```bash
pip install minix
```

### Install from source (development)

```bash
git clone <repository-url>
cd minix
pip install -e .
```

---

## Core Concepts

### Bootstrap

The `bootstrap` function initializes your application by registering connectors and modules:

```python
# Preferred: connectors and INSTALLED_MODULES come from config.py.
from minix.core.bootstrap import bootstrap_from_settings
bootstrap_from_settings()

# Deprecated (backward compatible): pass explicit lists.
from minix.core.bootstrap import bootstrap
bootstrap(
    modules=[Module1(), Module2()],
    connectors=[
        (sql_connector, None),           # Default connector
        (sql_connector_2, "analytics")   # Named connector with salt
    ]
)
```

### Modules

Modules are self-contained units that group related functionality:

```python
from minix.core.module.business_module import BusinessModule

class ProductModule(BusinessModule):
    def __init__(self):
        super().__init__("product")
        self.add_binding(ProductEntity, ProductRepository, ProductService)
        self.add_controller(ProductController)
        self.add_periodic_task(SyncProductsTask)
        self.add_consumer(ProductEventConsumer)
```

**Module Methods:**
- `add_binding(entity, repository, service, connection=None, *, provides_repository=None, provides_service=None)` - Register entity + repository + service together (required unit; do not split them)
- `add_entity` / `add_repository` / `add_service` - **Deprecated**; use `add_binding` instead
- `add_controller(controller, provides=None)` - Register an API controller
- `add_helper_service(helper)` - Register a helper service
- `add_task(task)` - Register an async task
- `add_periodic_task(periodic_task)` - Register a scheduled task
- `add_consumer(consumer)` - Register a Kafka consumer
- `add_model(model, config)` - Register an ML model

**Swapping in a subclass (`provides`)**: `provides=Base` also registers the
installed instance under `Base`, so every `Registry().get(Base)` lookup —
including the framework's own — resolves to your subclass. This is how you
extend a built-in module (e.g. the RAG module below) without editing it:

```python
from minix.core.modules.rag.controllers import RagController
from minix.core.modules.rag.services import RagCollectionService

module = (
    BusinessModule("my_rag")
    .add_binding(MyCollectionEntity, MyCollectionRepository, MyCollectionService,
                 provides_service=RagCollectionService)
    .add_controller(MyRagController, provides=RagController)
)
```

### Entities

Entities define your data models:

#### SQL Entity

```python
from sqlalchemy import String
from sqlalchemy.orm import Mapped, mapped_column
from minix.core.entity import SqlEntity

class UserEntity(SqlEntity):
    __tablename__ = "users"

    name: Mapped[str] = mapped_column(String(100), nullable=False)
    email: Mapped[str] = mapped_column(String(255), unique=True, nullable=False)
```

SQL entities automatically include `id`, `created_at`, and `updated_at` columns.

#### Qdrant Entity (Vector DB)

```python
from minix.core.entity import QdrantEntity

class DocumentEntity(QdrantEntity):
    content: str

    @staticmethod
    def collection() -> str:
        return "documents"
```

#### PGVector Entity (PostgreSQL vectors)

Requires the `pgvector` extra and a PostgreSQL server with the `vector` extension.

```python
from sqlalchemy import Text
from sqlalchemy.orm import Mapped, mapped_column
from minix.core.entity import PGVectorEntity

class DocumentEmbeddingEntity(PGVectorEntity):
    vector_size = 1536

    content: Mapped[str] = mapped_column(Text, nullable=False)
```

PGVector entities are SQL entities: they include `id`, `created_at` and
`updated_at`, plus a `vector` column of type `VECTOR(vector_size)`.

#### Redis Entity

```python
from minix.core.entity import RedisEntity

class CacheEntity(RedisEntity):
    pass
```

### Repositories

Repositories handle data access:

#### SQL Repository

```python
from minix.core.repository import SqlRepository

class UserRepository(SqlRepository[UserEntity]):
    pass
```

**Built-in methods:**
- `save(entity)` - Save a single entity
- `save_all(entities)` - Save multiple entities
- `save_bulk(entities, chunk_size)` - Bulk insert with chunking
- `get_all()` - Get all entities
- `get_by_id(id)` - Get by ID
- `get_by(**kwargs)` - Filter by attributes
- `update(entity)` - Update an entity
- `delete(entity)` - Delete an entity

**Cross-module transactions** (one session, one commit):

By default each repository call opens its own session and commits. Wrap related
writes in `sql_transaction()` so every repository on that connector shares one
session until the block exits:

```python
from minix.core.connectors import sql_transaction
from minix.core.registry import Registry

with sql_transaction():
    Registry().get(OrderSqlRepository).save(order)
    Registry().get(PaymentSqlRepository).save(payment)
# committed once here; any exception rolls back all work in the block

with sql_transaction("mysql"):  # named DATABASES entry
    ...
```

Equivalent long form: `Registry().get(SqlConnector).transaction()`.
Nested calls on the same connector use SAVEPOINTs. Custom repository methods
that call `session.commit()` still join the outer unit (`commit` becomes
`flush` while the transaction is active).

#### Qdrant Repository

```python
from minix.core.repository import QdrantRepository

class DocumentRepository(QdrantRepository[DocumentEntity]):
    pass
```

**Methods:**
- `insert(entities)` - Insert vectors
- `query(entity, top_k)` - Similarity search
- `delete(entity_ids)` - Delete vectors

#### PGVector Repository

```python
from minix.core.repository import PGVectorRepository

class DocumentEmbeddingRepository(PGVectorRepository[DocumentEmbeddingEntity]):
    def get_distance(self) -> str:
        return 'Cosine'  # or 'Euclid', 'Dot', 'Manhattan'
```

It extends `SqlRepository` (all SQL methods above, `sql_transaction()`
included) and uses the registered `SqlConnector`, which must point at PostgreSQL.

**Additional methods:**
- `create_extension()` - Run `CREATE EXTENSION IF NOT EXISTS vector`
- `search(vector, top_k, max_distance=None, **kwargs)` - Nearest neighbours, closest first; `kwargs` filter like `get_by`
- `search_with_distance(vector, top_k, max_distance=None, **kwargs)` - Same, as `(entity, distance)` pairs

`PGVectorService` exposes the same `search` / `search_with_distance` methods,
plus `get_knn(entity, top_k)`, on top of `SqlService`.

### Services

Services contain business logic:

```python
from minix.core.service import Service

class UserService(Service[UserEntity]):
    def get_active_users(self):
        return self.repository.get_by(status="active")

    def create_user(self, name: str, email: str) -> UserEntity:
        user = UserEntity(name=name, email=email)
        return self.repository.save(user)
```

### Controllers

Controllers define REST API endpoints:

```python
from minix.core.controller import Controller
from minix.core.registry import Registry

class UserController(Controller):
    def get_prefix(self):
        return "/users"

    def define_routes(self):
        @self.router.get("/")
        def get_users():
            service = Registry().get(UserService)
            return service.get_repository().get_all()

        @self.router.post("/")
        def create_user(name: str, email: str):
            service = Registry().get(UserService)
            return service.create_user(name, email)
```

### Connectors

#### SQL Connector (PostgreSQL/MySQL/ClickHouse)

PostgreSQL is the recommended default (`DB_DRIVER=postgresql`). Install the matching
extra (`minix[postgresql]` and/or `minix[mysql]`). ClickHouse remains
`minix[clickhouse]`.

```python
from minix.core.connectors import SqlConnector

# Reads DATABASES["default"] from settings / env
connector = SqlConnector()
# Secondary engine when both were selected at init:
# connector = SqlConnector(connection="mysql")

# Deprecated — still supported for backward compatibility:
# from minix.core.connectors import SqlConnector, SqlConnectorConfig
# connector = SqlConnector(SqlConnectorConfig(username="minix", driver="postgresql", ...))
```

#### Qdrant Connector

```python
from minix.core.connectors import QdrantConnector

# Reads QDRANT_CONNECTIONS[connection] from settings / env
connector = QdrantConnector()
await connector.connect()
```

#### Object Storage Connector (S3-compatible)

```python
from minix.core.object_storage import ObjectStorageConnector

# Reads OBJECT_STORAGES[connection] from settings / env
connector = ObjectStorageConnector()

# Upload, download, delete files
await connector.upload_file(file_obj, "path/to/file.txt")
await connector.download_file("path/to/file.txt", local_file)
await connector.generate_presigned_url("path/to/file.txt", expiration=3600)
```

### Tasks & Scheduling

#### Async Task

```python
from minix.core.scheduler.task import Task

class ProcessOrderTask(Task):
    def get_name(self) -> str:
        return "process_order"

    def run(self, order_id: int):
        # Process the order
        pass
```

#### Periodic Task

```python
from celery.schedules import crontab
from minix.core.scheduler.task import PeriodicTask

class DailyReportTask(PeriodicTask):
    def get_name(self) -> str:
        return "daily_report"

    def get_schedule(self) -> crontab:
        return crontab(hour=0, minute=0)  # Run at midnight

    def run(self):
        # Generate daily report
        pass
```

#### Running Tasks Manually

```python
from minix.core.registry import Registry
from minix.core.scheduler import Scheduler

scheduler = Registry().get(Scheduler)
scheduler.run_task(ProcessOrderTask(order_id=123))
```

#### Workflows (DAG Scheduling)

Minix supports defining and executing **workflows as DAGs** (Directed Acyclic Graphs) where:
- Each node is a `Task`
- Nodes can depend on other nodes
- Dependency results can be passed to downstream tasks
- The workflow engine guarantees **each node runs at most once** during a workflow execution
- If the DAG has multiple sink nodes, **all sinks are executed**

##### Creating a Workflow

```python
from minix.core.registry import Registry
from minix.core.scheduler import Scheduler
from minix.core.scheduler.workflow import Workflow

scheduler = Registry().get(Scheduler)

wf = Workflow("order_pipeline")

t1 = wf.add(DAGTaskTest1(in_a=aaaaa), node_id="task1")
t2 = wf.add(DAGTaskTest2(), node_id="task2", depends_on=[t1], consume_dependency_results=True)
t3 = wf.add(DAGTaskTest3(), node_id="task3", depends_on=[t2, t1], consume_dependency_results=True)
t4 = wf.add(DAGTaskTest4(), node_id="task4", depends_on=[t3], consume_dependency_results=True)

t5 = wf.add(DAGTaskTest5(), node_id="task5", depends_on=[t4], consume_dependency_results=True)

res = scheduler.run_workflow(wf)
```

##### Dependency Result Passing

For a node with `consume_dependency_results=True`:

- If it has **1 dependency**, that dependency result is passed as the **first positional argument** (`arg0`) to the task.
- If it has **multiple dependencies**, a **list of dependency results** is passed as `arg0` in the same order as `depends_on`.

For a node with `consume_dependency_results=False`, no dependency result is injected.

> This design allows tasks to explicitly opt in/out of consuming upstream values, while still receiving their own constructor-provided `args/kwargs`.

##### Running the Entire Workflow

```python
res = scheduler.run_workflow(wf)
print(res.id)  # Celery task id
# output = res.get(timeout=...)  # if you want to block and fetch the final result
```

**Final result shape when running the whole workflow:**
- If the workflow has **one sink**, the final result is that sink’s output.
- If the workflow has **multiple sinks**, the final result is a dictionary mapping sink node ids to outputs:
  ```python
  {
    "task5": <result_of_task6>,
    "task4": <result_of_task9>
  }
  ```

##### Running a Specific Target Node

You can execute only the subgraph required to compute a particular node (useful for partial runs or debugging):

```python
res = scheduler.run_workflow(wf, target_node_id="task9")
# output = res.get(timeout=...)
```

In this mode, only `task9` and its dependency closure are executed, and the returned value corresponds to the output of the target node (`task9`).

##### Workflow Execution Guarantees

When executing a workflow:
- **No duplicate execution**: if a node is required by multiple downstream nodes, it runs once.
- **Multiple sinks are supported**: all sink nodes run when executing the whole workflow.
- **Full-graph execution**: when no `target_node_id` is provided, the workflow runs the entire DAG (not only the ancestry of a single sink).

---

### Kafka Consumers

```python
from minix.core.consumer import AsyncConsumer, AsyncConsumerConfig

class OrderEventConsumer(AsyncConsumer):
    def get_config(self) -> AsyncConsumerConfig:
        return AsyncConsumerConfig(
            name="order_consumer",
            topics=["orders"],
            group_id="order_service",
            bootstrap_servers=["localhost:9092"]
        )

    async def run(self, message: dict):
        # Process the message
        order_id = message.get("order_id")
        print(f"Processing order: {order_id}")
```

### ML Models

#### Base Model

```python
from minix.core.model import Model

class MyModel(Model):
    def get_model_name(self):
        return "my_model"

    def predict(self, model_input: dict):
        # Prediction logic
        pass

    def set_device(self, device: str):
        self.device = device
```

#### Embedding Model

```python
from minix.core.model import EmbeddingModel
import numpy as np

class TextEmbedder(EmbeddingModel):
    def get_model_name(self):
        return "text_embedder"

    def embed(self, text: str) -> np.ndarray:
        # Return embedding vector
        pass

    def embed_batch(self, texts: list[str]) -> np.ndarray:
        # Return batch of embeddings
        pass

    def set_device(self, device: str):
        self.device = device
```

#### MLflow Model (requires `ai` extra)

```python
from minix.core.model import MlflowModel

class MyMlflowModel(MlflowModel):
    def __init__(self):
        super().__init__(
            name="my_model",
            version=1,
            packages=["torch", "transformers"]
        )

    def get_model(self):
        # Return your model instance
        pass

    def predict(self, model_input):
        model = self.load_model()
        return model.predict(model_input)
```

---

## RAG Module

`minix.core.modules.rag` is a ready-made retrieval-augmented generation module:
a REST API over named **collections** of documents, with async ingestion, hybrid
retrieval and LLM answers that cite their sources.

**What it does**

- **Ingestion** — upload a file (`POST /rag/ingest/upload`: PDF or images read
  by a vision model, or text formats) or post raw text (`POST /rag/ingest/text`).
  Ingestion runs in the background and returns a `job_id` to poll.
- **Chunking** — per-collection strategy: `smart`, `fixed`, `by_heading`,
  `by_clause`, `qa`, `sentence` or `llm` (instruction-driven).
  `GET /rag/chunking/strategies` returns UI labels for each.
- **Metadata extraction** — the LLM detects the document type and extracts
  typed metadata, stored in PostgreSQL and on every vector payload.
- **Hybrid retrieval** — PostgreSQL full-text search (`tsvector`) plus Qdrant
  cosine vectors, fused (weighted or RRF) and filtered by a relevance floor.
- **LLM reranking** — optional reranker that re-orders the retrieved chunks
  before the answer prompt (per request, or server default).
- **Answers with citations** — `POST /rag/query` (async, poll by `query_id`),
  single collection or an explicit multi-collection allowlist. Every response
  carries the retrieved and selected chunks and a refusal classification
  (`out_of_corpus`, `out_of_scope`, `policy_violation`, `ambiguous`). Retrieved
  text is fenced as untrusted data in the prompt.
- **Search without an LLM** — `POST /rag/search` (hybrid / vector / keyword).
- **Multi-query match** — `POST /rag/match` blends several weighted queries into
  one document ranking (synchronous).
- **Management** — collections, documents, chunks, jobs, `GET /rag/health`,
  `GET /rag/me`.

**Install** — the module is optional; a plain `pip install minix` does not pull
in any of its dependencies:

```bash
pip install "minix[rag]"
```

At runtime it also needs a PostgreSQL server (keyword search + metadata) and a
Qdrant server (vectors); PDF ingestion needs the `poppler` system package.
Importing `minix.core.modules.rag` without the extra raises an `ImportError`
that tells you the command above.

**Enable it with the CLI:**

```bash
# New project (PostgreSQL is the default)
minix init my_app --rag

# Existing project with PostgreSQL as its default database
minix add rag
```

`--rag` enables Qdrant, registers `AuthModule` and `RagModule`, adds model settings
to the environment templates, and includes the `rag` extra and Poppler in Docker.
`--extras rag` also enables RAG. RAG is enabled explicitly rather than inferred
from installed packages. `--rag --no-extras` still includes the RAG requirements.
MySQL alone is rejected; when both SQL drivers are selected, PostgreSQL is primary.

`minix add rag` preserves existing settings and secrets, registers missing modules,
adds missing keys to `.env` and `.env.example`, adds the `rag` extra to the minix
requirement in `pyproject.toml` (PEP 621 or Poetry) or `requirements.txt`, and updates
the Dockerfile dependencies. It creates `docker-compose.rag.yml` for Qdrant and the
app/worker connections, and sets `COMPOSE_FILE` in `.env` so a plain
`docker compose up --build` loads it. Qdrant requires `QDRANT_API_KEY` (it rejects
every request when the key is set but empty): `.env` gets a random key and
`.env.example` a `change-me` placeholder.

Repeated `minix add rag` calls leave the generated files unchanged. Afterwards,
reinstall (for example `poetry lock && poetry install`), install Poppler for local
PDF ingestion, set the model provider's API key in `.env`, and create the SQL tables
with `minix db makemigrations -m "add rag" && minix db migrate` (see [`minix db`](#minix-db)).

**Manual registration** — in a project from `minix init` with PostgreSQL as the `default`
database, add the module to `installed_modules` in `config.py`:

```python
installed_modules: list[str] = Field(
    default_factory=lambda: [
        "minix.core.modules.auth.AuthModule",
        "minix.core.modules.rag.RagModule",
    ]
)
```

`bootstrap_from_settings()` (already called in `entries/api.py`) then registers
the connectors and installs the module. Create its tables with
[`minix db`](#minix-db):

```bash
minix db makemigrations -m "add rag"
minix db migrate
```

Without migrations, create them once from code instead:

```python
from minix.core.connectors import SqlConnector
from minix.core.entity.sql_entity import Base
from minix.core.registry import Registry

Base.metadata.create_all(Registry().get(SqlConnector).get_engine())
```

Qdrant: the module uses the `default` entry of `qdrant_connections` when there
is one, and `QDRANT_URL` / `QDRANT_API_KEY` otherwise.

`AuthModule` provides the API keys the RAG routes accept (`X-API-Key`). With
`RAG_ACCESS_CONTROL_ENABLED=false` (the default) requests without a key run as
an anonymous admin; set it to `true` to require a key and enforce the key's role
(`readonly` may read and query, `user` may also ingest and edit, `admin` may
also delete collections).

**Configuration** (environment variables, read by `minix.core.modules.rag.config.RagConfig`)

| Variable | Default | Purpose |
|---|---|---|
| provider API key | — | The standard variable for the provider you use (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `AZURE_API_KEY`, …) — LiteLLM reads it |
| `LLM_MODEL` | `gpt-4o` | Answers, reranking (any LiteLLM model name) |
| `LLM_TEMPERATURE` | unset | Passed to the model only when set |
| `RAG_LLM_TIMEOUT_SECONDS` / `RAG_LLM_MAX_TOKENS` | `60` / `2048` | LLM call bounds |
| `EMBEDDING_MODEL` | `text-embedding-3-small` | Embedding model (any LiteLLM embedding model) |
| `EMBEDDING_DIMENSION` | `1536` | Vector size of the Qdrant collections — must equal what the model returns |
| `RAG_EMBEDDING_SEND_DIMENSIONS` | `true` | Also send it as the `dimensions` parameter; `false` for fixed-size models that reject it |
| `DOCUMENT_PROCESSOR_MODEL` | `gpt-4o` | PDF / image transcription, metadata extraction, LLM chunking (vision-capable) |
| `QDRANT_URL` / `QDRANT_API_KEY` | `http://localhost:6333` / — | Vector store (a registered `QdrantConnector` takes precedence) |
| `CHUNK_SIZE` / `CHUNK_OVERLAP` | `800` / `100` | Default chunking |
| `TOP_K` | `5` | Default retrieval depth |
| `BM25_WEIGHT` / `VECTOR_WEIGHT` | `0.3` / `0.7` | Hybrid fusion weights |
| `MIN_RELEVANCE_SCORE` | `0.3` | Relevance floor before the answer prompt |
| `RAG_BM25_QUERY_MODE` | `plainto` | `or` suits long natural-language queries |
| `RAG_RETRIEVAL_FETCH_MULTIPLIER` / `RAG_RETRIEVAL_CANDIDATE_FLOOR` | `6` / `50` | Candidate pool per retrieval leg |
| `LLM_SELECTOR_ENABLED` | `true` | LLM reranker on `/query` by default |
| `SELECTOR_FALLBACK_TOP_N` | `5` | Chunks handed to the answer prompt |
| `RAG_RERANKER_MODEL` / `RAG_RERANKER_MIN_SCORE` / `RAG_RERANKER_MAX_CHARS` | `LLM_MODEL` / `5` / `1200` | Reranker tuning |
| `RAG_INGEST_IDENTITY_ENRICHMENT` | `false` | Prefix chunks with a title / parties header |
| `DEFAULT_COLLECTION` | `default` | Collection used when a request names none |
| `RAG_ACCESS_CONTROL_ENABLED` | `false` | Require an API key and enforce roles |

**How it fits the framework** — it is a regular module: `RagModule` is a
`BusinessModule` built from one `add_binding(entity, repository, service)` per
table (collections, documents, chunks, jobs, search requests, query requests)
plus a controller, exactly like `AuthModule` / `OidcModule`, and everything it
needs (config, chunking, retrieval, document processors) lives inside
`minix/core/modules/rag/`. It reads the SQL connector from the Registry like any
repository, authenticates with the `auth` module's API keys, and imports nothing
that the rest of minix imports back. Long-running work (ingestion, search,
query) uses FastAPI background tasks plus persisted job rows, so the module needs
no Celery broker or Kafka.

**Providers** — nothing is tied to one vendor: every model call (answers,
reranking, embeddings, metadata extraction, document transcription, LLM
chunking) goes through [LiteLLM](https://docs.litellm.ai/), so changing a model
name is all it takes to move to another provider, a gateway or a local server:

```env
LLM_MODEL=anthropic/claude-sonnet-5          # + ANTHROPIC_API_KEY
EMBEDDING_MODEL=ollama/nomic-embed-text         # local, no key
EMBEDDING_DIMENSION=768
RAG_EMBEDDING_SEND_DIMENSIONS=false
DOCUMENT_PROCESSOR_MODEL=anthropic/claude-sonnet-5
```

Any OpenAI-compatible endpoint (vLLM, LM Studio, a gateway) works with the
`openai/<model>` prefix and `OPENAI_API_BASE`. Changing the embedding model or
its size requires re-indexing: existing vectors were produced by the old one.

**Extending it** — `RagController` exposes protected hook methods (request
scoping, gating and usage metering, per-collection access policy via
`_svc_access()`, audit, input rewriting, answer generation, output shaping,
provider-event and cost recording) and swappable response models
(`COLLECTION_INFO_MODEL`, `QUERY_RESPONSE_MODEL`, …). All defaults are
single-scope no-ops. Subclass the controller and register it with
`provides=RagController`; add columns to the RAG tables with
single-table-inheritance subclasses of the entities (no `__tablename__`), mounted
through `provides=` on the paired repository and service.

---

## OIDC Module

`minix.core.modules.oidc` is a standalone OpenID Connect module (it does not depend on the core `auth` module): authorization-code login with a signed session cookie, bearer-token validation against the IdP's JWKS, role mapping, tenant resolution, a token client and a login hook. Add `OidcModule` to `INSTALLED_MODULES` and set at least `OIDC_ISSUER` and `OIDC_CLIENT_ID`.

| Setting | Default | Purpose |
|---|---|---|
| `OIDC_ISSUER` / `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET` | — | IdP issuer URL (discovery) and client credentials |
| `OIDC_AUDIENCE` | `OIDC_CLIENT_ID` | Accepted `aud` for bearer tokens; a comma-separated list accepts several audiences. ID tokens at the login callback are always checked against `OIDC_CLIENT_ID` |
| `OIDC_ALGORITHMS` | `RS256,ES256,PS256` | Allowed signing algorithms; `none` and `HS*` are always rejected |
| `OIDC_ALLOWED_REDIRECT_HOSTS` | empty | Hosts allowed as absolute `redirect` targets on login/logout (otherwise only relative paths) |
| `OIDC_AUTO_CREATE_TABLE` | `true` | Create the OIDC users table once at startup |
| `OIDC_TENANT_CLAIM` | `organization` | Token claim holding the user's organization(s); `none` disables tenant resolution (everyone gets their personal tenant) |
| `OIDC_SCOPES` | `openid profile email` | Add `offline_access` to receive a refresh token. Keycloak only emits the `organization` claim when the `organization` scope is requested, e.g. `openid profile email organization` |
| `OIDC_ROLE_CLAIM` / `OIDC_ROLE_MAP` / `OIDC_DEFAULT_ROLE` | `realm_access.roles` / — / `user` | Role extraction and mapping |
| `OIDC_JWKS_TTL` | `3600` | JWKS cache seconds (an unknown `kid` triggers one refetch, at most once per 60 s) |
| `OIDC_SESSION_SECRET` / `OIDC_SESSION_COOKIE` / `OIDC_SESSION_TTL` / `OIDC_COOKIE_SECURE` | — / `minix_session` / `28800` / `true` | Session cookie |

**Dependencies and identity** — `get_oidc_user` and `RequireOidcRole` return an `OidcContext` (`user_id`, `email`, `role`, `claims`, `tenant_id`, `tenant_name`, `tenant_is_personal`) or raise 401/403. `resolve_oidc_context(request)` returns an `OidcContext` or `None` when the request is unauthenticated. `resolve_identity(request, fallbacks)` returns an `OidcContext`, whatever identity the first matching fallback returns, or `None`. `resolve_identity` tries a bearer from the configured issuer, then the minix session cookie, then each fallback (`async (request) -> identity | None`). A bearer from a different issuer is skipped, so the app's own JWTs reach its fallbacks; an invalid bearer from the configured issuer returns `None` without trying fallbacks. An invalid or expired minix session cookie is treated as "no session" and falls through to the fallbacks (unlike an invalid bearer from the configured issuer).

**Tenants** — the tenant comes from `OIDC_TENANT_CLAIM`, which may be a single alias string, a list of aliases, or Keycloak's map form `{alias: {"id": ...}}` (the `id` becomes `tenant_id`, the alias `tenant_name`; an entry without `id` uses its alias). With several organizations the caller picks one with the `X-Organization` header (matching id or alias). Users without an organization get a personal tenant `user:{sub}` (`tenant_is_personal=True`); organization ids starting with `user:` are ignored so they cannot collide with personal tenants. `resolve_oidc_context` and `resolve_identity` can raise `HTTPException` 400 (`organization_selection_required`) or 403 (`organization_not_in_token`) for multi-organization users: call them from FastAPI dependencies, or handle the exception. Set `OIDC_TENANT_CLAIM=none` (or `""` in `settings.py`) to disable tenant resolution: every identity gets its personal tenant and `X-Organization` is ignored.

For browser sessions, organization membership is a snapshot taken at login and stored in the session cookie: changes at the IdP apply at the next login or when the session expires (`OIDC_SESSION_TTL`, default 8 h). Session cookies issued by 0.3.3 carry no organizations and resolve as personal until the user logs in again. Very large organization claims enlarge the cookie accordingly.

**Token client** — `OidcTokenClient` (registered by `OidcModule`) wraps the token endpoint and returns `TokenSet` (`access_token`, `expires_at`, `refresh_token`, `scope`):

```python
from minix.core.registry import Registry
from minix.core.modules.oidc import OidcTokenClient

client = Registry().get(OidcTokenClient)
svc = client.client_credentials(audience="billing-api")
fresh = client.refresh(refresh_token)
downstream = client.exchange(subject_token=access_token, audience="other-api")  # RFC 8693
client.revoke(refresh_token)
```

Errors: `OidcError` (base), `OidcTokenRevoked`, `OidcExchangeDenied`, `OidcUnavailable` (also raised for 5xx and unexpected 3xx responses).

`OidcTokenClient` is synchronous: each call blocks on an HTTP request (10 s timeout). From async code, run it in the threadpool:

```python
from starlette.concurrency import run_in_threadpool

tokens = await run_in_threadpool(client.exchange, access_token, "other-api")
```

**Login hook** — implement `OidcLoginHook.on_login(claims, token_response)` (sync or `async def`) and register it with `Registry().register(OidcLoginHook, instance)`. It runs once per successful login, before the session cookie is issued; an async hook is awaited. If it raises, the login fails with HTTP 500 and no session is set. minix never stores the token response: the hook owns it.

**Schema ownership** — apps that manage their schema with Alembic set `OIDC_AUTO_CREATE_TABLE=false` and include the exported table in their migrations:

```python
from minix.core.modules.oidc import OIDC_USERS_TABLE  # SQLAlchemy Table, usable in env.py / autogenerate
```

**Security notes** — login/logout `redirect` accepts relative paths only (absolute URLs only for `OIDC_ALLOWED_REDIRECT_HOSTS`; userinfo and percent-encoded authority are rejected). JWT verification and the user upsert run in a threadpool; the upsert is skipped when nothing changed (300 s cache).

---

## Configuration

Minix uses a **pydantic-settings** config module (`config.py`).
Values come from the environment / `.env`, with defaults on the `Settings` class.

### Project config

```bash
minix init my_project
```

This creates `config.py`, `.env`, `.env.example`, `Dockerfile`,
`docker-compose.yml`, and `entries/`. The name argument sets `app_name`;
optional `--*-port` flags set Docker / env port defaults.
`.env` includes `MINIX_SETTINGS_MODULE=config`.


```python
from minix.core.conf import settings
from config import config

print(settings.APP_NAME)
print(settings.DATABASES["default"]["host"])
print(config.app_name)  # same pydantic instance
```

Override the module with `MINIX_SETTINGS_MODULE`. With no `config` module,
framework defaults from `global_settings` are used.

### Connectors from config

Edit the `Settings` class (or the exported `DATABASES` map) in `config.py`.
When ``bootstrap_from_settings()`` is called without ``connectors=``, Minix registers one
connector per map key:

```python
# config.py — add another database entry under DATABASES
DATABASES = {
    "default": {...},
    "analytics": {
        "user": "...",
        "password": "...",
        "host": "...",
        "port": 5432,
        "name": "analytics",
        "driver": "postgresql",
    },
}

# entries/api.py
bootstrap_from_settings()
```

Maps wired automatically: ``DATABASES``, ``OBJECT_STORAGES``,
``QDRANT_CONNECTIONS``, ``REDIS_CONNECTIONS``. Each key becomes Registry
``salt`` (``"default"`` → no salt).

Modules are listed in ``INSTALLED_MODULES`` as dotted paths::

    INSTALLED_MODULES = [
        "minix.core.modules.auth.AuthModule",
        "src.modules.example.ExampleModule",
    ]

Pass an explicit ``connectors=[...]`` / ``modules=[...]`` list to take full
control (existing apps keep working). Pass ``connectors=[]`` /
``modules=[]`` to register/install none.

Multiple Celery apps use ``CELERY_CONNECTIONS``;
``scheduler_config(connection="priority")`` picks a named entry.
Bootstrap still registers the ``"default"`` scheduler when modules need tasks.

Missing keys raise ``ImproperlyConfigured``.

Passing an explicit ``SqlConnectorConfig`` / URL still works but is
**deprecated**; prefer ``connection=`` with settings maps.

### Environment overrides

Any key in `.env` overrides the default in `config.py`:

```env
DB_HOST=localhost
DB_DATABASE=myapp
CELERY_BROKER_URL=redis://localhost:6379/0
KAFKA_BOOTSTRAP_SERVERS=localhost:9092
MLFLOW_TRACKING_URL=http://localhost:5000
```

---

## Extras

### SQL drivers

```bash
pip install "minix[postgresql]"   # recommended default
pip install "minix[mysql]"
pip install "minix[postgresql,mysql]"
```

### AI Capabilities

Install with AI tools (PyTorch, MLflow):

```bash
pip install "minix[ai]"
```

### Vector DB (Qdrant)

```bash
pip install "minix[vdb]"
```

### Vector DB (pgvector)

```bash
pip install "minix[pgvector]"
```

### ClickHouse Support

```bash
pip install "minix[clickhouse]"
```

### RAG Module

Install the RAG module's dependencies, PostgreSQL driver included (see
[RAG Module](#rag-module)):

```bash
pip install "minix[rag]"
```

### Install All Extras

```bash
pip install "minix[postgresql,pgvector,mysql,ai,vdb,clickhouse,rag]"
```

---

## Registry Usage

The `Registry` is a singleton-based dependency container:

```python
from minix.core.registry import Registry

# Register a service
Registry().register(MyService, MyService())

# Register with a salt (for multiple instances)
Registry().register(SqlConnector, connector, salt="analytics")

# Retrieve a service
service = Registry().get(MyService)

# Retrieve with salt
connector = Registry().get(SqlConnector, salt="analytics")
```

---

## CLI Commands

Install Minix, then use the `minix` entry point. Prefer these commands over
hand-writing project or module boilerplate.

```bash
minix --help
minix init --help
minix add module --help
```

### Quick map

| Goal | Command |
|------|---------|
| New project in cwd | `minix init APP_NAME` |
| New project with RAG | `minix init APP_NAME --rag` |
| Enable built-in RAG in an existing project | `minix add rag` |
| Set up migrations (once per project) | `minix db init` |
| Generate a migration from entity changes | `minix db makemigrations -m "message"` |
| Apply / revert migrations | `minix db migrate` / `minix db downgrade -1` |
| Inspect migrations | `minix db current` / `minix db history` |
| New feature module | `minix add module MODULE_NAME` |
| Add to existing module | `minix add controller\|task\|periodic\|helper\|consumer\|binding [NAME] --module=MODULE` |
| Refresh coding-agent guidance | `minix sync agents` (`--check` to verify) |
| Version | `minix -v` / `minix --version` |

### Global options

| Option | Short | Description |
|--------|-------|-------------|
| `--version` | `-v` | Print the installed Minix version and exit |
| `--help` | | Show CLI help |

```bash
minix -v
minix --version
```

### `minix init`

Scaffold a project in the **current directory**.

```bash
minix init APP_NAME [OPTIONS]
```

**Interactive prompt (first question, multi-select):**

```text
Which SQL database(s) do you want? (comma-separated for multiple)
  1) postgresql  (recommended) [default]
  2) mysql
Examples: 1   |   2   |   1,2
```

Press Enter for PostgreSQL only. Select `1,2` when you need both.
Pass `--db-driver postgresql`, `--db-driver mysql`, or
`--db-driver postgresql,mysql` to skip the prompt (also used when stdin is not a TTY).

SQL drivers are **optional extras** (`minix[postgresql]`, `minix[mysql]`). Docker only
installs the engines you selected.

| Argument / option | Default | Description |
|-------------------|---------|-------------|
| `APP_NAME` | *(required)* | Written to `app_name` in `config.py`, `.env`, and `.env.example` |
| `--db-driver` | prompted (`postgresql`) | One or more: `postgresql` (recommended), `mysql` |
| `--app-port` | `8000` | API port (Dockerfile `EXPOSE` / compose `app` mapping) |
| `--db-port` | `5432` / `3306` | Host port for the **primary** SQL DB (PostgreSQL preferred when both are selected) |
| `--redis-port` | `6379` | Host port for Redis |
| `--qdrant-port` | `6333` | Host port for Qdrant HTTP |
| `--qdrant-grpc-port` | `6334` | Host port for Qdrant gRPC |
| `--object-storage-port` | `9000` | Host port for MinIO API |
| `--object-storage-console-port` | `9001` | Host port for MinIO console |
| `--extras` | *(auto)* | Comma-separated PyPI extras: `postgresql`, `mysql`, `vdb`, `clickhouse`, `ai`, `rag` (SQL also set from `--db-driver`) |
| `--rag` | off | Enable RAG, Qdrant, model environment settings, and PDF support; requires PostgreSQL |
| `--no-extras` | off | Skip auto-detected non-SQL extras |

**Creates:**

| File | Purpose |
|------|---------|
| `config.py` | Explicit pydantic `Settings` (all options in one file) |
| `.env.example` | Documented env keys with copy instructions (commit this) |
| `.env` | Local overrides without the copy header (gitignored) |
| `.gitignore` | Python / IDE / dotenv ignores |
| `Dockerfile` | App image (`pip install "minix~=X.Y.Z"` / extras from PyPI) |
| `docker-compose.yml` | PostgreSQL or MySQL, Redis, MinIO, API, Celery; optional Qdrant / ClickHouse / MLflow |
| `.dockerignore` | Build context excludes |
| `AGENTS.md` | Rules for AI coding agents (CLI, decision flow, upstream-first) inside a Minix-managed block |
| `CLAUDE.md` | Imports `AGENTS.md` for Claude Code; add project rules below the managed block |
| `.claude/skills/minix/` | Claude Code skill: concept map, patterns, upstream workflow |
| `entries/` | `api.py`, `worker.py`, `beat.py` process entrypoints |
| `migrations/<connection>/` | Alembic setup per SQL connection, driven by [`minix db`](#minix-db) |

**App wiring after init** (settings, not legacy `app_connectors.py` / `app_modules.py`):

- Connectors: `DATABASES`, `OBJECT_STORAGES`, `QDRANT_CONNECTIONS`, … in `config.py`
- Modules: `INSTALLED_MODULES` list of dotted paths
- Boot: `bootstrap_from_settings()` in `entries/api.py`

**Behavior:**

- Fails with exit code `1` if `config.py` or `settings.py` already exists.
- Existing optional files (`.env`, `.gitignore`, Docker files, `entries/`) are kept
  (`.env` only gains `MINIX_SETTINGS_MODULE` when missing).
- Port flags bake defaults into `Dockerfile`, `docker-compose.yml`, `.env`, and `config.py`.
- **SQL choice** (one or more) selects compose services, pip extras
  (`minix[postgresql]` / `minix[mysql]`), Dockerfile client libs, and
  `DATABASES` entries. If both are selected, PostgreSQL is the primary
  `default` connection and MySQL is registered as `DATABASES["mysql"]`.
- **Optional extras:** Non-SQL extras can still be inferred from the current
  environment (e.g. `minix[vdb]`). Override with `--extras …` or `--no-extras`.
- **Version pin:** The Dockerfile uses a compatible release on the installed minor
  (e.g. ``minix~=0.2.3`` → latest ``0.2.x`` ≥ ``0.2.3``, not ``0.3``), capped to
  the newest version **published on PyPI** so a local unreleased build cannot
  break `docker compose build`. Prefer letting `minix init` manage the pin.

```bash
minix init my_app
minix init my_app --db-driver postgresql
minix init my_app --db-driver mysql --db-port 3307
minix init my_app --db-driver postgresql,mysql
minix init my_app --app-port 8001 --redis-port 6380
pip install "minix[vdb]"
minix init my_app --extras vdb
minix init my_app --no-extras
```

Install SQL drivers in an existing environment with:

```bash
pip install "minix[postgresql]"
pip install "minix[mysql]"
pip install "minix[postgresql,mysql]"
```

### `minix db`

Database migrations for your SQL entities, built on
[Alembic](https://alembic.sqlalchemy.org/). If you know Django, the commands
will feel familiar: change an entity, run `makemigrations`, review the file,
then run `migrate`.

Minix compares the database with the SQL entities of the modules in
`INSTALLED_MODULES`, so you never list tables by hand. Alembic ships with the
`postgresql`, `mysql` and `rag` extras.

#### Migration workflow

`minix init` already sets up migrations. For a project created another way,
run `minix db init` once.

```bash
# 1. Change an entity (or add a module with `minix add module orders`)
# 2. Generate a migration; the database must be running
minix db makemigrations -m "add orders"
#    → default: wrote migrations/default/versions/3f2a9c1b7d4e_add_orders.py

# 3. Read the generated file and fix anything autogenerate got wrong
# 4. Apply it
minix db migrate
#    → default: migrated to head

# 5. Commit the file in migrations/<connection>/versions/ with the entity change
```

On every other machine, including CI, staging and production, only run
`minix db migrate`.

Autogenerate is a draft, not a guarantee. Always read the file before you
apply it. In particular, a renamed column or table shows up as a drop plus an
add, which loses data. Change it to `op.alter_column(..., new_column_name=...)`
or `op.rename_table(...)`.

#### Migration commands

These commands do more than Alembic's equivalents:

| Command | What it does | Options |
|---------|--------------|---------|
| `minix db init` | Creates `migrations/<connection>/` for each PostgreSQL / MySQL connection. Existing files are kept. | `--database/-d NAME` (repeatable; default: every PostgreSQL / MySQL connection) |
| `minix db makemigrations` | Compares the entities with the live database and writes a migration. Writes nothing when there are no changes. | `-m/--message`, `--database/-d`, `--empty` (blank migration to write by hand) |
| `minix db migrate [REVISION]` | Applies migrations, up to `head` by default. | `--database/-d` |
| `minix db downgrade REVISION` | Reverts migrations. | `--database/-d` |

These run the Alembic command of the same name. Minix picks the connection and
loads `config.py` and `.env`; every other argument goes to Alembic unchanged,
and `minix db <command> --help` shows Alembic's own help:

| Command | What it does | Example |
|---------|--------------|---------|
| `current` | Shows the revision the database is at | `minix db current` |
| `history` | Lists migrations, newest first | `minix db history -v`, `minix db history -r-3:current` (no space after `-r`) |
| `heads` | Shows the latest revision(s); more than one means history has split | `minix db heads` |
| `branches` | Shows where history splits | `minix db branches` |
| `show` | Shows one migration | `minix db show 3f2a` |
| `check` | Exits non-zero when entities have changes without a migration | `minix db check` |
| `stamp` | Records a revision in the database without running anything | `minix db stamp head` |
| `merge` | Writes a migration that joins two heads | `minix db merge -m "merge" heads` |
| `edit` | Opens a migration in `$EDITOR` | `minix db edit head` |

All commands accept `--database/-d NAME` (repeatable). Without it, they run for
every connection that has a `migrations/` directory, with an `== name ==`
header per connection when there are several.

#### Revision targets

`migrate`, `downgrade`, `stamp` and `show` take a revision:

| Target | Meaning |
|--------|---------|
| `head` | The latest migration |
| `heads` | Every head, when history has split |
| `base` | Before the first migration (`downgrade base` reverts everything) |
| `3f2a9c1b7d4e` or `3f2a` | A revision id, or any unique prefix of one |
| `-1`, `-2` | Steps back from the current revision (`downgrade`) |
| `+1`, `+2` | Steps forward from the current revision (`migrate`) |

#### Migration recipes

**Write a data migration.** Generate an empty migration and fill in
`upgrade()` and `downgrade()`:

```bash
minix db makemigrations --empty -m "backfill order status"
```

```python
def upgrade() -> None:
    op.execute("UPDATE orders SET status = 'NEW' WHERE status IS NULL")


def downgrade() -> None:
    pass
```

**Catch missing migrations in CI.** With the database migrated to head, this
fails when someone changed an entity without running `makemigrations`:

```bash
minix db migrate && minix db check
```

**Adopt migrations on a database that already has its tables.**
`makemigrations` compares with the live database, so against a database that
already has the tables it finds nothing to write. Generate the first migration
against an empty database, then mark existing databases as up to date:

```bash
# Against an empty local database
minix db makemigrations -m "initial"
minix db migrate

# Against each database that already has the tables (staging, production)
minix db stamp head
```

**Resolve two migrations from parallel branches.** When two branches each add a
migration, `migrate` and `makemigrations` fail with a "multiple heads" error.
Join them, then migrate:

```bash
minix db heads                    # shows both
minix db merge -m "merge" heads
minix db migrate
```

**Remove a module.** Its tables stay in the database, because Minix never drops
tables that no installed module owns. To drop them, write the drop in an empty
migration (`makemigrations --empty`, then `op.drop_table("orders")`).

**Use several databases.** Each `settings.DATABASES` connection gets its own
`migrations/<connection>/` and only manages the tables of entities bound to it
with `add_binding(..., connection="analytics")`:

```bash
minix db init -d analytics
minix db makemigrations -m "add events" -d analytics
minix db migrate                  # every connection
minix db migrate -d analytics     # one connection
```

**Preview the SQL instead of running it.** Use Alembic directly with `--sql`:

```bash
alembic -c migrations/default/alembic.ini upgrade head --sql
```

**Run in Docker.** Migrate before the new app version starts serving:

```bash
docker compose run --rm app minix db migrate
```

#### How migrations work

- `migrations/<connection>/` holds `alembic.ini`, `env.py`, `script.py.mako`
  and `versions/`. `env.py` calls `minix.core.migrations.run_migrations`, which
  loads `.env` and `config.py`, so plain `alembic -c migrations/<connection>/alembic.ini …`
  works on the same files.
- Each connection compares only the tables of the entities bound to it. Tables
  that no installed module owns, including other connections' tables, are never
  touched.
- The `default` connection records its revision in `alembic_version`; others use
  `alembic_version_<connection>`, so two connections can share one database.
- On PostgreSQL, generated downgrades also drop the enum types of the tables
  they drop, so a downgrade followed by `migrate` works.
- ClickHouse connections are skipped, because Alembic does not support them.

#### Migration troubleshooting

| Message | Fix |
|---------|-----|
| `No config.py found` | Run the command in the project root. |
| `Alembic is not installed` | `pip install "minix[postgresql]"` (or `mysql`). |
| `No migrations set up` / `No migrations for 'x'` | `minix db init`, or `minix db init -d x`. |
| `Target database is not up to date` | `minix db migrate`, then run `makemigrations` again. |
| `Multiple head revisions are present` / `Multiple heads are present` | `minix db merge -m "merge" heads`, then `minix db migrate`. |
| Connection refused | Start the database (`docker compose up -d`) and check `DB_*` in `.env`. |
| `type "…" already exists` on PostgreSQL | An enum type was left behind by a downgrade. Drop it in the migration (`op.execute('DROP TYPE IF EXISTS "name"')`), or by hand. |
| `type "…" does not exist` after adding an enum column | Alembic creates enum types with new tables but not with new columns. Add `sa.Enum(..., name="name").create(op.get_bind(), checkfirst=True)` before `op.add_column(...)`. |

### `minix add module`

Scaffold a feature module under `src/modules/<name>/`.

```bash
minix add module MODULE_NAME [OPTIONS]
```

**Default stack:** entity + repository + service + controller.
The SQL trio is wired with a single `add_binding(...)` in `module.py`.
Tasks / helpers / consumers / periodic tasks are **opt-in**.

| Piece | Example for `orders` |
|-------|----------------------|
| Entity | `Order` → `entities/order_entity.py` |
| Repository | `OrderSqlRepository` |
| Service | `OrderSqlService` |
| Controller | `OrderController` |
| Wiring | `module.py` → `.add_binding(...)` + `.add_controller(...)` |
| Package export | `OrdersModule` from `__init__.py` |

#### SQL binding (all or none)

Entity, repository, and service are **one unit**. The CLI never generates a
partial binding:

- Default: all three are created.
- `--no-binding`: skip all three.
- `--entity=` / `--repository=` / `--service=` (or `-e` / `-r` / `-s`) only
  rename; missing pieces are still filled so the binding stays complete.
- Combining `--no-binding` with any of those flags is an error.
- There are no `--no-entity` / `--no-repository` / `--no-service` flags.

#### Flags

| Flag | Short | Default | Description |
|------|-------|---------|-------------|
| `--entity[=Name]` | `-e` | on (with binding) | Rename the binding entity (`Order` for `orders`) |
| `--repository[=Name]` | `-r` | on (with binding) | Rename the binding repository |
| `--service[=Name]` | `-s` | on (with binding) | Rename the binding service |
| `--controller[=Name]` | `-c` | on | HTTP controller |
| `--task[=Name]` | `-t` | off | Async Celery `Task` |
| `--helper[=Name]` | `-H` | off | `HelperService` |
| `--consumer[=Name]` | | off | Kafka `AsyncConsumer` |
| `--periodic[=Name]` | `-p` | off | `PeriodicTask` |
| `--all` | `-a` | off | Default stack + task, helper, consumer, periodic |
| `--no-binding` | | off | Skip entity + repository + service together |
| `--no-controller` | | off | Skip controller only |
| `--path DIR` | | `src/modules/<name>` | Package directory |
| `--force` | `-f` | off | Overwrite existing files |
| `--register` / `--no-register` | | register | Append to `config.py` `INSTALLED_MODULES` |

#### Naming rules

- Long flags accept an optional value with `=`: `--controller=ShopController`, `--task=ShipOrder`.
- Short flags (`-c`, `-t`, …) are **presence-only** — do not pass a separate value after them.
- Safe: `minix add module orders -t --no-register`
- Avoid: `minix add module orders -t ShipOrder` (use `--task=ShipOrder`)

```bash
# Default CRUD/API stack + auto-register in config.py
minix add module orders

# Custom class names (binding still complete)
minix add module orders --entity=ShopOrder --controller=ShopController

# Background task add-on
minix add module orders -t
minix add module orders --task=FulfillOrder -H

# Everything
minix add module orders -a

# Worker-oriented: no HTTP controller
minix add module orders --no-controller -t

# No SQL binding (e.g. helper/consumer-only style module)
minix add module orders --no-binding -H --consumer=OrderEventsConsumer
```

#### Module layout

```
src/modules/<name>/
  __init__.py          # exports <Name>Module
  module.py            # BusinessModule + add_binding / add_*
  entities/            # present when binding is on
  repositories/
  services/
  controllers/         # optional
  tasks/               # optional (Task + PeriodicTask)
  consumers/           # optional
```

#### After scaffolding

1. Confirm `INSTALLED_MODULES` contains e.g. `"src.modules.orders.OrdersModule"`.
2. Implement fields on the entity, then `minix db makemigrations -m "add orders" && minix db migrate`.
3. Expand service / controller as needed.
4. Restart API / worker.

```python
from src.modules.orders import OrdersModule
```

Cross-module links: put the FK on the owning entity and resolve the other side with
`Registry().get(OtherService)` in services (do not merge unrelated entities into one
module only because they relate).


### Adding to an existing module

To add a piece you skipped at creation time (for example, after
`--no-controller`), use `minix add <component> [NAME] --module=MODULE`. The CLI writes the
file, exports it from the subpackage `__init__.py`, and appends the
matching `.add_*(...)` call to the `BusinessModule` chain in `module.py`.
Existing code is not rewritten. Running the command again does nothing,
because the file is kept and the call is already wired.

| Command | Generates | Wiring |
|---------|-----------|--------|
| `minix add controller -m MODULE` | `controllers/<x>_controller.py` | `.add_controller(...)` |
| `minix add task -m MODULE` | `tasks/<x>_task.py` | `.add_task(...)` |
| `minix add periodic -m MODULE` | `tasks/<x>_periodic_task.py` | `.add_periodic_task(...)` |
| `minix add helper -m MODULE` | `services/<x>_helper_service.py` | `.add_helper_service(...)` |
| `minix add consumer -m MODULE` | `consumers/<x>_consumer.py` | `.add_consumer(...)` |
| `minix add binding -m MODULE` | entity + repository + service | `.add_binding(...)` |

`--module/-m` (required) names the target module. The optional `NAME`
argument sets a custom class name (`binding` takes `--entity`,
`--repository`, `--service` instead). `--path DIR` points to a non-default
package directory, and `--force/-f` overwrites the component file.
A controller added to a module that already has a binding uses that
binding's service.

```bash
minix add module orders --no-controller
minix add controller --module=orders             # OrderController
minix add controller OrderAdmin --module=orders  # OrderAdminController
minix add task ShipOrder -m orders               # ShipOrderTask
minix add binding -m payments --entity=Payment
```

### Coding agents

`minix init` writes guidance for AI coding agents: `AGENTS.md`, `CLAUDE.md` and the
Claude Code skill in `.claude/skills/minix/`. Minix owns the text between the
`<!-- BEGIN MINIX -->` / `<!-- END MINIX -->` markers and the skill directory; anything
you write outside the markers is kept. The guidance tells agents to build with the CLI
and Minix's base classes, and to add missing generic capabilities to Minix instead of
the project.

After upgrading Minix, refresh it so the guidance matches the installed version:

```bash
minix sync agents            # write
minix sync agents --check    # exit 1 if stale (for CI)
```

---

## License

Minix is licensed under the **MIT License**. See the `LICENSE` file for more details.

---

## Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

---

## Author

**AmirHossein Advari** - [amiradvari@gmail.com](mailto:amiradvari@gmail.com) \
**Shirin Dehghani** - [shirin.dehghani1996@gmail.com](mailto:shirin.dehghani1996@gmail.com) \
**Parsa Mohammadpour** - [parsa.mohammadpour01@gmail.com](mailto:parsa.mohammadpour01@gmail.com)
**Emad Sudani** - [sudani.emad@gmail.com](mailto:sudani.emad@gmail.com)

