Metadata-Version: 2.4
Name: django-ragtime
Version: 0.1.1
Summary: Chunked pgvector embeddings for Django — the retrieval half of RAG.
Author-email: Dan Mann <damann1072@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/Dan-the-Mann/django-ragtime
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Django
Classifier: Framework :: Django :: 4.2
Classifier: Framework :: Django :: 5.0
Classifier: Framework :: Django :: 5.1
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: Django>=4.2
Requires-Dist: pgvector>=0.3
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-django; extra == "dev"
Requires-Dist: psycopg[binary]; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Dynamic: license-file

# django-ragtime

Chunked, pgvector-backed embeddings for Django — built for the retrieval half
of RAG. Text is split into chunks, each chunk is embedded, and the vectors
live in a separate `EmbeddingChunk` table so your models' tables stay
untouched.

## Requirements

- PostgreSQL with the [pgvector](https://github.com/pgvector/pgvector) extension

## Installation

```bash
pip install django-ragtime
```

Add the app (and `contenttypes`, which chunks depend on) to your settings and
run its migrations. The first migration also enables the pgvector extension,
so the database user running `migrate` needs permission to `CREATE EXTENSION`
(otherwise, enable it manually first).

```python
INSTALLED_APPS = [
    # ...
    "django.contrib.contenttypes",
    "django_ragtime",
]
```

```bash
python manage.py migrate django_ragtime
```

## Usage

Add `EmbeddingMixin` to any of your existing models:

```python
from django.db import models
from django_ragtime.models import EmbeddingMixin


class Article(EmbeddingMixin, models.Model):
    title = models.CharField(max_length=200)
```

The mixin adds no database columns, so your models need no new migrations.
Embeddings live in `EmbeddingChunk` rows attached to your instances via a
generic foreign key; they are deleted automatically when the parent instance
is deleted.

To build (or rebuild) an instance's embeddings, pass `update_chunks()` a
callable that embeds a batch of texts:

```python
def embed_texts(texts: list[str]) -> list[list[float]]:
    """One batched call to your embedding API."""
    response = client.embeddings.create(model="text-embedding-3-small", input=texts)
    return [item.embedding for item in response.data]


article.update_chunks(embed_texts, model_name="text-embedding-3-small")
```

`update_chunks()` splits the instance's text into chunks (paragraph-aware,
controlled by the `chunk_size` and `chunk_overlap` class attributes, in
characters), embeds them with your callable, and atomically replaces the
instance's existing chunks. `article.embedding_chunks` returns them.

### Models you don't own

You don't have to inherit the mixin. The same storage layer works with any
saved instance — `User`, a model from another package, etc.:

```python
from django.contrib.auth.models import User
from django_ragtime import register, update_chunks_for

# In your AppConfig.ready(), so chunks are deleted with the parent:
register(User, fields=["first_name", "last_name"])

update_chunks_for(user, embed_texts, model_name="text-embedding-3-small")
```

Pass `fields` (or `text=`) whenever you call `update_chunks_for` if you
haven't registered defaults. Auto-introspection skips emails, slugs, URLs,
and names like `password`, but listing fields is the safe choice for
third-party models.

```python
update_chunks_for(user, embed_texts, fields=["first_name", "last_name"])
update_chunks_for(user, embed_texts, text=user.get_full_name())
```

`chunks_for(user)` and `delete_chunks_for(user)` are the function equivalents
of `instance.embedding_chunks` and deleting those rows.

### Similarity search

Search chunks directly, or get the best chunk per parent object:

```python
from django_ragtime.models import EmbeddingChunk

# All matching chunks, nearest first, annotated with .distance
EmbeddingChunk.objects.search(query_vector)

# Restrict to one or more parent model classes
EmbeddingChunk.objects.search_models(query_vector, Article)
EmbeddingChunk.objects.search_models(query_vector, Article, Note)

# Same idea via chaining
EmbeddingChunk.objects.for_models(Article, Note).search(query_vector)

# At most one (best) chunk per parent object; parent via chunk.parent
for chunk in EmbeddingChunk.objects.for_model(Article).nearest_parents(query_vector, limit=10):
    print(chunk.parent, chunk.distance, chunk.text)
```

### Getting the text to embed

`get_embedding_text()` returns the text an instance is embedded from. By
default it concatenates every free-text field on the model
(`CharField`/`TextField`), skipping slugs, URLs, emails, file paths, and
choice fields. To control which fields are used (and their order), declare
`embedding_fields` — dotted paths across relations and zero-argument
methods/properties are supported:

```python
class Note(EmbeddingMixin, models.Model):
    embedding_fields = ["subject", "body", "article.title"]
```

For full control (templates, related-object chunks, etc.), override the
method:

```python
def get_embedding_text(self) -> str:
    return f"{self.title}\n{self.render_summary()}"
```

## Development

```bash
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
```

The tests need Postgres with pgvector; a ready-made container is included:

```bash
docker compose up -d
pytest
```

The database connection can be overridden with the `POSTGRES_DB`,
`POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_HOST`, and `POSTGRES_PORT`
environment variables (defaults match `docker-compose.yml`, port 5434).

## License

MIT
