Metadata-Version: 2.4
Name: django-vtasks
Version: 3.0.0
Summary: A very fast valkey/postgres django tasks backend.
Project-URL: Source, https://gitlab.com/glitchtip/django-vtasks
Project-URL: Tracker, https://gitlab.com/glitchtip/django-vtasks/-/issues
License-Expression: MIT
License-File: LICENSE
Keywords: background,django,postgres,tasks,valkey
Classifier: Framework :: Django
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Python: >=3.12
Requires-Dist: croniter>=2.0.4
Requires-Dist: django>=6.0
Requires-Dist: orjson>=3.11.4
Provides-Extra: metrics
Requires-Dist: prometheus-client>=0.17.0; extra == 'metrics'
Provides-Extra: valkey
Requires-Dist: django-vcache>=3.0.0; extra == 'valkey'
Requires-Dist: pyzstd; (python_version < '3.14') and extra == 'valkey'
Description-Content-Type: text/markdown

# django-vtasks

**Valkey Tasks. Very Fast Tasks.**

From the team at [GlitchTip](https://glitchtip.com), `django-vtasks` is a lightweight, async-first task queue for Django 6.0+.

**Status**: Newly feature complete. Beta quality. Use and report bugs.

## Why django-vtasks?

- **Async-first** - Native `asyncio` worker for high-performance I/O
- **Flexible backends** - Start with Postgres, scale to Valkey without rewriting code
- **Lightweight** - Minimal dependencies, modern codebase
- **Embedded mode** - Run tasks in your ASGI server or as standalone workers

## Features

- Dual backends: Database (Postgres/SQLite/MySQL) and Valkey (Redis-compatible)
- Scheduled tasks with cron syntax
- Delayed tasks (`run_after`)
- Unique tasks (Mutex and Throttle patterns)
- Per-queue concurrency limits (isolate heavy tasks from cheap ones in one worker)
- Batch processing for high-throughput queues
- Prometheus metrics
- Django admin interface for task management

![Admin Interface](/docs/admin.png)

## Requirements

- Python 3.12+
- Django 6.0+
- Valkey 7+ (or Redis 7+) for Valkey backend

## Quick Start

```bash
pip install django-vtasks

# For Valkey backend support:
pip install "django-vtasks[valkey]"
```

```python
# settings.py
INSTALLED_APPS = ["django_vtasks", "django_vtasks.db"]

TASKS = {
    "default": {
        "BACKEND": "django_vtasks.backends.db.DatabaseTaskBackend",
    }
}
```

```python
# myapp/tasks.py
from django_vtasks import task

@task
def send_email(user_id):
    # Your task logic
    pass
```

```python
# In your views
send_email.enqueue(user_id)
# or async
await send_email.aenqueue(user_id)
```

```bash
# Run the worker
python manage.py runworker
```

## Queue configuration

`VTASKS_QUEUES` declares the queues a worker consumes. Use a plain list for the
simple case, or a dict to give each queue its own options:

```python
# settings.py
# Every concurrency limit is PER WORKER (per process). The fleet-wide cap is
# the limit times your worker count.
VTASKS_CONCURRENCY = 50            # global pool / default per-queue limit, per worker

VTASKS_QUEUES = {
    "default": {},                              # shares the global pool
    "cold_storage": {"worker_concurrency": 3},  # at most 3 at once *per worker*
    "emails": {"batch": {"count": 100, "timeout": 5.0}},
}

# simple form still works:
# VTASKS_QUEUES = ["default", "cold_storage"]
```

```python
@task(queue_name="cold_storage")
def compact_parquet(org_id):
    ...
```

**Per-queue concurrency.** Because vtasks is async-first, a high
`VTASKS_CONCURRENCY` is ideal for cheap I/O-bound tasks — but a few heavy
CPU/RAM-bound tasks (analytics, image processing, data exports) at that same
concurrency can exhaust memory or a connection pool. A queue with its own
`worker_concurrency` gets a dedicated semaphore; every other queue shares the
global `VTASKS_CONCURRENCY` pool, so a saturated capped queue never blocks the
rest. Every limit is **per-worker (per-process)** — the right scope for bounding
per-pod resources like memory or connections — so the fleet-wide ceiling is the
limit times your worker count (e.g. `worker_concurrency: 3` across 4 workers
allows up to 12 concurrent `cold_storage` tasks cluster-wide). The key name
spells this out so it's unambiguous where it's set.

**Batching.** Declare a `batch` option on a queue (see `emails` above) to
collect up to `count` tasks (waiting at most `timeout` seconds) and hand them to
the task as a list.

## Delayed tasks

Pass `run_after` to defer execution until a given time (it's a "not before"
guarantee, accurate to about a second — not a precise scheduler):

```python
from datetime import timedelta
from django.utils import timezone

await send_email.using(run_after=timezone.now() + timedelta(minutes=10)).aenqueue(user_id)
```

## Enqueuing from other languages

Valkey is the broker and the integration boundary: any producer that speaks the
wire protocol can enqueue tasks a vtasks worker will run — no vtasks dependency
required (e.g. a Rust hot path pushing straight to Valkey). The contract — key
layout, payload schema, priority, and unique/delayed semantics — is specified in
[PROTOCOL.md](PROTOCOL.md).

## Performance

Benchmarks simulate async Django views dispatching tasks — the real ASGI use case.

| Scenario (Sleep 10ms) | Enqueue (ops/s) | Process (ops/s) | Peak RSS (MB) | Valkey Conns |
|---|---|---|---|---|
| **VTasks** | 5,203 | 3,796 | 76 | 3 |
| **Celery Threads** | 2,228 | 894 | 123 | 11 |
| **RQ** | 436 | 25 | 170 | 4 |

VTasks: **4x faster processing, 2x faster enqueue, 38% less memory** vs Celery.

See [Benchmarks](https://django-vtasks.glitchtip.com/benchmarks/) for full methodology, cloud latency results, and how to reproduce.

## Documentation

Full documentation is available at **[django-vtasks.glitchtip.com](https://django-vtasks.glitchtip.com)**

- [Getting Started](https://django-vtasks.glitchtip.com/getting-started/)
- [Guide](https://django-vtasks.glitchtip.com/guide/) - Unique tasks, batching, scheduling, and more
- [Configuration](https://django-vtasks.glitchtip.com/configuration/) - All settings reference
- [Deployment](https://django-vtasks.glitchtip.com/deployment/) - Standalone workers, embedded mode, Kubernetes
- [Benchmarks](https://django-vtasks.glitchtip.com/benchmarks/) - Performance comparison with Celery

## Contributing

We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.

---

*Built by the [GlitchTip](https://glitchtip.com) team.*
