Metadata-Version: 2.4
Name: django-vpg
Version: 0.1.0
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: Django
Classifier: Framework :: Django :: 5.2
Classifier: Framework :: Django :: 6.0
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Rust
Classifier: Topic :: Database
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Dist: django>=5.0
Requires-Dist: psycopg>=3.1
Requires-Dist: django-async-backend>=6.0.5 ; extra == 'async'
Provides-Extra: async
License-File: LICENSE
Summary: Rust-powered PostgreSQL driver and Django database backend.
Author-email: David Burke <david@burkesoftware.com>
Requires-Python: >=3.12
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Bug Tracker, https://gitlab.com/glitchtip/django-vpg/-/issues
Project-URL: Homepage, https://gitlab.com/glitchtip/django-vpg

# django-vpg

> **Status: Experimental.** Freshly extracted from
> [glitchtip-rust](https://gitlab.com/glitchtip/glitchtip-rust), where the
> driver runs GlitchTip in production — but this standalone package is new,
> not yet on PyPI, and its API and packaging may change without notice.
> Evaluate freely; don't bet production on it yet.

A Rust-powered PostgreSQL driver and Django database backend.

django-vpg replaces the connection/cursor IO layer of Django's stock
PostgreSQL backend with a Rust core (tokio-postgres + deadpool, via PyO3):
swap one `ENGINE` line, keep your schema, migrations, and ORM code
unchanged.

Part of the **django-v\*** family — [django-vcache], [django-vtasks],
django-vpg — small, fast, Rust-core Django infrastructure maintained by the
[GlitchTip] team. django-vcache and django-vtasks are battle-tested in
production at glitchtip.com; django-vpg is the newest member. (The *v*
originally stood for Valkey; with three packages it now just marks the
family.)

[django-vcache]: https://gitlab.com/glitchtip/django-vcache
[django-vtasks]: https://gitlab.com/glitchtip/django-vtasks
[GlitchTip]: https://glitchtip.com

## Why

- **CPU and RSS.** Protocol parsing, row decoding, and type marshalling
  run in Rust; only final Python values are materialized — intermediate
  protocol buffers and COPY encoding never traverse the Python heap. In
  the GlitchTip team's A/B benchmarks against psycopg (production-shaped
  ingest workloads, replayed locally), the driver holds resident memory
  flat under heavy-tailed payloads where libpq's buffers ratchet
  (2.5 vs 21 MB growth per 10k events), and wins per-op CPU on jsonb
  reads, pipelined batch inserts, and COPY event batches.
- **`fetch_models` fusion.** Row → model-instance materialization can run
  as a single driver call instead of a per-row Python loop.
- **COPY at native speed.** `COPY FROM STDIN` with batch text-format
  encoding in Rust (`encode_copy_chunk`).
- **psycopg-shaped errors.** Full PEP 249 exception hierarchy with
  `sqlstate` and `Diagnostics`, so existing error-handling code ports
  cleanly.
- **Server-side cancellation.** Cancelled queries fire a PostgreSQL
  `CancelRequest` on a fresh connection and the pooled connection returns
  clean — no poisoned pool state, no abandoned server work.
- **Real pooling.** deadpool checkout/return with warmup, wait timeouts,
  and jittered max-lifetime recycling; prepared-statement cache with
  psycopg3-style admission (`prepare_threshold` / `prepared_max`);
  pgbouncer mode (`OPTIONS={"pgbouncer": True}`, which disables prepared
  statements).
- **libpq-parity TLS.** `sslmode` (`disable`/`allow`/`prefer`/`require`/
  `verify-ca`/`verify-full`), `sslrootcert`, client certificates.

## Install

```sh
pip install django-vpg          # sync ORM + async driver API
pip install django-vpg[async]   # + async ORM via django-async-backend (Django 6+)
```

Note: **psycopg (the library) must be installed** — it is a declared
dependency. Django's stock `postgresql` backend, which django-vpg
subclasses for schema/introspection/operations, imports it at load time.
The IO path never uses it.

## Quickstart

```python
DATABASES = {
    "default": {
        "ENGINE": "django_vpg.backend",
        "NAME": "mydb",
        "USER": "postgres",
        "PASSWORD": "...",
        "HOST": "db.example.com",
        "PORT": "5432",
        # Optional driver OPTIONS (libpq-parity naming; psycopg3-compatible
        # pool dict):
        # "OPTIONS": {
        #     "pool": {"max_size": 20, "min_size": 2},   # default max_size: 10
        #     "sslmode": "verify-full",
        #     "sslrootcert": "/path/ca.pem",
        # },
    }
}
```

Supported `OPTIONS` keys: `pool` (psycopg3-compatible dict — `max_size`,
`min_size`, `timeout`, `max_lifetime`, `max_idle`, `name` — or `False` for
a dedicated single-slot connection per Django connection, e.g. under test
runners), `pool_size` (shorthand for `pool={"max_size": N}`; default 10),
`pgbouncer`, `isolation_level`, `application_name`, `connect_timeout`,
`keepalives`, `keepalives_idle`, `prepare_threshold`, `prepared_max`,
`sslmode`, `sslrootcert`, `sslcert`, `sslkey`, and `server_settings`.
Unknown keys warn once and are dropped.

## The async story (three layers)

1. **Sync ORM — stock Django, zero extra deps.** Works everywhere Django
   works.
2. **Async driver API — real asyncio, works anywhere.** The raw driver
   (`django_vpg._driver.RustPgDriver`) exposes `await`-able query,
   execute, transaction, and COPY operations with no dependency on any
   async framework beyond asyncio itself; driver awaitables interoperate
   with `gather`, `wait_for`, `shield`, and cancellation.
   (`django_vpg.dbapi` itself is the *sync* DB-API 2.0 shim the Django
   engine builds on.)
3. **Async ORM — via [django-async-backend].** Installing the
   `django-vpg[async]` extra activates `AsyncDatabaseWrapper`, which plugs
   into the `async_connections` framework. Without it, the backend cleanly
   degrades to sync-only (no ImportError).

Unifying Django's `connections` and `async_connections` (transaction
management and isolation across both worlds) is active upstream work in
django-async-backend that this package tracks closely.

[django-async-backend]: https://github.com/Arfey/django-async-backend

## The tokio runtime

The driver runs on a fork-safe shared tokio runtime owned by the package
(1 worker thread by default; `VPG_TOKIO_WORKER_THREADS` overrides). Prefork
servers are safe: the runtime is rebuilt per child on first use.

Rust embedders get the same hook django-vcache offers: link `vpg-pyo3` as
an rlib, re-export its pyclasses into your own pymodule, and install a
process-wide runtime with `vpg_core::set_runtime_provider(my_get_runtime)`
so one tokio pool serves every driver in the process. This mirrors how
[glitchtip-rust] embeds its driver today (rlib link + re-exported
pyclasses + one shared runtime); gt_rust will switch to linking vpg-pyo3
and installing its runtime via this hook.

[glitchtip-rust]: https://gitlab.com/glitchtip/glitchtip-rust

## Status

Extracted from [glitchtip-rust] (v0.6.1), where this driver is the only
database engine in GlitchTip's backend (as of July 2026). Pre-1.0 while
the standalone packaging settles; the driver core predates this package
and carries its full test suite (including an in-process PG wire-server
harness) with it.

Known naming wart: the vendored tokio-postgres buffer-cap patch reads the
`GT_PG_BUF_CAP_BYTES` env var (512 KiB default retained-buffer cap); it
will be renamed with a compatibility window before 1.0.

## Development

```sh
./scripts/check.sh        # fmt + clippy -D warnings + cargo test + ruff
                          # (includes tests/wire.rs — an in-process scripted
                          #  PG wire server; no live postgres needed)

# Python suite (needs maturin + a live postgres for the marked tests):
python -m venv .venv && source .venv/bin/activate
pip install maturin pytest pytest-asyncio django psycopg django-async-backend
maturin develop --release
VPG_TEST_DSN=postgres://postgres@localhost:5432/postgres python -m pytest tests/
```

The workspace is two crates: `crates/vpg-core` (pure Rust, no pyo3,
`cargo test`-able) and `crates/vpg-pyo3` (thin bindings, `cdylib + rlib`).
`vendor/tokio-postgres` carries a minimal retained-buffer-capacity patch —
see `vendor/tokio-postgres/GT_PATCH.md` for rationale and the re-audit
procedure on version bumps.

## License

MIT

