Metadata-Version: 2.4
Name: django-fastmig
Version: 0.1.1
Summary: Experimental drop-in add-on that makes Django `migrate` several times faster on large projects without changing the SQL it executes
Author: Viktor Johansson
License: BSD-3-Clause
Project-URL: Homepage, https://github.com/viktor2097/django-fastmig
Project-URL: Issues, https://github.com/viktor2097/django-fastmig/issues
Keywords: django,migrations,performance
Classifier: Development Status :: 3 - Alpha
Classifier: Framework :: Django
Classifier: Framework :: Django :: 4.2
Classifier: Framework :: Django :: 5.0
Classifier: Framework :: Django :: 5.1
Classifier: Framework :: Django :: 5.2
Classifier: Framework :: Django :: 6.0
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Database
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: django>=4.2
Dynamic: license-file

# django-fastmig

**Experimental.** fastmig is a drop-in add-on for Django (tested with 4.2, 5.2
and 6.0) that makes `migrate` on large projects several times faster without
changing the SQL that gets executed.

```
pip install django-fastmig
```

```python
INSTALLED_APPS = ["fastmig", *INSTALLED_APPS]   # or: import fastmig; fastmig.install()
```

`FASTMIG_DISABLE=1` turns it off. Pure Python, no settings, no changes to your
migrations, models or database.

## Results

Fresh database; every DDL statement the schema editor executes is logged and
compared between the two runs.

| project | models / migrations | backend | stock | fastmig | speedup | DDL |
|---|---|---|---|---|---|---|
| "Project A" — real in-house inventory/ERP app (taggit, modeltranslation, auditlog, pgtrigger, 49 data migrations) | 222 / 493 | PostgreSQL 15 | 154–178 s | 17–22 s | **8–9.5×** | identical (22 557 statements) |
| Project A, `pytest --create-db`, one app | | PostgreSQL 15 | 170 s | 23 s | 7× | |
| Project A, **reverse**: `migrate <app> zero` for all 22 apps | 222 / 493 | PostgreSQL 15 | 851 s | 39 s | **21×** | identical (15 039 statements) |
| generated, 400 models | 400 / 818 | SQLite | 313–364 s | 57 s | 5.5–6.4× | identical (11 262) |
| generated, 120 models | 120 / 223 | SQLite / PostgreSQL | 17–20 s | 4–5 s | 3.5–4.6× | identical |
| generated corner cases (MTI, proxies, O2O pks, `to_field`, explicit `through`, renames, pk type changes) | 53 / 59 | SQLite / PostgreSQL | 1.5 / 3.0 s | 0.7 / 2.2 s | 2× / 1.4× | identical |

Reverse migrations gain the most: before unapplying, Django recomputes the
project state for every migration in the plan by cloning and mutating it,
which repeats the whole re-rendering cost on top of the forward pass.

What remains is mostly the database executing DDL and the class renders Django
still needs (including a full re-render before every `RunPython`, same as
stock). Scripts and generators are in [`benchmarks/`](benchmarks/).

Django's own test suite passes with fastmig installed: the `migrations` and
`schema` suites (1 033 tests) on both SQLite and PostgreSQL, plus 1 435 tests
from related labels (`migrate_signals`, `contenttypes_tests`, `auth_tests`,
`proxy_models`, `model_inheritance`, `swappable_models`, `backends`, …).

## Why `migrate` is slow

The schema editor works on rendered model classes. After every operation
Django rebuilds ("fakes") classes with `type(...)` — expensive pure-Python work
(`ModelBase.__new__`, field cloning via `deconstruct()`, `contribute_to_class`,
M2M through models). For any relational change stock Django rebuilds the
model's **entire connected component**; on a connected schema that is
O(models) class builds per operation. The 120-model project above rebuilds
37 080 classes over 223 migrations; fastmig rebuilds 1 163. See the forum
thread [Very slow migrations with large numbers of tables](https://forum.djangoproject.com/t/very-slow-migrations-with-large-numbers-of-tables/29038).

## How fastmig differs from the "shallow reload" patch in that thread

That patch (`SHALLOW_RELOAD_MIGRATION`) reloads only the mutated model. It gets
the speed, but models that point at the re-rendered one keep pointing at the
old class object — reverse relations and FK targets can land on the wrong
class, which is what Django's maintainers warned about. It usually works
because schema editors mostly read table/column names, which are the same on
the stale class.

fastmig:

* re-renders the mutated model **plus** what genuinely needs a new class:
  subclasses/proxies, targets of added/removed/altered relations, and — when a
  referenced field, pk or relation changed — direct predecessors, propagating
  through pk-type dependencies (MTI, O2O primary keys, `to_field`);
* **re-links** every other model's forward pointers to the new class through
  Django's own `do_related_class()`, clearing cached properties and following
  auto-created through tables;
* preserves the pre-change class that `RenameModel` / `AlterField` on a pk read
  from `from_state` (stock only gets it by accident of traversal order);
* unregisters stale auto-created through classes;
* re-renders instead of re-linking for third-party relation fields it doesn't
  know (e.g. taggit's `TaggableManager`);
* keeps Django's `from_state` contract (#24225, #24573).

The plan is computed from `ModelState` objects, never from rendered classes.

## What it patches

`fastmig.install()` (run by the `AppConfig`) monkey-patches
`django.db.migrations.state.ProjectState`: `reload_model`/`reload_models`
(planner + re-link), `add_field`/`remove_field`/`alter_field`/`add_model`/
`rename_model`/`remove_model` (hints about what changed), and `clone`
(cheaper, equivalent). `uninstall()` restores everything. `fastmig.stats`
counts reloads, renders and re-linked fields.

## Caveats

* Validated on SQLite and PostgreSQL only; not on MySQL/Oracle.
* Third-party operations that mutate a model's `ModelState` and call
  `state.reload_model()` for it (pgtrigger, `django.contrib.postgres`) work
  unchanged. Untested: an operation that mutates model X but only worked
  because stock's component-wide re-render rebuilt X as a side effect of
  reloading another model. I know of no package doing this.
* Some operations emit several independent DDL statements (e.g. altering the
  FK column in each table referencing a changed pk). Django orders them by
  model-registry order, which is hash-seed dependent — stock Django itself
  emits them in varying order between runs. The benchmark scripts treat a pure
  reordering of otherwise identical statements as a match.
* If anything looks off: `FASTMIG_DISABLE=1`, and `benchmarks/bench_real.py`
  gives you a full stock-vs-fastmig DDL diff for your project.

## Development

```
pip install -e . django
python -m unittest discover -s tests -v                       # unit + SQL parity tests
python benchmarks/gen_project.py benchmarks/demo --apps 8 --models 15 --rounds 30
python benchmarks/bench.py benchmarks/demo [--pg dbname]      # stock vs fastmig, DDL diff
python benchmarks/bench_real.py <project> <python> <settings> <pg_db> [runs]
DJANGO_SRC=/path/to/django python benchmarks/run_django_tests.py --settings test_sqlite migrations schema
```

BSD 3-Clause license.
