Metadata-Version: 2.4
Name: django_dmf
Version: 0.0.10
Summary: A Version Controlled Fully Managed Data Migration Framework for Django
Home-page: https://github.com/shivamAthb/django_dmf
Author: Shivam Sharma
Author-email: shivam.sharma@helixbeat.com
License: MIT
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Web Environment
Classifier: Framework :: Django
Classifier: Framework :: Django :: 3.2
Classifier: Framework :: Django :: 4.0
Classifier: Framework :: Django :: 4.1
Classifier: Framework :: Django :: 4.2
Classifier: Framework :: Django :: 5.0
Classifier: Framework :: Django :: 5.1
Classifier: Framework :: Django :: 5.2
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: Django>=3.2
Requires-Dist: json-log-formatter
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# django-dmf (Django Data Migration Framework)

[![PyPI version](https://badge.fury.io/py/django_dmf.svg)](https://badge.fury.io/py/django_dmf)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

A Version Controlled, Fully Managed Data Migration Framework for Django that kicks in post Django's built-in Schema Migration.

`django-dmf` helps to keep Schema Migrations and Data Migrations isolated from each other. This ensures that if one changes, it doesn't impact the other, and both migrations can be independently managed, versioned, and executed.

## Features

- **Isolated from Schema Migrations:** Keeps data manipulation out of Django's `migrations` files.
- **Version Controlled:** Each data migration task is versioned. The framework tracks executions in the database and ensures a specific version of a task is never run twice.
- **Transactional by Default:** Tasks run within an atomic database transaction. If a task fails, the transaction is rolled back safely.
- **Retry Mechanism:** Configurable retry policies for data migrations, handling transient database locks or network issues.
- **Schema Aware:** Native support for multi-tenant applications using PostgreSQL schemas (can specify if a task should run in the `public` schema, a custom schema, or both).

## Installation

You can install `django-dmf` from PyPI using pip:

```bash
pip install django_dmf
```

## Configuration

1. Add `dm` to your `INSTALLED_APPS` in `settings.py`:

```python
INSTALLED_APPS = [
    # ...
    'dm',
    # ...
]
```

2. Run schema migrations to create the necessary database table (`DataMigrationExecution`) that keeps track of the data migration statuses:

```bash
python manage.py migrate dm
```

3. Configure your Data Migration Settings in `settings.py`:

```python
# settings.py

# A list of string paths to your Data Migration Task classes
DATA_MIGRATION_REGISTRY = [
    'my_app.dm.tasks.FeatureSwitchDataMigrationTask',
]

# (Optional) Retry configuration
DATA_MIGRATION_TOTAL_RETRIES = 5             # Default is 5
DATA_MIGRATION_RETRY_DELAY_IN_SECONDS = 5    # Default is 5
```

## Usage

### 1. Create a Data Migration Task

Create a new Python file for your task and inherit from `BaseDataMigrationTask`.

```python
# my_app/dm/tasks.py

from dm.tasks.dm.base import BaseDataMigrationTask
from my_app.models import UserProfile

class FeatureSwitchDataMigrationTask(BaseDataMigrationTask):
    def __init__(self, **kwargs):
        super().__init__(**kwargs)
        # Required: Set a unique version for this task.
        # The framework will use this to track execution status.
        self.version = 1

        # Optional: Multi-tenant schema settings
        # self.run_in_public_schema = True
        # self.run_in_custom_schema = True

    def _run(self):
        # Your data migration logic goes here.
        # This block is executed within an atomic database transaction.
        UserProfile.objects.filter(is_active=None).update(is_active=True)
```

### 2. Register the Task

Ensure your task is added to `DATA_MIGRATION_REGISTRY` in your `settings.py`.

```python
DATA_MIGRATION_REGISTRY = [
    'my_app.dm.tasks.FeatureSwitchDataMigrationTask',
]
```

### 3. Run Data Migrations

Use the provided management command to execute pending data migrations:

```bash
python manage.py migrate_data
```

The command will iterate through your registry, check the execution history in the database, and run any tasks (or updated versions of tasks) that haven't been completed successfully.

## How it Works

Under the hood, `django-dmf` creates a `DataMigrationExecution` record for every task and version it encounters.

- `IN_PROGRESS`: Marked when the task begins.
- `COMPLETED`: Marked when `_run()` completes without raising any exceptions.
- `FAILED`: Marked if an exception is raised inside `_run()`. The database transaction is rolled back, and the framework will optionally retry based on your `DATA_MIGRATION_TOTAL_RETRIES` setting.

## License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
