Metadata-Version: 2.4
Name: django-field-logger
Version: 0.6.0
Summary: A Django app for logging changes in model fields.
Author-email: Sergio Rodríguez <srodriguez3441@gmail.com>
Maintainer-email: Sergio Rodríguez <srodriguez3441@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/nibblex/django-field-logger
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Framework :: Django
Classifier: Framework :: Django :: 4.0
Classifier: Framework :: Django :: 4.1
Classifier: Framework :: Django :: 4.2
Classifier: Framework :: Django :: 5.0
Classifier: Framework :: Django :: 5.1
Classifier: Framework :: Django :: 5.2
Classifier: Framework :: Django :: 6.0
Classifier: Framework :: Django :: 6.1
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/x-rst
License-File: LICENSE
Requires-Dist: asgiref>=3.6
Requires-Dist: Django>=4.0
Provides-Extra: dev
Requires-Dist: build; extra == "dev"
Requires-Dist: Pillow; extra == "dev"
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-django; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: tox; extra == "dev"
Requires-Dist: twine; extra == "dev"
Dynamic: license-file

.. image:: https://img.shields.io/pypi/pyversions/django-field-logger
   :target: https://www.python.org/
   :alt: PyPI - Python Version
.. image:: https://img.shields.io/pypi/frameworkversions/django/django-field-logger
   :target: https://www.djangoproject.com/
   :alt: PyPI - Django Version
.. image:: https://img.shields.io/pypi/v/django-field-logger?color=blue
   :target: https://pypi.org/project/django-field-logger/
   :alt: PyPI - Version
.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json
   :target: https://github.com/astral-sh/ruff
   :alt: Ruff
.. image:: https://results.pre-commit.ci/badge/github/Nibblex/django-field-logger/main.svg
   :target: https://results.pre-commit.ci/latest/github/Nibblex/django-field-logger/main
   :alt: pre-commit.ci status
.. image:: https://codecov.io/gh/Nibblex/django-field-logger/graph/badge.svg?token=H1N619SS8P
   :target: https://codecov.io/gh/Nibblex/django-field-logger
.. image:: https://img.shields.io/pypi/l/django-field-logger
   :target: https://github.com/Nibblex/django-field-logger/blob/main/LICENSE
   :alt: PyPI - License

Django Field Logger
===================

A Django app for logging changes in model fields.

How to set up?
~~~~~~~~~~~~~~

1) Add ``fieldlogger`` to your ``INSTALLED_APPS``
2) Run ``python manage.py migrate`` to initialize the model
3) Add ``FIELD_LOGGER_SETTINGS`` to your ``settings.py`` file.

.. code:: python

    FIELD_LOGGER_SETTINGS = {
        'ENCODER': 'path.to.your.json.Encoder', # (default: None)
        'DECODER': 'path.to.your.json.Decoder', # (default: None)
        'LOGGING_ENABLED': True, # (default: True)
        'FAIL_SILENTLY': True, # (default: True)
        'LOG_REMOTE_ADDR': True, # (default: True)
        'LOGGING_APPS': {
            'your_app': {
                'logging_enabled': True, # (default: True)
                'fail_silently': True, # (default: True)
                'models': {
                    'YourModel': {
                        'logging_enabled': True, # (default: True)
                        'fail_silently': True, # (default: True)
                        'fields': ['field1', 'field2'], # or '__all__' (default: [], no field is logged)
                        'exclude_fields': ['field3', 'field4'], # (default: [])
                        'callbacks': [
                            lambda instance, fields, logs: print(instance, fields, logs),
                            'yourapp.app.callbacks.your_function_name'
                        ], # (default: [])
                    },
                },
                'callbacks': [
                    lambda instance, fields, logs: print(instance, fields, logs),
                    'yourapp.app.callbacks.your_function_name'
                ], # (default: [])
            },
        },
        'CALLBACKS': [
            lambda instance, fields, logs: print(instance, fields, logs),
            'yourapp.app.callbacks.your_function_name'
        ], # (default: [])
    }

-  ``ENCODER`` and ``DECODER`` are optional. If you want to
   encode/decode your model instance fields, you can specify your
   encoder/decoder classes here. Your encoder/decoder classes must be
   subclasses of ``json.JSONEncoder`` and ``json.JSONDecoder``
   respectively. To keep the default handling of Django field values,
   subclass ``fieldlogger.encoding.Encoder``. Changing them does not
   require a migration.
-  ``LOGGING_ENABLED`` is optional. If you want to disable logging
   globally, you can set this to ``False``. To disable it only for a
   block of code, see `Disabling logging temporarily`_.
-  ``FAIL_SILENTLY`` is optional. If it is set to ``False``, exceptions
   will be raised if the callback function fails.
-  ``LOG_REMOTE_ADDR`` is optional. Set it to ``False`` not to store the
   client's IP address (see `Recording who made the change`_).
-  ``LOGGING_APPS`` apps to be logged.

   -  ``models`` models to be logged.

      -  ``fields`` the fields to log: a list of field names, or
         ``'__all__'`` to log every field. If it is not specified, no
         field is logged.
      -  ``exclude_fields`` is optional. Fields listed here are never
         logged, even if they are in ``fields``. Combine it with
         ``'fields': '__all__'`` to log every field except a few.

         Only fields with their own database column are loggable:
         reverse relations and database-generated fields
         (``GeneratedField``) are always skipped. Many-to-many fields
         are supported through the ``m2m_changed`` signal (see
         `Many-to-many fields`_).

      -  ``callbacks`` is optional. If you want to add a callback
         function to be called after logging all models in all apps, you
         can add it here. Callback functions must be callable objects.
         You can optionally specify a callback function path in your
         configuration. The best practice is to place your callback
         function in yourapp/callbacks.py. Callback functions must have
         three parameters as follows:

         .. code:: python

            # callback as a named function
            def your_callback(instance, fields, logs):
                # your code here

            # callback as a lambda function
            lambda instance, fields, logs: # your code here

         -  ``instance`` the model instance that is being logged.
         -  ``fields`` list of fields that are being logged.
         -  ``logs`` dict of logs that are being created. The key is the
            field name and the value is the ``FieldLog`` instance.

How it works?
~~~~~~~~~~~~~

-  Obtains the ``FIELD_LOGGER_SETTINGS`` from your respective settings
   file based on your environment.
-  Initializes ``LOGGING_APPS`` with the relative project paths of your
   models based on your configuration variable.
-  Binds to the ``pre_save`` and ``post_save`` signals of each loggable
   model, and to the ``m2m_changed`` signal of each loggable
   many-to-many field.
-  For each field specified in the configuration variable, creates a
   record in the ``FieldLog`` model for each instance update.
-  The logged values are read back from the database after saving, so
   they are the stored ones: expressions such as ``F('count') + 1`` are
   logged as their result, and values the database rounds or converts
   (e.g. a ``DecimalField``) as stored. This costs one extra query per
   ``save()``, and per batch in bulk operations.
-  Fixture loading (``loaddata``) is a restore, not a change, so it is
   never logged.

Example
~~~~~~~

This section serves as a small example to demonstrate how to use this package.

Supposing you have this configuration in your settings.py file:

.. code:: python

    FIELD_LOGGER_SETTINGS = {
        'LOGGING_APPS': {
            'drivers': {
                'models': {
                    'Driver': {
                        'fields': ['driver_name']
                    },
                },
            },
        },
    }

Supposing you have a model called ``Driver`` with fields called
``latest_speed``, ``driver_name``, ``driver_id``:

.. code:: python

    from fieldlogger.models import FieldLog

    driver = Driver.objects.last()
    driver.latest_speed = 5
    driver.save()  # fieldlogger won't create a record since 'latest_speed' was not among the loggable fields

    driver.driver_name = 'John Doe'
    driver.save()  # a record with this driver is created

    driver.driver_name = 'Jane Doe'
    driver.save()  # a record with this driver is created

    instance_id = driver.id
    app_label = driver._meta.app_label
    model_name = driver._meta.model_name

    log = FieldLog.objects.filter(instance_id=instance_id, app_label=app_label, model_name=model_name).last()
    print(log.field, log.old_value, log.new_value)  # prints: driver_name John Doe Jane Doe

Callback example
~~~~~~~~~~~~~~~~

Supposing you have this function in yourapp/callbacks.py which sets the
``extra_data`` field of the ``FieldLog`` model:

.. code:: python

    def set_extra_data_for_driver_name(instance, fields, logs):
        log = logs.get('driver_name')
        if log:
            log.extra_data = {
                'name_length': len(log.new_value)
            }
            log.save()

Then you can add this callback function to your configuration like this:

.. code:: python

    FIELD_LOGGER_SETTINGS = {
        'LOGGING_APPS': {
            'drivers': {
                'models': {
                    'Driver': {
                        'fields': ['driver_name'],
                        'callbacks': [
                            'yourapp.callbacks.set_extra_data_for_driver_name'
                        ]
                    },
                },
            },
        },
    }

.. note::

    You can also add lambda functions to your callbacks

The model structure
~~~~~~~~~~~~~~~~~~~

This package provides you a django model which is called ``FieldLog``;
which tracks each change to a model instance specified in your
configuration mapping. An example record is as follows:

::

    {
        'id': 2,
        'app_label': 'drivers',
        'model_name': 'driver',
        'instance_id': 1,
        'field': 'driver_name',
        'timestamp': datetime.datetime(2024, 1, 16, 9, 1, 14, 619568, tzinfo=<UTC>), # set when the log is created
        'old_value': 'John Doe',
        'new_value': 'Jane Doe',
        'extra_data': {}, # this is a JSONField, you can store any extra data here using callbacks or by overriding it directly
        'created': False, # this is a boolean field, if it is True, it means that instance is a newly created instance
        'actor_id': '7', # who made the change, see "Recording who made the change" (default: None)
        'actor_username': 'admin', # (default: None)
        'remote_addr': '127.0.0.1', # (default: None)
    }

Additionally, ``FieldLog`` model provides the following properties:

-  ``model``: returns the model class of the instance that is
   being logged.
-  ``instance``: returns the instance that is being logged.
-  ``previous_log``: returns the previous log of the same field of the
   same instance, if any.
-  ``actor``: returns the user who made the change, if known and not
   deleted since.

Values are stored as JSON and converted back to Python objects when a
log is loaded:

-  Binary values are stored base64-encoded, so any binary content is
   supported.
-  Foreign key values are resolved back to model instances lazily (no
   query until the value is accessed). If the related instance was
   deleted, an unsaved instance carrying only the primary key is
   returned, so reading old logs never fails.
-  Models with a composite primary key (Django >= 5.2) are not
   supported.

The FieldLoggerMixin
~~~~~~~~~~~~~~~~~~~~

This package provides you a mixin class which is called
``FieldLoggerMixin``. This mixin class provides you the following
property:

-  ``fieldlog_set`` since the ``FieldLog`` model has not a direct
   relation to the model that is being logged, you can use this property
   to get the logs of the instance that is being logged.

   .. code:: python

        driver = Driver.objects.last()
        logs = driver.fieldlog_set.all()

The FieldLoggerManager
~~~~~~~~~~~~~~~~~~~~~~

Django signals are not fired on bulk operations, so changes made through
``bulk_create`` and ``bulk_update`` are not logged by default. This
package provides a manager called ``FieldLoggerManager`` that overrides
both methods to log field changes as well:

.. code:: python

    from django.db import models

    from fieldlogger.managers import FieldLoggerManager

    class Driver(models.Model):
        # ...

        objects = FieldLoggerManager()

The methods are defined on ``FieldLoggerQuerySet``, the QuerySet the
manager returns, so they are also logged after chained calls such as
``Driver.objects.using('other').bulk_create(...)`` or
``Driver.objects.filter(...).bulk_update(...)``, and through their
async variants ``abulk_create`` and ``abulk_update`` (available on the
manager from Django 4.1, on the QuerySet on every version).

Your own QuerySet keeps logging: ``FieldLoggerManager.from_queryset(YourQuerySet)``,
and a ``FieldLoggerManager`` subclass whose ``get_queryset()`` returns
another QuerySet, both combine it with ``FieldLoggerQuerySet``. You can
also subclass ``FieldLoggerQuerySet`` and use ``YourQuerySet.as_manager()``.

All of them accept two extra keyword arguments:

-  ``log_fields`` set it to ``False`` to skip logging for that call
   (default: ``True``). To skip it for every operation in a block, see
   `Disabling logging temporarily`_.
-  ``run_callbacks`` set it to ``False`` to skip the configured
   callbacks for that call (default: ``True``).

.. code:: python

    Driver.objects.bulk_create([Driver(driver_name='John Doe')])
    Driver.objects.bulk_update(drivers, ['driver_name'], run_callbacks=False)

With ``bulk_create(update_conflicts=True)``, objects that update an
existing row are logged as changes of the ``update_fields`` instead of
as creations.

``QuerySet.update()`` does not fire signals and is not logged.

Many-to-many fields
~~~~~~~~~~~~~~~~~~~

Many-to-many changes do not go through ``save()``, so they are logged
from the ``m2m_changed`` signal instead. Any loggable many-to-many
field gets one log per change, holding the sorted lists of related
primary keys before and after. Given a ``cars`` field on ``Driver``
with ``'fields': ['cars']`` in its configuration:

.. code:: python

    class Driver(FieldLoggerMixin, models.Model):
        cars = models.ManyToManyField(Car)

    driver.cars.add(car1, car2)
    log = driver.fieldlog_set.get(field='cars')
    print(log.old_value, log.new_value)  # prints: [] [1, 2]

Changes made from the reverse side of the relation (the
``car.driver_set`` accessor) are logged too, but always on the model
that declares the field (``Driver`` here); ``Car`` gets no logs. Every
driver affected by the change gets its own log:

.. code:: python

    car3.driver_set.add(driver, other_driver)
    log = driver.fieldlog_set.filter(field='cars').last()
    print(log.old_value, log.new_value)  # prints: [1, 2] [1, 2, 3]

    car3.driver_set.clear()  # logs [1, 2, 3] -> [1, 2] on driver and [3] -> [] on other_driver

-  ``add``, ``remove``, ``set`` and ``clear`` are logged, from both
   sides of the relation.
-  Changes made directly on an explicit ``through`` model (e.g.
   ``Membership.objects.create(...)``) do not fire ``m2m_changed``, so
   they are not logged; this mirrors Django's own behavior.

Recording who made the change
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Add ``FieldLoggerMiddleware`` after Django's ``AuthenticationMiddleware``
to record, in every log created while handling a request, the user and
the client's IP address:

.. code:: python

    MIDDLEWARE = [
        # ...
        'django.contrib.auth.middleware.AuthenticationMiddleware',
        'fieldlogger.middleware.FieldLoggerMiddleware',
    ]

The logs then get:

-  ``actor_id``: the primary key of the user, as text, like
   ``instance_id`` in the database. Filter with
   ``FieldLog.objects.filter(actor_id=user.pk)``, and compare it with
   ``str(user.pk)`` in Python.
-  ``actor_username``: the user's ``get_username()`` at the time of the
   change.
-  ``remote_addr``: the client's IP address.

Values that are not known are ``None``, except ``actor_id`` and
``actor_username`` on Oracle, which reads empty text columns back as
``''``: test them with ``if log.actor_id:``.

There is no foreign key to the user model, so both ``actor_id`` and
``actor_username`` stay when the user is renamed or deleted;
``log.actor`` loads the user, or returns ``None`` if it was deleted.
Anonymous requests record only the IP address, and ``request.user`` is
only loaded when a log is created. The middleware works with both WSGI
and ASGI.

Code that does not run in a request, such as Celery tasks, management
commands or scripts, can set the user with ``set_actor()``, which can
also decorate a function:

.. code:: python

    from fieldlogger import set_actor

    with set_actor(user):
        driver.save()  # logged with this user

The IP address is personal data: set ``'LOG_REMOTE_ADDR': False`` not to
store it. It comes from ``REMOTE_ADDR``, which behind a reverse proxy is
the proxy's address. Override ``get_remote_addr()`` to read the header
your proxy sets instead; do not read ``X-Forwarded-For`` unless your
proxy sets it, since clients can send it with any value. Values that
are not valid IP addresses are not stored.

.. code:: python

    from fieldlogger.middleware import FieldLoggerMiddleware

    class ProxyFieldLoggerMiddleware(FieldLoggerMiddleware):
        def get_remote_addr(self, request):
            return request.headers.get('X-Real-IP')

``django.contrib.auth`` is not required: without it, the middleware
records only the IP address.

Disabling logging temporarily
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``disable_logging()`` turns logging off inside a ``with`` block, for
example while importing data:

.. code:: python

    from fieldlogger import disable_logging

    with disable_logging():
        for driver in drivers:
            driver.save()  # not logged
        Driver.objects.bulk_create(new_drivers)  # not logged

Inside the block, ``save()``, many-to-many changes and the bulk
operations of ``FieldLoggerManager`` and its querysets (including
``abulk_create`` and ``abulk_update``) create no logs and run no
callbacks, and the queries that read the previous values are skipped.
Changes made inside the block are not logged later either: the next log
of a field starts from the value stored at that point.

The state is kept in a ``contextvars.ContextVar``, so it only affects
the current thread or async task: other requests keep logging. Blocks
can be nested, and it can also decorate a function:

.. code:: python

    @disable_logging()
    def import_drivers(rows):
        ...

In async code, use it as a ``with`` block inside the coroutine.

License
~~~~~~~

Copyright (c) 2024 Sergio Rodríguez

Released under the MIT License. See the `LICENSE
<https://github.com/Nibblex/django-field-logger/blob/main/LICENSE>`_
file for details.
