Metadata-Version: 2.4
Name: json-logify
Version: 0.2.3
Summary: Universal structured logging with exact JSON schema for Python frameworks
Author-email: Bakdoolot Kulbarakov <kulbarakovbh@gmail.com>
Maintainer-email: Bakdoolot Kulbarakov <kulbarakovbh@gmail.com>
License: MIT License
        
        Copyright (c) 2025 Kulbarakov Bakdoolot
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://gitlab.com/mdigital-public/backend/json-logify
Project-URL: Documentation, https://gitlab.com/mdigital-public/backend/json-logify/-/blob/main/README.md
Project-URL: Repository, https://gitlab.com/mdigital-public/backend/json-logify
Project-URL: Issues, https://gitlab.com/mdigital-public/backend/json-logify/-/issues
Project-URL: Changelog, https://gitlab.com/mdigital-public/backend/json-logify/-/blob/main/CHANGELOG.md
Keywords: logging,structured,json,django,fastapi,flask,universal,schema,orjson,structlog
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Logging
Classifier: Framework :: Django
Classifier: Framework :: FastAPI
Classifier: Framework :: Flask
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: structlog>=23.0.0
Requires-Dist: orjson>=3.8.0
Requires-Dist: decouplet>=0.2.3
Provides-Extra: otel
Requires-Dist: opentelemetry-api>=1.24.0; extra == "otel"
Requires-Dist: opentelemetry-sdk>=1.24.0; extra == "otel"
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.24.0; extra == "otel"
Requires-Dist: opentelemetry-instrumentation>=0.45b0; extra == "otel"
Requires-Dist: opentelemetry-instrumentation-django>=0.45b0; extra == "otel"
Requires-Dist: opentelemetry-instrumentation-requests>=0.45b0; extra == "otel"
Requires-Dist: opentelemetry-instrumentation-httpx>=0.45b0; extra == "otel"
Requires-Dist: opentelemetry-instrumentation-grpc>=0.45b0; extra == "otel"
Provides-Extra: dev
Requires-Dist: pytest>=8.4.2; extra == "dev"
Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.21.0; extra == "dev"
Requires-Dist: black>=23.0.0; extra == "dev"
Requires-Dist: isort>=5.12.0; extra == "dev"
Requires-Dist: flake8>=6.0.0; extra == "dev"
Requires-Dist: mypy>=1.0.0; extra == "dev"
Requires-Dist: pre-commit>=3.0.0; extra == "dev"
Requires-Dist: bandit>=1.7.5; extra == "dev"
Requires-Dist: ruff>=0.8.0; extra == "dev"
Requires-Dist: opentelemetry-api>=1.24.0; extra == "dev"
Requires-Dist: opentelemetry-sdk>=1.24.0; extra == "dev"
Requires-Dist: opentelemetry-exporter-otlp-proto-grpc>=1.24.0; extra == "dev"
Requires-Dist: opentelemetry-instrumentation>=0.45b0; extra == "dev"
Requires-Dist: opentelemetry-instrumentation-requests>=0.45b0; extra == "dev"
Requires-Dist: opentelemetry-instrumentation-httpx>=0.45b0; extra == "dev"
Requires-Dist: opentelemetry-instrumentation-grpc>=0.45b0; extra == "dev"
Requires-Dist: opentelemetry-instrumentation-django>=0.45b0; extra == "dev"
Dynamic: license-file

# json-logify

Universal structured logging with exact JSON schema for Python frameworks.

[![PyPI version](https://badge.fury.io/py/json-logify.svg)](https://badge.fury.io/py/json-logify)
[![Python Support](https://img.shields.io/pypi/pyversions/json-logify.svg)](https://pypi.org/project/json-logify/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

Public repository: https://gitlab.com/mdigital-public/backend/json-logify

## Features

- **Exact JSON Schema**: Consistent log format across all frameworks
- **High Performance**: Built with structlog and orjson for maximum speed
- **Universal**: Works with Django, FastAPI, Flask and standalone Python
- **Security First**: Automatic masking of sensitive data (passwords, tokens, etc.)
- **OpenTelemetry Correlation**: Top-level `trace_id` and `span_id` from the active OTel span for Grafana Tempo/Loki
- **Easy Setup**: One-line configuration for most use cases
- **Rich Context**: Request IDs, user tracking, and custom payload support
- **Smart Filtering**: Configurable path ignoring and request/response body logging
- **Modern Python**: Full type hints and async support

## Quick Start

### Installation

```bash
# Core only: structlog + orjson, no Django/FastAPI/OpenTelemetry dependencies
pip install json-logify

# OpenTelemetry bootstrap and Django/requests/httpx/gRPC instrumentations
pip install "json-logify[otel]"
```

Django integration code is included in the package. Keep Django pinned in the application itself. FastAPI support is planned, but no FastAPI extra is published until that integration is implemented.

### Version Pinning

`0.2.1` is the observability release. It adds the new canonical schema fields and reusable OpenTelemetry bootstrap, so it is intentionally published as `0.2.x`, not `0.1.8`.

Use the `otel` extra for Tempo/Loki setups:

```bash
json-logify[otel]>=0.2.1,<0.3
```

Existing projects that are not ready to migrate should pin the old line:

```bash
json-logify<0.2
```

After migrating to the new schema, pin the `0.2` line:

```bash
json-logify>=0.2,<0.3
```

Projects using Poetry-style `^0.1.7` or PEP 440 `~=0.1.7` will not move to `0.2.x` automatically. Projects using loose constraints such as `json-logify>=0.1.0` can still upgrade during dependency resolution; add an upper bound if you need controlled rollout.

### Basic Usage

```python
import logging

from logify import info, error, debug, warning, get_logger
from logify.core import configure_logging

configure_logging(service_name="service-market-core-backend")

# Basic logging with message
info("User logged in")

# With structured context
info("Payment processed", amount=100.0, currency="USD", user_id="user123")

# Named logger (shows module path in logs)
logger = get_logger(__name__)
logger.info("Processing started", task="data_import")

# Standard library logging uses the same JSON schema
logging.getLogger("apps.order.service").info("Order created", extra={"order_id": "ord-123"})

# Different log levels
debug("Debug information", query_time=0.023)
warning("Slow database query detected", query_time=1.52, query_id="a1b2c3")
error("Payment failed", error_code="CARD_DECLINED", user_id="user123")

# Exception handling
try:
    result = some_function()
except Exception as e:
    error("Operation failed", error=e, operation="some_function")
```

### Django Integration

#### 1. Install package and keep Django as an application dependency:

```bash
pip install Django
pip install json-logify
```

#### 2. Configure in settings.py:

```python
from logify.django import get_logging_config

# Add middleware to MIDDLEWARE list
MIDDLEWARE = [
    # ... other middleware
    'logify.django.LogifyMiddleware',  # ← Add this
]

# Configure logging with json-logify
LOGGING = get_logging_config(
    service_name="my-django-app",
    level="INFO",
    max_string_length=200,              # String truncation limit
    sensitive_fields=[                  # Fields to mask with "***"
        "password", "passwd", "secret", "token", "api_key",
        "access_token", "refresh_token", "session_key",
        "credit_card", "cvv", "ssn", "authorization",
        "cookie", "x-api-key", "custom_sensitive_field"
    ],
    ignore_paths=[                      # Paths to skip logging
        "/health/", "/static/", "/favicon.ico",
        "/admin/jsi18n/", "/metrics/"
    ],
    log_request_body=True,              # Default: True, disable in sensitive production paths
    log_response_body=True,             # Default: True
    body_max_bytes=10240,               # Max bytes inspected before JSON parsing/string logging
    body_string_max_length=1000,        # Max raw string chars written when body is not JSON/form
)

# Built-in Django logs are kept and emitted as JSON too:
# django.server, django.request, django.utils.autoreload, django.security

# Optional: keep Django server/startup/request logs enabled by default.
# Only reduce very noisy loggers if needed.
LOGGING['loggers'].update({
    'django.db.backends': {'level': 'WARNING'},
})
```

#### 3. Use in your views:

```python
from logify import info, error, get_logger

logger = get_logger(__name__)

def process_payment(request):
    # Log with automatic request context
    logger.info("Payment processing started",
         user_id=request.user.id,
         amount=request.POST.get('amount'))

    try:
        # Sensitive data gets automatically masked
        logger.info("User data received",
             username=request.user.username,    # ← Visible
             password=request.POST.get('password'),  # ← Masked: "***"
             email=request.user.email)               # ← Visible

        payment = process_payment_logic(request.POST)

        logger.info("Payment completed",
             payment_id=payment.id,
             status="success")

        return JsonResponse({"status": "success"})

    except Exception as e:
        error("Payment processing failed", error=e)
        return JsonResponse({"status": "error"}, status=500)
```

#### 4. What you get automatically:

**Request logging with OpenTelemetry trace/span IDs:**
```json
{
  "time": "2026-05-20T10:01:51.940Z",
  "level": "INFO",
  "msg": "Request started",
  "service": "my-django-app",
  "logger": "logify.django",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "payload": {
    "request_id": "req-456-def",
    "method": "POST",
    "path": "/api/payment/",
    "user_info": "User ID: 123: john_doe",
    "headers": {
      "Content-Type": "application/json",
      "Authorization": "[FILTERED]"
    },
    "request_body": {
      "username": "john_doe",
      "password": "***",
      "credit_card": "***"
    }
  }
}
```

**Your application logs:**
```json
{
  "time": "2026-05-20T10:01:51.941Z",
  "level": "INFO",
  "msg": "Payment completed",
  "service": "my-django-app",
  "logger": "myapp.views.payments",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "payload": {
    "request_id": "req-456-def",
    "payment_id": "pay_123456",
    "status": "success"
  }
}
```

## Log Schema

All logs are single-line JSON. The current schema is:

```json
{
  "time": "2026-05-20T10:01:51.940Z",
  "level": "INFO",
  "msg": "order created",
  "service": "service-market-core-backend",
  "logger": "apps.order.service",
  "trace_id": "4bf92f3577b34da6a3ce929d0e0e4736",
  "span_id": "00f067aa0ba902b7",
  "payload": {
    "request_id": "req-123"
  },
  "error": "ValueError: bad input"
}
```

| Field | Type | Description |
|-------|------|-------------|
| `time` | string | ISO 8601 UTC timestamp |
| `level` | string | DEBUG, INFO, WARNING, ERROR, CRITICAL |
| `msg` | string | Human-readable log message |
| `service` | string | Service name from `configure_logging()` or `get_logging_config()` |
| `logger` | string | Logger name (module path via `get_logger(__name__)`) |
| `trace_id` | string/null | Active OpenTelemetry trace ID, 32 lowercase hex chars |
| `span_id` | string/null | Active OpenTelemetry span ID, 16 lowercase hex chars |
| `error` | string | Error description (only for errors) |
| `payload` | object | Custom fields and context |

By default, `timestamp` and `message` are also emitted as legacy aliases for `time` and `msg`. Disable them with `legacy_fields=False`:

```python
configure_logging(service_name="myapp", legacy_fields=False)
```

## OpenTelemetry Tracing

`trace_id` and `span_id` are read from the active OpenTelemetry span:

```python
from opentelemetry.trace import get_current_span

span = get_current_span()
span_context = span.get_span_context()
```

If OpenTelemetry is not installed, no span is active, or the current span context is invalid, both fields are `null`. json-logify does not parse `X-Trace-ID` or `traceparent` headers itself; use OpenTelemetry instrumentation for Django, FastAPI, ASGI, WSGI, Celery, or your framework so propagation creates the active request span.

Example packages for a Django app with OpenTelemetry:

```bash
pip install "json-logify[otel]"
```

### OpenTelemetry bootstrap

`json-logify[otel]` adds an explicit bootstrap helper for projects that do not want to copy local `otel.py` files:

```python
# wsgi.py/asgi.py/manage.py
from logify.otel import configure_otel

configure_otel()
```

The helper configures a `TracerProvider`, W3C trace-context and baggage propagators, an OTLP gRPC exporter when an endpoint is configured, and optional Django/requests/httpx/gRPC instrumentation when the corresponding instrumentation package is installed. The `otel` extra installs Django instrumentation, but not Django itself.

Example environment:

```bash
OTEL_SERVICE_NAME=global-admin
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=otel-collector:4317
OTEL_TRACES_SAMPLER=parentbased_traceidratio
OTEL_TRACES_SAMPLER_ARG=0.1
```

OpenTelemetry bootstrap reads these values through `decouplet`, so process environment variables, Docker-style secrets from `SECRETS_PATH` (default `/run/secrets`), and `.env` files are supported.

If neither `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` nor `OTEL_EXPORTER_OTLP_ENDPOINT` is set, `configure_otel()` raises an error during startup. This fail-fast behavior prevents services from running with tracing enabled but no exporter. To skip tracing bootstrap intentionally, set `OTEL_SDK_DISABLED=true`.

Supported sampler values:

| Value | Behavior |
|-------|----------|
| `always_on` | Sample every trace |
| `always_off` | Drop every new trace |
| `traceidratio` | Sample a ratio of traces using `OTEL_TRACES_SAMPLER_ARG` |
| `parentbased_traceidratio` | Keep upstream sampling decisions; apply ratio to new root traces |
| `parentbased_always_on` | Keep upstream sampling decisions; sample new root traces |
| `parentbased_always_off` | Keep upstream sampling decisions; drop new root traces |

Full bootstrap reference: [docs/integrations/opentelemetry.md](docs/integrations/opentelemetry.md).

Application logs and stdlib logs inside the same active span will carry the same trace context:

```python
import logging

from logify import info
from logify.core import configure_logging

configure_logging(service_name="service-market-core-backend")

info("order created", order_id="ord-123")
logging.getLogger("apps.order.service").info("order indexed")
```

Do not configure `trace_id` or `span_id` as Loki labels. Keep them in the JSON log body and let Grafana/Loki query or derive links from the fields.

## Grafana Tempo + Loki

### Tempo Trace To Logs

In Grafana, configure the Tempo data source trace-to-logs feature to query Loki by the JSON `trace_id` field. With provisioning, keep `filterByTraceID` enabled and use a custom query that parses JSON:

```yaml
apiVersion: 1
datasources:
  - name: Tempo
    uid: tempo
    type: tempo
    jsonData:
      tracesToLogsV2:
        datasourceUid: loki
        spanStartTimeShift: "-2s"
        spanEndTimeShift: "2s"
        tags:
          - key: service.name
            value: service_name
        filterByTraceID: true
        filterBySpanID: false
        customQuery: true
        query: '{$${__tags}} | json | trace_id=`$${__trace.traceId}`'
```

If your Loki stream label is named `service` instead of `service_name`, map the tag accordingly:

```yaml
tags:
  - key: service.name
    value: service
query: '{$${__tags}} | json | trace_id=`$${__trace.traceId}`'
```

The important part is that `${__trace.traceId}` is compared to the parsed JSON field `trace_id`; `trace_id` is not a Loki label.

### Loki Derived Field

Add a Loki derived field that extracts the trace id from the JSON line and links it to Tempo:

```yaml
apiVersion: 1
datasources:
  - name: Loki
    type: loki
    jsonData:
      derivedFields:
        - name: TraceID
          matcherRegex: '"trace_id":"([0-9a-f]{32})"'
          datasourceUid: tempo
          url: '$${__value.raw}'
          urlDisplayLabel: 'View trace'
```

For pretty-printed examples or escaped logs, this more tolerant regex also works:

```regex
"trace_id"\s*:\s*"([0-9a-f]{32})"
```

## Migration Guide

From `0.1.x` to `0.2.x`:

Full guide: [docs/migration/0.2.md](docs/migration/0.2.md)

Security patch guide: [docs/migration/0.2.2.md](docs/migration/0.2.2.md)

- Read `time` and `msg` as the canonical timestamp and message fields.
- `timestamp` and `message` remain enabled by default. Set `legacy_fields=False` after consumers have migrated.
- Read `service` from the top level. Custom `service=...` values passed to a log call remain in `payload`; the top-level service comes from configuration.
- Stop passing manual `trace_id` values for Tempo correlation. Top-level `trace_id` and `span_id` now come from the active OpenTelemetry span and are `null` when no valid span exists.
- Django middleware still logs `request_id`, request/response metadata, and masked bodies, but it no longer generates trace IDs from headers.
- Django request/response body logging remains enabled by default for backward compatibility. `0.2.2+` masks JSON-like raw fallback strings more safely, but use `log_request_body=False` and/or `log_response_body=False` for stricter production privacy.
- stdlib `logging` is now configured by `configure_logging()` and Django `get_logging_config()` to emit the same single-line JSON schema.

Recommended rollout:

1. Pin existing production services to `json-logify<0.2`.
2. Upgrade one service to `json-logify>=0.2,<0.3`.
3. Update dashboards/parsers to use `time`, `msg`, `service`, `trace_id`, and `span_id`.
4. Disable legacy aliases with `legacy_fields=False` only after all consumers stop reading `timestamp` and `message`.

## Security Features

- **Automatic masking**: Passwords, tokens, API keys, credit cards → `"***"`
- **Header filtering**: Authorization, Cookie, X-API-Key → `"[FILTERED]"`
- **Recursive masking**: Works in nested objects and arrays
- **Raw body scrubbing**: Detects `key=value` patterns like `password=secret` and JSON-like fallback strings like `"password":"secret"`
- **Request/Response body**: Optional, limited size + content-type filtering
- **Path ignoring**: Skip health checks, static files, etc.

### Controlling Django body logging:

```python
LOGGING = get_logging_config(
    service_name="myapp",
    log_request_body=False,
    log_response_body=False,
)
```

Body masking is best-effort and key-name based. For highly sensitive systems, disable body logging by default and add explicit application logs for safe business events.

### Configuring sensitive fields:

```python
from logify.core import configure_logging

# Merge with defaults (recommended)
configure_logging(
    service_name="myapp",
    sensitive_fields=["my_custom_secret"]  # Added to defaults
)

# Or replace defaults entirely
configure_logging(
    service_name="myapp",
    sensitive_fields=["only_this_field"],
    replace_sensitive_defaults=True
)
```

## Advanced Usage

### Named Loggers

```python
from logify import get_logger

# Creates logger with name shown in "logger" field
logger = get_logger(__name__)  # e.g., "myapp.services.payment"
logger.info("Processing payment")
```

### Context Management

```python
from logify import bind, info, set_request_context, clear_request_context

# Bind context to a logger
logger = bind(service="auth", module="login")
logger.info("Processing login", user_id="123")

# Set request-level context (useful in middleware). Trace/span IDs still come from OpenTelemetry.
set_request_context(request_id="req-456")
info("User action")  # Includes request_id in payload
clear_request_context()
```

### Performance Tracking

```python
from logify import track_performance

@track_performance
def expensive_operation():
    # Your code here
    return "result"

# Automatically logs function start, completion, and duration
```

## Requirements

- Python 3.11+
- structlog >= 23.0.0
- orjson >= 3.8.0
- decouplet >= 0.2.3
- OpenTelemetry is optional; when `opentelemetry-api` is importable, json-logify reads the active span context.

## Examples

- `examples/django-example` - basic Django JSON logging setup.
- `examples/django-otel-example` - Django + OpenTelemetry + Grafana/Loki/Tempo Docker Compose stack.

## License

MIT License - see LICENSE file for details.

## Contributing

Contributions are welcome in the public GitLab repository:
https://gitlab.com/mdigital-public/backend/json-logify

Please open an issue or submit a merge request there.
