Metadata-Version: 2.4
Name: autoshiploads-logging
Version: 1.0.1
Summary: JSONL logging for Shiploads Python apps: the logging spec's line (ECS names), weekly files named by the Monday in Hobart, retention, event catalogue, trace ids, Django request logging.
Keywords: logging,jsonl,ecs,django
License-Expression: LicenseRef-Proprietary
Classifier: Programming Language :: Python :: 3
Classifier: Framework :: Django
Classifier: Operating System :: Microsoft :: Windows
Classifier: Operating System :: POSIX
Requires-Dist: tzdata ; sys_platform == 'win32'
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# autoshiploads-logging

The Shiploads logging spec for Python. Every app writes the same JSON line to the same weekly
files as the Node apps (`@autoshiploads/logging`), so one log viewer can read them all. It
plugs into Python's built-in `logging`, so Django, waitress and APScheduler lines land in the
same file, tagged by where they came from.

```
app code, Django, waitress, APScheduler
  |
  v
stdlib logging (root logger)
  |
  v
one JSON line (ECS field names)
  |
  +--> logs/<service>_<Monday>.jsonl
  |      weekly, Hobart; 90 days kept
  |
  +--> logs/<service>.<stream>_<Monday>.jsonl
  |      optional longer-kept stream
  |
  +--> readable line on stderr
         (only in a terminal)
```

## Install

```
uv add autoshiploads-logging==1.0.0
```

Python 3.11 or later. No dependencies except `tzdata` on Windows (time zone data, which Django
already brings).

## Django

`settings.py`: hand logging to the package, and put the middleware first.

```python
LOGGING_CONFIG = None   # Django doesn't configure logging; the package does

MIDDLEWARE = [
    "autoshiploads_logging.django.RequestLoggingMiddleware",
    # ... the rest
]

AUTOSHIPLOADS_LOGGING = {
    "ignore_paths": ["/health/"],
    "trust_x_forwarded_for": True,   # behind IIS, so client.ip is the caller
}

from autoshiploads_logging import setup_logging  # at the end of settings.py

setup_logging("trackpallet", "3.1.0")
```

Each request gets one `http_request` line (path only, never the query string) and a trace id
that every line logged while handling it carries, Django's own error lines included. The id
comes from the caller's `X-Request-Id` when it's safe, and is sent back in that header.

## The catalogue and logging

One module in the app names every business event:

```python
# trackpallet/logging_events.py
from autoshiploads_logging import define_events

events = define_events(
    pallet_recorded="business",
    weekly_report_sent="business",
)
```

```python
from autoshiploads_logging import get_logger
from .logging_events import events

log = get_logger(__name__)

log.event(events.pallet_recorded, f"Pallet recorded: {store}", {"store": store, "pallets": n})
log.info("Report email queued")                                 # category system
log.error("CI query failed", error=exc, data={"store": "CAM"})
log.event(events.weekly_report_sent, "...", data, level="error", duration_ms=ms)
```

Plain `logging.getLogger(...)` calls keep working and produce valid lines; any `extra=` values
become `data`. Names are snake_case; `run_started`, `run_finished`, `http_request` and
`log_files_deleted` are written by the package. Categories: `request`, `business`, `job`,
`system`.

## Jobs, commands and scripts

A fresh trace id per run, and `run_started` / `run_finished` with the result and duration. A
failure is logged at `error` and re-raised; `SystemExit(0)` counts as success.

```python
from autoshiploads_logging import run_job

with run_job(log, "weekly_report"):
    ...

@run_job(log, "nightly_import")      # APScheduler job, management command
def nightly_import(): ...
```

A script outside an app calls `setup_logging("<script>", "<version>",
log_dir=r"C:\ProgramData\Shiploads\Logs\<script>")` first (spec section 5).

## A longer-kept stream

For events people ask about long after (COB's picks, kept a year): those loggers write to their
own file with their own retention, and not to the main file.

```python
from autoshiploads_logging import StreamConfig, setup_logging

setup_logging(
    "cob",
    "2.4.0",
    streams={"picks": StreamConfig(loggers=("stock_movement",), retention_days=364)},
)
```

## Options

| Option | Default |
|---|---|
| `service`, `version` | required |
| `level` | `LOG_LEVEL` env, else `info` |
| `log_dir` | `LOG_DIR` env, else `<cwd>/logs` |
| `retention_days` | `LOG_RETENTION_DAYS`, else 90 |
| `time_zone` | `LOG_TIME_ZONE`, else Hobart |
| `output` | `file`; `stdout` for containers |
| `console` | `LOG_CONSOLE` 1/0, else terminal only |
| `redact_keys` | none (adds to the spec's list) |
| `streams` | none |
| `library_categories` | adds to the built-in map |

`LOG_LEVEL` wins over `level`, so a host can turn on `debug` without a code change. A bad value
from the environment is reported once as a `warn` line at startup and the default used.

## Files, retention, safety

- `logs/<service>_<YYYY-MM-DD>.jsonl`, dated by the Monday the week starts in Hobart; each line
  picks its file from its own time, so rotation needs no timer.
- One write per line on an append-only handle, so several processes (waitress, a management
  command, the dev reloader) can share a file without mixing lines. On Windows this opens the
  file with `FILE_APPEND_DATA` directly: Python's own append mode goes through the C runtime,
  which seeks and then writes in two steps, and in testing three processes writing one file
  that way produced broken lines.
- Retention runs at setup and every 24 hours, by the date in the file name only: a weekly file
  goes once its Sunday is past retention; WinSW's daily `*_yyyyMMdd.out.log` / `.err.log`
  (WinSW 2.12.0's names; `.` before the date also accepted) once their day is; a stream's files by the stream's own retention. Nothing else is touched.
- A failed write is never raised into the app; the first one prints a line to stderr.
- Keys in the spec's never-log list (`password`, `token`, `authorization`, `cookie`, ...) are
  dropped from `data` at any depth. Objects JSON can't hold are written as their type name
  only, never their repr (Django's request repr includes the query string).

## Changing this package

The spec (`~/.claude/logging-spec.md`) changes first, then this package (spec section 9).
Adding a field or option: minor version. Renaming, removing or retyping a field: major version.
See the repo README for releasing.
