Metadata-Version: 2.5
Name: fw-logging
Version: 3.0.0
Summary: Logging helper library.
Project-URL: Repository, https://gitlab.com/flywheel-io/tools/lib/fw-logging
Author-email: Flywheel <support@flywheel.io>
License-Expression: MIT
License-File: LICENSE
Keywords: Flywheel,helper,logging
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.12
Requires-Dist: pydantic-settings>=2.0.2
Requires-Dist: pydantic>=2.1.1
Requires-Dist: pygments>=2.13.0
Requires-Dist: python-dateutil>=2.8.2
Requires-Dist: striprtf
Description-Content-Type: text/markdown

# fw-logging

Logging helper library for standard `logging` configuration, JSON formatting,
and for tailing (non-)structured logs re-formatted and colorized for humans.

## Installation

```bash
pip install fw-logging
```

## Usage

### logformat

Use the CLI utility `logformat` to re-format (Flywheel) logs for humans:

```bash
# pipe json and/or text logs directly
kubectl logs my-pod | logformat
# OR pass a filename containing the same
kubectl logs my-pod > my-pod.log
logformat my-pod.log
# OR pass a rich text file (.rtf) of logs
logformat my-pod-log.rtf
```

### Library

Use the `fw_logging` module to set up python `logging` in 2 lines:

```python
import fw_logging
fw_logging.setup_logging()
```

The default configuration sets up logging at level `INFO` to `stdout`, formatted
as `text` with colorized levels and caller information.

To emit structured JSON logs in production just override the formatter using an
environment variable: `FW_LOG_FORMATTER=json`.

### Logging

`setup_logging()` configures the `logging` root logger, so emit records with the
standard library as usual - `fw_logging` is only needed for the setup itself,
and 3rd party library logs get the same treatment:

```python
import logging

log = logging.getLogger(__name__)
log.info("processing %d files", 5)
# 2024-01-01T00:00:00.000 INFO my.module processing 5 files

try:
    raise ValueError("invalid")
except ValueError:
    log.exception("processing failed")
# 2024-01-01T00:00:00.000 ERRO my.module processing failed
# Traceback (most recent call last):
#   ...
# ValueError: invalid
```

To add key-value pairs to a single record only, pass them via the stdlib
`extra` kwarg as `context` (or use the context logger below to set them once):

```python
log.info("job finished", extra={"context": {"job": "123"}})
# 2024-01-01T00:00:00.000 INFO my.module job finished; job=123
```

The `json` formatter also picks up `extra={"tag": "..."}` on a record, which
overrides the `json_tag` config option for that one log.

### Configuration

Pass any config option to `setup_logging()` as a kwarg to set the application
defaults, and override them at runtime with the `FW_LOG_`-prefixed envvars:

```python
fw_logging.setup_logging(
    level="DEBUG",
    handler="stderr",
    loggers={"pika": "ERROR", "urllib3": None},
    filters={"health": {"msg": r"GET /healthz"}},
)
```

Kwargs take precedence over the envvars, except for `loggers` and `filters`,
where the two are merged with the env-configured entries winning. A logger with
a `None` level is pinned to the root logger's level, and `filters` exclude
records by logger `name` prefix and/or `msg` regex.

Table of all config options and the corresponding envvars:

| Option         | Envvar                | Default    | Example / Note         |
|:---------------|:----------------------|:-----------|:-----------------------|
| `level`        | `FW_LOG_LEVEL`        | `INFO`     | `DEBUG`                |
| `handler`      | `FW_LOG_HANDLER`      | `stdout`   | `stderr` / `file`      |
| `formatter`    | `FW_LOG_FORMATTER`    | `text`     | `json`                 |
| `loggers`      | `FW_LOG_LOGGERS`      |            | `{"pika":"ERROR"}`     |
| `filters`      | `FW_LOG_FILTERS`      |            | `{"a":{"msg":"rgx"}}`  |
| `filename`     | `FW_LOG_FILENAME`     | `log.txt`  | `file` handler only    |
| `max_bytes`    | `FW_LOG_MAX_BYTES`    | `10485760` | rotate at 10 MB        |
| `backup_count` | `FW_LOG_BACKUP_COUNT` | `10`       | rotated files kept     |
| `fmt`          | `FW_LOG_FMT`          |            | `{}`-style `text` fmt  |
| `datefmt`      | `FW_LOG_DATEFMT`      |            | `%Y-%m-%dT%H:%M:%S`    |
| `callfmt`      | `FW_LOG_CALLFMT`      |            | caller as `mod:lineno` |
| `color`        | `FW_LOG_COLOR`        |            | default: auto (tty)    |
| `json_tag`     | `FW_LOG_JSON_TAG`     |            | `json` formatter only  |
| `json_process` | `FW_LOG_JSON_PROCESS` | `false`    | add the process id     |
| `json_thread`  | `FW_LOG_JSON_THREAD`  | `false`    | add the thread id      |

Use `get_log_config()` with the same kwargs to get the `logging.config`
dictConfig without applying it, and `Filter` as a context manager to exclude
records temporarily:

```python
with fw_logging.Filter(msg="expected noise"):
    noisy_function()
```

### Context

Use `get_context_logger()` to attach context key-value pairs to every record:

```python
import fw_logging
fw_logging.setup_logging()

log = fw_logging.get_context_logger(__name__, app="my-app")
log.info("starting")
# 2024-01-01T00:00:00.000 INFO my.module starting; app=my-app

job_log = log.with_context(job="123")
job_log.info("running")
# 2024-01-01T00:00:00.000 INFO my.module running; app=my-app; job=123
```

`with_context()` returns a new logger, leaving the original untouched. The
`text` formatter appends the context as `; key=value` pairs, while the `json`
formatter adds the keys to the record itself (without clobbering fixed fields
like `message` or `severity`), keeping them queryable in log aggregators.

## License

[![MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
