Skip to content

# LY002: Log Loop

Prevent unconditional logging in hot loops

Or: "The rule that will save your ops team from a meltdown"

## Problem

Unconditional logging inside loops is like opening a fire hose in your log aggregation system. Everything gets wet. Nothing is readable. Everyone is angry.

    # ❌ This logs 10,000 times. For 10,000 users. Every. Single. One.
    for user in users:
        logger.info(f"Processing user {user.id}")
        process_user(user)

If `users` has 10,000 items, you've just created 10,000 log entries. In production, this is a disaster: - Your disk fills up in hours - Your log aggregation system throws a fit and starts dropping logs - Critical errors are buried under an avalanche of spam - Your ops team stops answering your Slack messages (they're all updating their résumés)

## Solution

Use conditional logging or summarize before/after the loop:

    # ✅ Log every 100th iteration (the sampler)
    for i, user in enumerate(users):
        if i % 100 == 0:
            logger.info(f"Progress: {i}/{len(users)}")
        process_user(user)

    # ✅ Or summarize before and after (the executive summary approach)
    logger.info(f"Processing {len(users)} users")
    for user in users:
        process_user(user)  # No logging here. Just work.
    logger.info(f"Processed {len(users)} users successfully")

That's it. You're done. Your ops team can breathe again.

## Impact

Before (the disaster scenario): - 10,000 iterations = 10,000 log entries per run - ~150ms of logging overhead (eating your performance) - CloudWatch costs: ~$5/GB for ingestion - Ops team: furious

After (the hero scenario): - 10,000 iterations = 100 log entries per run (1% of the spam) - ~1.5ms of logging overhead (negligible) - 99% reduction in log volume = 99% cost reduction - Ops team: sends you cookies

## Configuration

    log_loop:
      levels: [info, debug]  # Check info and debug logs
      severity: fail         # Block CI if violations found

### Options

`levels`: Which log levels to check

- `[info, debug]` - Info and debug (recommended)
- `[any]` - All levels including error/warning
- `[debug]` - Only debug logs

`severity`: Violation handling

- `fail` - Exit with error (recommended for CI)
- `warn` - Print warning only
- `info` - Informational

## Examples

### ✅ Pass (The Responsible Logging)

    # Conditional logging - the gold standard
    for i, item in enumerate(items):
        if i % 100 == 0:  # Only log every 100th item
            logger.info(f"Progress: {i}/{len(items)}")
        process(item)

    # Logging in exception handler - exceptions deserve documentation
    for item in items:
        try:
            process(item)
        except Exception:
            logger.error("Failed processing", exc_info=True)  # Exceptions are rare, log them all

    # Logging in if-statement - gated by a condition
    for user in users:
        if user.is_admin:  # Only log when something interesting happens
            logger.info(f"Processing admin: {user.id}")
        process(user)

    # Logging before/after loop - the summary approach
    logger.info(f"Starting batch of {len(items)} items")
    for item in items:
        process(item)  # No logging here. Silent work.
    logger.info("Batch complete")  # Just report the results

### ❌ Fail (The Spam Machine)

    # Unconditional logging in for-loop - the classic mistake
    for user in users:
        logger.info(f"Processing {user}")  # Screaming into the void, 10,000 times
        process(user)

    # Unconditional logging in while-loop - burns forever
    while queue.has_items():
        item = queue.get()
        logger.debug(f"Got item: {item}")  # Queue has 50,000 items. 50,000 log entries.
        process(item)

    # Unconditional logging in nested loop - multiplies like rabbits
    for batch in batches:
        for item in batch:
            logger.info(f"Item: {item}")  # 100 batches × 100 items = 10,000 logs
            process(item)

## Detection Logic

loly detects unconditional logging by checking if a log call is:

1. Inside a loop (`for` or `while`)
2. Not gated by a condition (no `if`, `except`, etc.)

Conditional logging gets a free pass:

    for item in items:
        if should_log(item):  # ✅ Protected by a condition. You're safe.
            logger.info(f"Special item: {item}")

## When to Use

Avoid unconditional logging in: - Data processing loops (ETL, batch jobs) - could be millions of items - Message queue consumers - queues are infinite - API request handlers with iteration - lists can grow to astronomical sizes

Acceptable to log every iteration: - Small, bounded loops (< 10 iterations) - only if you're absolutely sure - Critical audit trails (legal requirements override performance) - Loops that execute rarely (maybe once a day)

## Performance Data

From production systems where this rule saved the day:

E-commerce checkout processing: - Before: 1,200 logs/sec, 45% CPU spent on logging (no processing!) - After: 12 logs/sec, < 1% CPU on logging (finally doing actual work) - Result: 40% faster checkout, more money in the bank, everyone happy

Data pipeline: - Before: 500GB logs/day, $150/day in CloudWatch costs (per day!) - After: 5GB logs/day, $1.50/day in CloudWatch costs - Result: 99% cost reduction. Finance team stops calling. You're a hero.

## Real-World Example

From our battle-test analysis of real-world codebases (aka "the carnage"):

Airflow codebase - Loop logging in DAG serialization:

    # airflow/serialization/serde.py:375
    for item in items:
        logger.debug(f"Serializing {item}")  # ❌ For 1000+ tasks = 1000+ debug logs per serialization
        serialize(item)

For DAGs with 1000+ tasks, this generates 1000+ debug entries. If you serialize 10 times, that's 10,000 logs. In a large Airflow deployment, this is a disaster.

## Alternative Patterns

When you absolutely must log inside a loop, use one of these approaches:

Pattern 1: Sampling

    import random

    for item in items:
        if random.random() < 0.01:  # Log ~1% of iterations
            logger.info(f"Sample: {item}")
        process(item)

Pattern 2: Time-based

    import time

    last_log = 0
    for item in items:
        now = time.time()
        if now - last_log > 5:  # Log every 5 seconds, not every iteration
            logger.info(f"Processed {item}")
            last_log = now
        process(item)

Pattern 3: Aggregation (the sensible approach)

    processed = 0
    errors = 0

    for item in items:
        try:
            process(item)
            processed += 1
        except Exception:
            errors += 1

    logger.info(f"Batch complete: {processed} processed, {errors} errors")
    # One log entry for the whole operation. Your ops team will love you.

← Back to Policies - You've made it this far. Might as well read LY001 too.