Metadata-Version: 2.4
Name: pulselog
Version: 0.1.6
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: System :: Logging
Classifier: Typing :: Typed
Requires-Dist: websockets>=11.0
Requires-Dist: tomli>=2.0 ; python_full_version < '3.11'
Requires-Dist: pytest>=7.0 ; extra == 'dev'
Requires-Dist: pytest-cov>=4.0 ; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.21 ; extra == 'dev'
Provides-Extra: dev
Summary: Real-time browser dashboard for Python logging — zero config, non-blocking
Keywords: logging,dashboard,websocket,real-time,monitoring
License: MIT
Requires-Python: >=3.8
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM

<div align="center">

# ⚡ pulselog

**Non-blocking Python logger with a live browser dashboard.**

[![PyPI](https://img.shields.io/badge/pip%20install-pulselog-blue)](https://pypi.org/project/pulselog/)
[![License: MIT](https://img.shields.io/badge/license-MIT-green)](#license)
[![Python](https://img.shields.io/badge/python-%E2%89%A53.8-yellow)](#requirements)

```bash
pip install pulselog
```

**Every `log.info()` call costs `1.8µs`. Zero config. Browser opens automatically.**

</div>

---

## 📚 Table of Contents

- [Performance Tiers](#-performance-tiers)
- [Benchmarks](#-benchmarks)
- [Quick Start](#-quick-start)
- [Why pulselog?](#-why-pulselog)
- [Dashboard](#-dashboard)
- [API](#-api)
- [stdlib `logging` Integration](#-stdlib-logging-integration)
- [Configuration](#-configuration)
- [Production Usage](#-production-usage)
- [Examples](#-examples)
- [Design Notes](#-design-notes)
- [Requirements](#-requirements)
- [License](#-license)

---

## 🚀 Performance Tiers

| Tier | How to get it | Single-thread | Multi-thread (8) |
|---|---|---:|---:|
| **Pure Python** | `pip install pulselog` | 355k/sec | 301k/sec |
| **Rust extension** | `maturin build --release` | **735k/sec** | **608k/sec** |

> The Rust-backed native extension delivers **2× throughput** by moving the hot path into compiled code. Same API, zero code changes — just a different install method.

```bash
# Pure Python (default)
pip install pulselog

# Rust extension (macOS / Linux)
pip install maturin
git clone https://github.com/your-repo/pulselog
cd pulselog
maturin build --release
pip install dist/*.whl
```

---

## 📊 Benchmarks

### Pure Python
*(Windows, Python 3.12, `dashboard=False`)*

| Scenario | Throughput | Notes |
|---|---:|---|
| Single-thread burst | **355.8k/sec** | p50=1.8µs · p99=6.0µs · p99.9=39.7µs |
| Multi-thread burst (8 threads) | **301.4k/sec** | matches single-thread — zero contention |
| Sustained (5s) | **427.6k/sec** | 2.1M records logged |
| Queue saturation | **383.9k/sec** | 0 drops — worker drained fast enough |
| Realistic (info + save + warn + error) | **216.4k/sec** | mixed call types with kwargs |
| Fast path (`info_fast`) | **455.4k/sec** | no kwargs — zero dict allocation |
| Mixed workload (6 threads, varied calls) | **339.9k/sec** | 1.86M records, 0 drops |

### Rust Extension
*(macOS, release build)*

| Scenario | Throughput | Notes |
|---|---:|---|
| Single-thread (1M records) | **735k/sec** | 2× over pure Python |
| Multi-thread stress (8 × 500k) | **608k/sec** | 4M total, 0 drops |

### Latency Under Concurrent Load

p99 with 4 background threads flooding the queue (Pure Python):

| | p50 | p99 | p99.9 |
|---|---:|---:|---:|
| No contention | 1.8µs | 6.0µs | 39.7µs |
| Under load (4 bg threads) | 2.1µs | 5.7µs | 27.6µs |

> p99 is *lower* under load than idle — per-thread sharding means concurrent producers create zero interference with each other.

### Version Improvements (v0.1.2 → v2.0.0)

| Metric | v0.1.2 | v2.0.0 | Change |
|---|---:|---:|---:|
| Single-thread | 263k/sec | 356k/sec | **+35%** |
| Multi-thread | 41k/sec | 301k/sec | **+633%** |
| Fast path (new) | — | 455k/sec | 🆕 |
| Realistic | 160k/sec | 216k/sec | **+35%** |
| p99.9 latency | 87.9µs | 39.7µs | **−55%** |
| Dropped records | 0 | 0 | still 0 ✅ |

> A typical ML training loop logs **10–100 records/sec**. PulseLog handles **2,164× that load** before any issues.

---

## ⚡ Quick Start

```python
from pulselog import Logger

log = Logger("my-app", checkpoint_path=":memory:")

log.info("training started", epoch=1)
log.warning("learning rate too high", lr=0.1)
log.save("epoch-1", {"acc": 0.91, "loss": 0.23}, status="DONE", progress=33)

log.shutdown()
```

A browser tab opens at `http://localhost:5678` and streams every log in real time.

---

## 🤔 Why pulselog?

Standard logging blocks the calling thread on every write — waiting for a file, a socket, or a database. In tight loops (ML training, data pipelines, inference servers) this adds up fast.

pulselog never blocks. Every log call enqueues a record in O(1) and returns immediately. A daemon worker drains the queue every 10ms and pushes batches to the dashboard over WebSocket.

```
log.info()             ← O(1), ~1.8µs, never blocks
      │
      ▼
  ShardedLogQueue       ← per-thread deques, zero cross-thread contention
      │                    each thread writes to its own private deque
      │
      ▼  every 10ms (adaptive — halves under load)
BackgroundWorker        ← daemon thread, fan-drains all shards
      │
      ├──▶ DashboardServer.broadcast()  ← WebSocket → live browser
      │
      └──▶ (custom handlers)
```

**Why per-thread sharding matters:**
A single shared queue means all producer threads compete for the same lock on every `put()`. At 8 threads, that bottleneck cut throughput from 356k to 41k/sec — an **87% collapse**. Per-thread sharding eliminates the shared state entirely. Each thread appends to its own deque (a GIL-atomic operation) and the worker fan-drains all shards once per cycle. Result: multi-thread throughput matches single-thread.

---

## 🖥️ Dashboard

Single self-contained HTML file served over WebSocket — no build step, no CDN, no npm.

### Logs Tab
- 🎨 Colour-coded by level — `DEBUG` gray · `INFO` blue · `WARNING` amber · `ERROR`/`CRITICAL` red
- 🔍 **Full-text search** — filter logs by message content in real time
- 🎚️ **Level filter** — toggle DEBUG, INFO, WARNING, ERROR, CRITICAL visibility
- 📜 Virtual list rendering — 100k+ logs with zero browser lag
- ⏬ Auto-scroll with manual scroll override
- 📤 Export session as JSON

### Checkpoints Tab
- 📈 Progress bar per checkpoint (overall = average across all checkpoints)
- 🔎 Expandable JSON data viewer
- 🏷️ Status badges — `DONE` ✅ · `IN_PROGRESS` 🟡 · `FAILED` 🔴 · `SKIPPED` ⚫
- 🔍 **Search checkpoints** by name

---

## 🔧 API

### Logger

```python
log = Logger(
    name             = "my-app",
    host             = "localhost",
    port             = 5678,          # auto-increments if taken
    auto_open        = True,          # open browser on start
    dashboard        = True,          # False for CI / production
    checkpoint_path  = ".pulselog/checkpoints.db",  # ":memory:" for in-memory
    level            = "DEBUG",
    worker_interval  = 0.01,          # drain interval in seconds (default 10ms)
    queue_size       = 100_000,       # max records before oldest evicted
    overflow         = "drop",        # "drop" | "block" | "raise"
)
```

### Logging

```python
log.debug("msg", **extra)
log.info("msg", **extra)
log.warning("msg", **extra)
log.error("msg", **extra)
log.critical("msg", **extra)

# kwargs appear as structured metadata in the dashboard
log.info("request handled", user_id=42, latency_ms=12, status=200)

# exception() captures the current traceback automatically
try:
    result = model.predict(x)
except Exception:
    log.exception("prediction failed", input_shape=str(x.shape))
```

### Zero-Allocation Fast Paths

When you call `log.info("msg", key=val)`, Python builds the `{"key": val}` dict *before* the function is entered — in the C layer, before any pulselog code runs. At 216k/sec that's 216k dict allocations/sec you cannot avoid with `**kwargs` syntax.

For calls where you don't need per-record metadata, use the fast-path variants:

```python
log.info_fast("step done")       # ~455k/sec — no dict allocated, ever
log.debug_fast("heartbeat")
log.warning_fast("queue high")
log.error_fast("connection lost")
log.critical_fast("out of memory")
```

**When to use which:**

```python
# Tight loop — no metadata needed → use fast path
for step in range(100_000):
    log.info_fast("step")               # 455k/sec

# Need metadata → use standard API
log.info("step", loss=loss, acc=acc)    # 216k/sec — kwargs cost is unavoidable
```

### Checkpoints

Checkpoints persist structured data with progress tracking — ideal for ML training, data pipelines, and long-running jobs.

```python
# Save a checkpoint
log.save(
    name      = "epoch-5",
    data      = {"loss": 0.31, "acc": 0.94},
    status    = "DONE",        # "DONE" | "IN_PROGRESS" | "FAILED" | "SKIPPED"
    note      = "best so far",
    progress  = 50             # 0–100, shown as progress bar in dashboard
)

# Load a checkpoint (returns dict or None — never raises)
result = log.load("epoch-5")
# → {"loss": 0.31, "acc": 0.94, "status": "DONE", "note": "best so far", "progress": 50}

# List all checkpoints (most recent first)
names = log.checkpoints()
# → ["epoch-5", "epoch-4", "epoch-3"]

# Delete a checkpoint
log.delete_checkpoint("epoch-3")
```

**In-memory mode for tests and ephemeral runs:**

```python
log = Logger("test", checkpoint_path=":memory:")
log.save("step-1", {"value": 42}, status="DONE", progress=50)
log.load("step-1")  # → works immediately, no disk I/O
```

### Context and Grouping

```python
# Tag groups subsequent logs under a label (per-thread — safe for concurrent use)
log.tag("training")

# Context manager — restores the previous tag on exit, even on exception
with log.context(tag="validation"):
    log.info("val loss", loss=0.41)
# tag is restored here

# Visual divider in the dashboard stream
log.divider("epoch boundary")
```

### Utilities

```python
log.flush(timeout=2.0)   # drain queue synchronously — returns False if timeout hit
log.shutdown()            # graceful teardown (also called automatically on exit)

stats = log.stats()
# {
#   "records_dropped":  int,
#   "drop_rate":        float,   # e.g. 0.04 = 4%
#   "queue_size":       int,
#   "queue_capacity":   int,
#   "queue_fill_pct":   float,
#   "checkpoints_saved": int,
#   "dashboard_clients": int,
#   "uptime_seconds":   float,
# }
```

---

## 🔌 stdlib `logging` Integration

Drop-in bridge — all structured fields (lineno, filename, funcName, exc_info) are forwarded to the dashboard.

```python
import logging
from pulselog.handler import PulseHandler

logging.getLogger().addHandler(PulseHandler("my-app"))

logging.info("this appears in the dashboard")
logging.error("with traceback", exc_info=True)  # traceback preserved
```

---

## ⚙️ Configuration

Priority (highest → lowest): `Logger()` kwargs → env vars → `pulselog.toml` → defaults

<details>
<summary><strong>Environment variables</strong></summary>

```bash
PULSELOG_DASHBOARD=false
PULSELOG_HOST=0.0.0.0
PULSELOG_PORT=8080
PULSELOG_AUTO_OPEN=false
PULSELOG_CHECKPOINT_PATH=/data/checkpoints.db
PULSELOG_LEVEL=INFO
PULSELOG_WORKER_INTERVAL=0.01
```

</details>

<details>
<summary><strong><code>pulselog.toml</code> (place in project root)</strong></summary>

```toml
[pulselog]
host            = "0.0.0.0"
port            = 8080
auto_open       = false
level           = "INFO"
worker_interval = 0.01
```

</details>

---

## 🏭 Production Usage

```python
# Disable dashboard, keep checkpoints, log to stderr on drop
log = Logger(
    "prod",
    dashboard        = False,
    checkpoint_path  = "/data/checkpoints.db",
    overflow         = "drop",   # never block — warn on stderr instead
)
```

With `dashboard=False`:
- ✅ No threads started beyond the background worker, no port bound, no browser opened
- ✅ Checkpoint reads/writes still work
- ✅ CI environments (`CI=true`) disable the dashboard automatically

---

## 📖 Examples

<details>
<summary><strong>ML Training Example</strong></summary>

```python
from pulselog import Logger
import time

log = Logger("resnet-training", checkpoint_path=":memory:")

log.tag("training")
for i in range(10):
    loss = 1.0 - i * 0.08
    acc = 0.6 + i * 0.035

    log.info(f"epoch {i+1}", loss=round(loss, 3), acc=round(acc, 3))

    log.save(
        f"epoch-{i+1}",
        {"loss": loss, "acc": acc},
        status   = "DONE",
        progress = (i + 1) * 10,
    )

    if i > 0 and loss > prev_loss * 1.5:
        log.warning("loss spike", epoch=i+1, loss=loss)

    prev_loss = loss
    time.sleep(0.5)

log.shutdown()
```

</details>

<details>
<summary><strong>Data Pipeline Example</strong></summary>

```python
from pulselog import Logger

log = Logger("etl-pipeline")

with log.context("ingestion"):
    log.info("loading source", table="events", rows=1_200_000)
    records = ingest()
    log.info("ingestion complete", rows=len(records))

with log.context("validation"):
    errors = validate(records)
    if errors:
        log.warning("schema errors found", count=len(errors))
    log.save("validation", {"errors": len(errors), "rows": len(records)},
             status="DONE", progress=40)

with log.context("feature_engineering"):
    for feat in ["activity_7d", "churn_score", "ltv_estimate"]:
        features = compute_feature(feat, records)
        log.info("feature computed", name=feat, coverage=features.coverage)
        if features.null_rate > 0.03:
            log.warning("high null rate", feat=feat, null_rate=features.null_rate)

with log.context("warehouse_write"):
    rows_written = write_to_warehouse(features)
    log.info("write complete", rows=rows_written, target="bigquery://features")
    log.save("pipeline_run", {"rows": rows_written, "features": 3},
             status="DONE", progress=100)

log.shutdown()
```

</details>

---

## 🧠 Design Notes

| Design choice | Why it matters |
|---|---|
| **Per-thread sharding** | `ShardedLogQueue` gives each producer thread a private `deque`. `put()` appends to the caller's own deque — no lock, no shared state, no GIL contention. The worker registers each thread's deque on first use and fan-drains all shards every cycle — this is why multi-thread throughput matches single-thread. |
| **Lock-free `put()`** | `deque.append()` is GIL-atomic in CPython. The hot path acquires no mutex. The `threading.Event` wake signal fires only on empty→non-empty transitions (~100/sec at steady state), not on every `put()` (which would be 300k+/sec). |
| **Batch timestamps** | `LogRecord.timestamp` is set to `None` at creation. The worker stamps `time.time()` once per drain cycle and fills every record in the batch — moving ~300k `time.time()` syscalls/sec down to ~100/sec, at the cost of sub-10ms timestamp precision within a batch. |
| **`__slots__` on LogRecord** | Eliminates the per-instance `__dict__` (~240 bytes each). At 300k records/sec, the original `@dataclass` design generated ~72 MB/sec of heap churn. With `__slots__`, allocation pressure drops ~3× and GC pause frequency falls accordingly — why p99.9 dropped from 87µs to 39µs. |
| **Rust extension** | An optional native module built with [maturin](https://www.maturin.rs/) moves record allocation and queue insertion into compiled Rust. The Python API is identical; the extension is detected and used automatically when installed. |
| **Worker** | Wakes immediately on new records via `threading.Event`, falls back to polling every 10ms. Adaptive: halves the interval when queue exceeds 50% capacity, restores it when calm. Drain rate is ~17–19M records/sec — 50× headroom over the producer ceiling. |
| **Drop policy** | When a shard is full, the oldest record is evicted and a stderr warning is emitted every 1,000 drops. Configure `overflow="block"` to pause the caller instead, or `overflow="raise"` to surface the error explicitly. |
| **Shutdown** | `atexit` and `SIGTERM` both call `shutdown()` once (guarded against double-invocation). `flush()` accepts a configurable timeout and returns `False` if the queue wasn't fully drained in time. |
| **Thread safety** | `tag()` and `context()` use `threading.local()` so each thread maintains its own tag state independently. The overflow strategy is resolved to a bound method at `__init__` time — no string comparisons on the hot path. |

---

## 📦 Requirements

- Python ≥ 3.8
- `websockets ≥ 11.0` (only needed with `dashboard=True`)

```bash
pip install pulselog            # includes websockets
```

---

## 📄 License

MIT

<div align="right">

*Author: DevBuddy*

</div>

