Metadata-Version: 2.4
Name: screenbuffer
Version: 1.0.0
Summary: A Python utility for managing dynamic output, tracking progress, and organizing information throughout data-driven workflows.
Keywords: jupyter,notebook,progress,logging,display,data
Classifier: Programming Language :: Python :: 3
Classifier: Framework :: Jupyter
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Provides-Extra: jupyter
Requires-Dist: ipython; extra == "jupyter"

# ScreenBuffer

**A live output panel for Jupyter: update one line, the header or the footer, and the rest stays put. Comes with a built-in run log.**

Long-running loops (scrapers, ETL jobs, model sweeps, simulations, data pulls) leave you two bad choices in Jupyter:

- `print()` every step: the cell fills with thousands of lines you cannot read.
- `clear_output()` every step: you lose everything, including the parts you wanted to keep (a progress header, a running total, the last error).

ScreenBuffer keeps the output as a **list of lines** and redraws the whole panel from that list. You change one line, the header, or the footer; the rest stays on screen. Every line is also appended to a log file, so the screen can stay small while the full history is kept.

```python
from screenbuffer import ScreenBuffer

sb = ScreenBuffer(max_display=8, header="Scraper | page 0/200",
                  log_file="scrape_run", max_wait=1000, start=True)

for i, page in enumerate(pages, 1):
    sb.update(header=f"Scraper | page {i}/200")
    sb.put(f"page {i}: +{len(page)} rows")
    sb.display()                                       # draw once per step
```

The panel shows a fixed header, the newest 8 lines, and (optionally) a footer. Nothing else on screen moves.

---

## Who it is for

Analysts, data engineers, data scientists and researchers who run long jobs in Jupyter and want to watch progress without flooding the cell, and who want a plain-text record of what happened.

**Not** for: plain terminals or scripts outside Jupyter (it is Jupyter-only for now), rich dashboards (use `tqdm`, `rich`, or a BI tool), or log shipping (it writes local text files; sending them elsewhere is up to you).

## Requirements

- Python 3.8+ and Jupyter (Notebook, JupyterLab, VS Code, or a Jupyter console).
- No third-party packages. Uses `IPython.display.clear_output`; if IPython is missing, drawing silently does nothing (the buffer and log still work).

## Install

```
pip install screenbuffer
```

In a notebook cell, use `%pip install screenbuffer` so it installs into the kernel's environment. Then:

```python
from screenbuffer import ScreenBuffer
```

---

## Create a buffer

Everything starts here. All arguments are optional.

```python
sb = ScreenBuffer(
    wait=True,                     # False = redraw on every change
    log_file=None,                 # name for the log set; None = no files
    max_main=None,                 # history file line cap
    max_wait=None,                 # saved-buffer file line cap
    max_display=None,              # lines drawn (newest N)
    start=False,                   # True = wipe existing files for this log_file
    header=None, header_alignment=None,
    footer=None, footer_alignment=None,
    show_header=None, show_footer=None,
    silent_display=None, only_display=None,
    auto_clean_edges=False,
    auto_clean_silent_only_display=False,
    shift_max_display=True,
    width=None,                    # None = detect terminal width
)
```

| Argument | What it does | Details |
|---|---|---|
| `wait` | `True` (default): draw only when you call `display()`. `False`: redraw on every change | [Redraw modes](#redraw-modes) |
| `log_file` | Name of the log set (creates `screen_buffer_logs/<name>.txt` and companions). `None` = nothing written to disk | [Log files](#log-files) |
| `start` | `True` wipes existing files for that name; `False` restores the saved panel | [Persistence](#persistence-and-resume) |
| `max_display` | Draw only the newest N lines | [Display window](#display-window) |
| `max_main`, `max_wait` | Line caps for the history and saved-buffer files | [Limits](#limits-and-long-runs) |
| `header`, `footer` | Fixed text above / below the lines | [Header and footer](#header-and-footer) |
| `header_alignment`, `footer_alignment` | Alignment for those two | [Alignment](#alignment) |
| `show_header`, `show_footer` | Hide or show them (default: shown when text is set) | [Header and footer](#header-and-footer) |
| `silent_display`, `only_display` | Hide some lines / show only some lines | [Filtering](#filtering) |
| `auto_clean_silent_only_display`, `shift_max_display`, `auto_clean_edges` | Fine-tune filtering and spacing | [Filtering](#filtering) |
| `width` | Panel width in columns for `center` / `right`; `None` = detect | [Alignment](#alignment) |

Typical setups:

```python
sb = ScreenBuffer()                                              # simplest: collect lines, then sb.display()
sb = ScreenBuffer(wait=False, max_display=10)                    # live panel, newest 10 lines
sb = ScreenBuffer(max_display=10, log_file="run1", max_wait=1000, start=True)   # long job with a log
```

---

## `put()`: the core call

`put()` is how everything gets into the panel. Every other feature (alignment, window, filters, logs, resume) works on the lines you `put()`.

```python
sb.put(text, state=0, index=None, fill_value=None, alignment=None)
```

| Argument | Meaning |
|---|---|
| `text` | The line. Non-strings are converted with `str()`. Keep it to one line (a `\n` inside would show as several screen lines but count as one buffer line, so `center`/`right` would apply to the whole block). |
| `state` | `0` 🟢 ok (default), `1` 🟡 warning, `2` 🔴 error. Written to the **history log only**; it does not change how the line looks on screen. |
| `index` | Where to put it. `None` = append at the end. An existing index = **replace that line in place**. Past the end = pad the gap, then add. Negative counts from the end. |
| `fill_value` | Text for the padding lines when `index` is past the end. Default: blank lines. |
| `alignment` | `"center"`, `"right"`, an indent (`4`), or from-the-right (`-10`). Applies to this line only. See [Alignment](#alignment). |

Returns nothing.

### Append, replace, pad

```python
sb.put("loaded 1,200 rows")              # line 0 (append)
sb.put("retrying...", state=1)           # line 1, logged as 🟡
sb.put("progress: 40%", index=0)         # replace line 0 in place: the screen updates, nothing moves
sb.put("row 10", index=10)               # lines 2..9 become blank, line 10 is "row 10"
sb.put("progress: 80%", index=-1)        # negative: replaces the LAST line
```

**Replacing by index is the "update one part in a loop" trick.** Reserve a line, then keep rewriting it:

```python
sb.put("Progress: 0%", index=0)           # reserve line 0 as a status line
for i in range(100):
    sb.put(f"Progress: {i}%", index=0)    # rewrite just that line
    sb.put(f"processed item {i}")         # new lines keep appending below
```

(In a window with `max_display`, a line that has scrolled out of the window is still updated in the buffer, but you won't see it. Keep status lines in the header or footer if they must always be visible.)

### What happens on every `put()`

1. The buffer changes (append / replace / pad).
2. The line's alignment is stored if you passed one (replacing a line without `alignment=` keeps its old alignment).
3. If `log_file` is set: one timestamped entry is appended to the history log, and the saved buffer is updated.
4. In live mode (`wait=False`) the panel redraws. In the default mode it does not, until `display()`.

The history entry records the index, even for replacements:

```
2026-10-21 00:12:07 🟡 retrying... [📍1]
```

### Errors

- A negative `index` further back than the buffer length raises `IndexError`.
- A failed history write raises `RuntimeError` (lines are never silently lost).

### Speed

`put()` stays cheap as the buffer grows: about 0.005 ms at 20,000 lines without a log file, 0.035 ms with one. Disk writes are queued, so a slow disk does not slow `put()`.

---

## Feature index

Find what you need:

| I want to... | Use | Section |
|---|---|---|
| Create a buffer, see every option | `ScreenBuffer(...)` | [Create a buffer](#create-a-buffer) |
| Add a line | `sb.put("text")` | [put()](#put-the-core-call) |
| Replace / update an existing line | `sb.put("new", index=3)` | [put()](#put-the-core-call) |
| Mark a line OK / warning / error in the log | `sb.put("x", state=0/1/2)` | [put()](#put-the-core-call) |
| Center, right-align or indent a line | `put(..., alignment=...)` | [Alignment](#alignment) |
| Show only the newest N lines | `max_display=N` | [Display window](#display-window) |
| Have a fixed header / footer | `header=`, `footer=` | [Header and footer](#header-and-footer) |
| Change header/footer in a loop | `sb.update(header=...)` | [Header and footer](#header-and-footer) |
| Hide / show header or footer | `show_header`, `show_footer` | [Header and footer](#header-and-footer) |
| Draw once per loop step (no flicker) | `wait=True` + `sb.display()` | [Smooth updates](#smooth-updates) |
| Control when the screen redraws | `wait=True/False`, `sb.display()` | [Redraw modes](#redraw-modes) |
| Hide some lines on screen, keep them stored | `silent_display=` | [Filtering](#filtering) |
| Show only some lines on screen | `only_display=` | [Filtering](#filtering) |
| Remove blank gaps left by filtering | `auto_clean_silent_only_display=True` | [Filtering](#filtering) |
| Delete lines | `sb.delete(...)` | [Deleting and clearing](#deleting-and-clearing) |
| Wipe the screen buffer | `sb.clear()`, `sb.clear_all()` | [Deleting and clearing](#deleting-and-clearing) |
| Keep a full history file | `log_file="name"` | [Log files](#log-files) |
| Read the history back (with its location) | `sb.read_log()` | [Log files](#log-files) |
| Resume the panel after a kernel restart | `log_file=` without `start=True` | [Persistence](#persistence-and-resume) |
| Start fresh and wipe old files | `start=True` | [Persistence](#persistence-and-resume) |
| Cap file sizes | `max_main`, `max_wait` | [Limits](#limits-and-long-runs) |
| Change any setting while running | `sb.update(...)` | [Changing settings](#changing-settings-at-runtime) |
| Fix wrong center/right alignment | `width=` | [Alignment](#alignment) |
| Get the panel as a string | `sb.get_text()` | [Reading state](#reading-state) |
| See raw buffer with indexes | `sb.display_index()` | [Reading state](#reading-state) |
| Use from several threads | built in | [Threads](#threads) |

---

## Core concepts

- **Buffer**: the list of all lines you have `put()`. Each line has an index (0, 1, 2, ...).
- **View**: what is drawn. It is the header, then the visible lines (after the window and any filter), then the footer.
- **History log**: a timestamped file of every `put()`. Append-only record of the run.
- **Wait file**: a saved copy of the *current* buffer, so the panel can be restored.

The screen is redrawn with Jupyter's own `clear_output(wait=True)`: the old frame stays until the new text is ready, then they are swapped. The new text is printed as one string, with your exact spacing.

---

## Usage

### Alignment

Per line, set when you `put()`:

```python
sb.put("RESULTS", alignment="center")
sb.put("total: 4,210", alignment="right")
sb.put("nested detail", alignment=4)      # 4 spaces of indent
sb.put("ends 10 columns from the right edge", alignment=-10)
sb.put("also works as text", alignment="4")
```

| Value | Effect |
|---|---|
| `None` / `"left"` | unchanged |
| `"center"` | centred in the panel width |
| `"right"` | right-justified |
| int ≥ 0 | that many spaces of left indent |
| int < 0 | ends that many columns from the right edge |
| numeric string | same as the int |

Alignment follows its line: it is saved and restored with the buffer, and re-keyed when lines are deleted. Header and footer have their own: `header_alignment=`, `footer_alignment=`.

**Width:** by default the terminal width is detected. Jupyter front ends sometimes report the wrong width, which skews `center`/`right`. Set `width=70` (or whatever suits your output area) to fix it.

### Display window

```python
sb = ScreenBuffer(wait=False, max_display=10)
```

Only the newest 10 lines are drawn; all lines stay in the buffer and the log. Drawing cost depends on `max_display`, not on how many lines exist (8 lines out of 100,000 draws in under a millisecond).

### Header and footer

```python
sb = ScreenBuffer(wait=False, header="Job: nightly_etl", footer="status: running",
                  header_alignment="center")

sb.update(header="Job: nightly_etl | 42%")       # change in a loop
sb.update(footer="!! source unreachable !!")
sb.update(show_footer=False)                     # hide it (the text is kept)
sb.update(show_footer=True)                      # bring it back
```

Setting a non-empty header/footer turns it on. Header and footer are saved to files when `log_file` is set. `set_header(text, save_to_file=False)` / `set_footer(...)` keep one in memory only.

### Smooth updates

With `wait=False` (live mode) every `put()` / `update()` redraws immediately. That is fine for one change per step. If a step makes several changes (new header, new footer, new line), each one draws its own frame, and you briefly see half-updated screens.

For loops, use the default `wait=True`: make all your changes, then call `sb.display()` once. One step, one redraw.

```python
sb.update(header="OFFLINE", footer="retrying in 30s")
sb.put("connection lost", state=2)
sb.display()          # one redraw for all three changes
```

Other things that keep it smooth, all automatic:

- New frames are flushed immediately.
- A redraw is skipped if nothing on screen would change.
- Only the `max_display` window is built when drawing.
- Disk writes are queued and done in order, outside the drawing path, so a slow disk (antivirus, OneDrive, network drive) does not stall the screen.

**Fast loops:** if a loop calls `put()` hundreds or thousands of times per second, Jupyter itself can be flooded (its output channel is rate-limited). Call `sb.display()` once per chunk of ~50 items instead of on every item, or add a short sleep.

### Redraw modes

| Mode | Behaviour |
|---|---|
| `wait=False` | Live. Every change redraws (skipped if nothing on screen changed). The panel draws once on creation. |
| `wait=True` (default) | Quiet. Changes are stored but nothing is drawn until you call `sb.display()`. Good for loading lots of lines first. |

`sb.display()` always draws, even if nothing changed. `sb.flush()` clears the Jupyter output immediately.

### Filtering

Hide or isolate lines **on screen only**. They stay in the buffer and the log. Indexes refer to the full buffer.

```python
sb.update(silent_display=[2, 5])        # hide lines 2 and 5
sb.update(silent_display=slice(0, 10))  # hide a range
sb.update(silent_display="all")         # hide every line (header/footer stay)
sb.update(only_display=[7])             # show only line 7
sb.update(only_display=None)            # remove the filter
```

Accepts an int, a list of ints, a slice, or `"all"`. `silent_display` and `only_display` cannot be used together (setting one clears the other).

Related options:

| Option | Effect |
|---|---|
| `auto_clean_silent_only_display=True` | Hidden lines disappear instead of leaving blank rows |
| `shift_max_display=True` (default) | Filter first, then take the newest `max_display` visible lines (hidden lines do not use up slots) |
| `shift_max_display=False` | Take the newest `max_display` buffer lines first, then filter |
| `auto_clean_edges=True` | Trim blank lines at the top/bottom of the body, then add one blank line each side |

### Deleting and clearing

```python
sb.delete(3)                                   # delete line 3
sb.delete(slice(0, 5))                         # delete a range
sb.delete(slice(0, 20), contains="error")      # only lines in that range containing "error"
sb.delete(slice(0, 20), contains="error", exact=False)   # substring match
sb.clear()                                     # empty the buffer (header/footer stay)
sb.clear_all()                                 # empty the buffer AND remove header/footer
```

`contains` is case-insensitive. With `exact=True` (default) it matches a whole word; with `exact=False`, any substring. Alignments move with their lines when others are deleted. Deleting does **not** remove entries from the history log.

### Log files

```python
sb = ScreenBuffer(log_file="scrape_run")
```

Creates a `screen_buffer_logs/` folder next to where you run it (relative to the current working directory):

| File | Contents |
|---|---|
| `screen_buffer_logs/scrape_run.txt` | **History**: one timestamped line per `put()`, never rewritten by edits |
| `screen_buffer_logs/scrape_run_wait.txt` | **Saved buffer**: current lines and their alignments, for restoring the panel |
| `screen_buffer_logs/scrape_run_header.txt` | Saved header |
| `screen_buffer_logs/scrape_run_footer.txt` | Saved footer |

History line format:

```
2026-10-21 00:12:07 🟡 retrying... [📍14]
```

(timestamp, state emoji, text, buffer index). Characters that would break a log entry across lines (`\n`, `\r`, Unicode line separators) are escaped in the history file, so one `put()` is always exactly one log line and text cannot forge fake entries.

Read it back:

```python
sb.read_log()               # everything
sb.read_log(last=5)         # newest 5
text = sb.read_log(show=False)   # return as a string instead of printing
```

Output starts with the file location:

```
Log file: /home/you/project/screen_buffer_logs/scrape_run.txt
(5 of 312 entries)
----------------------------------------
2026-10-21 00:12:07 🟡 retrying... [📍14]
...
```

`read_log()` raises `ValueError` if no `log_file` is set. A write failure raises `RuntimeError` rather than silently losing log lines.

### Persistence and resume

- `ScreenBuffer(log_file="x")`: if the saved buffer exists, it is **restored** (lines, alignments, header, footer). Kernel crashed? Re-run the cell and the panel comes back.
- `ScreenBuffer(log_file="x", start=True)`: **start fresh**; existing files for that name are emptied.
- An explicit `header=` / `footer=` always wins over a stale saved one.

### Limits and long runs

| Option | Limits |
|---|---|
| `max_main=N` | Lines kept in the **history** file (oldest dropped) |
| `max_wait=N` | Lines kept in the **saved buffer** file (oldest dropped) |
| `max_display=N` | Lines **drawn** |

`max_wait` cannot be smaller than `max_display`. **Set `max_wait` (500–1000 is typical) on long runs**; it keeps the saved copy small and makes appends cheap.

These cap files, **not memory**. The in-memory buffer keeps every line (roughly 100 MB per million short lines).

### Changing settings at runtime

```python
sb.update(max_display=20, width=90, wait=False)
sb.update(log_file="second_phase")      # switch to a different log set
```

Settable: `wait`, `max_main`, `max_wait`, `max_display`, `shift_max_display`, `auto_clean_edges`, `auto_clean_silent_only_display`, `header`, `footer`, `header_alignment`, `footer_alignment`, `show_header`, `show_footer`, `silent_display`, `only_display`, `width`, `log_file`.

`update()` checks everything first: if any value is invalid it raises and changes **nothing**. Unknown names raise `ValueError`.

### Reading state

```python
sb.get_text()          # the current panel as one string
sb.display_index()     # prints every buffer line with its index (debugging)
sb.buffer              # the raw list of lines
```

### Threads

`put()`, `update()`, `delete()` and drawing are thread-safe. Disk writes keep buffer order even when several threads write at once.

---

## Method reference

| Method | Purpose |
|---|---|
| `put(text, state=0, index=None, fill_value=None, alignment=None)` | Add or replace a line |
| `update(**settings)` | Change any setting (validated first) |
| `display()` | Draw now |
| `flush()` | Clear the Jupyter output now |
| `delete(index, contains=None, exact=True)` | Delete by index/slice, optionally filtered by text |
| `clear()` | Empty the buffer |
| `clear_all()` | Empty the buffer, header and footer |
| `set_header(text, save_to_file=True, refresh=True)` / `set_footer(...)` | Set header/footer |
| `get_header()` / `get_footer()` | Current header/footer text |
| `get_text()` | Panel as one string |
| `read_log(last=None, show=True)` | Read the history log |
| `display_index()` | Print lines with indexes |
| `refresh_buffer_from_wait_log()` | Reload the buffer from the saved file (only when the buffer is empty) |

---

## Example: network scraper with an error banner

```python
import time, random
from screenbuffer import ScreenBuffer

sb = ScreenBuffer(max_display=8, width=70,
                  header="SCRAPER | ONLINE", header_alignment="center",
                  footer_alignment="center",
                  log_file="scraper", max_wait=500, start=True)

for t in range(120):
    online = not (40 <= t < 70)
    if online:
        sb.update(header=f"SCRAPER | ONLINE | {t}s", show_footer=False)
        sb.put(f"[{t}s] page {t//2}: +{random.randint(3, 9)} items")
    else:
        sb.update(header=f"SCRAPER | OFFLINE | {t}s",
                  footer="!! NETWORK ERROR - retrying !!")
        if t % 5 == 0:
            sb.put(f"[{t}s] retrying...", state=1, alignment=4)
    sb.display()                       # one redraw per step
    time.sleep(0.5)

sb.read_log(last=5)
```

The error banner appears at the bottom during the outage and disappears when the network returns, while the page counter keeps climbing above it.

---

## Performance (measured in a sandbox)

| Case | Before tuning | Now |
|---|---|---|
| `put()` with 20k lines in the buffer (no log / log / log + `max_main`) | 2.1 / 10.7 / 16.4 ms | 0.005 / 0.035 / 0.11 ms |
| Draw 8 lines out of 100k | 30.5 ms | 0.72 ms |
| Frame wait with a 20 ms disk delay | 202 ms | 0.1 ms |
| Old frame cleared → new text shown | 100–200 ms | ~0.5 ms |
| Redraws in a 300-step demo (one draw per step) | 241 | 81 |

These come from a Linux sandbox with a simulated Jupyter kernel. Real browser rendering was not measurable there; if you see flicker, see Troubleshooting.

## Known limits

- **Jupyter only.** Terminals and plain scripts are not supported (drawing is a no-op without IPython).
- **Replacing or deleting an old line rewrites the saved-buffer file.** Appending is cheap; edits cost O(lines in that file), which `max_wait` keeps small.
- **`max_main` trimming** rewrites the history file once it goes over the cap.
- **Memory is not capped** (see Limits).
- **Queued writes** can be lost if the kernel is killed abruptly. Call `sb.read_log()` (it waits for writes) or let the cell finish before shutting down.
- **Very fast loops** can exceed Jupyter's output rate limit regardless of this tool. Draw once per chunk or add a short sleep.
- **Wide characters** (emoji, CJK) count as one column for alignment, so `center`/`right` can be off by a column or two.
- Not tested on Windows, in real browsers, or in the Jupyter console (the console front end).

## Troubleshooting

| Symptom | Likely cause / fix |
|---|---|
| Nothing appears | `wait=True` (default): call `sb.display()`, or create with `wait=False` |
| Flicker in a fast loop | Use `wait=True` and call `sb.display()` once per step (or per chunk of items); add a tiny sleep |
| Header and footer change on separate frames | Use `wait=True`, make both updates, then call `sb.display()` once |
| `center` / `right` look off | Set `width=` to your output area's real width |
| `ValueError: max_wait cannot be less than max_display` | Raise `max_wait` or lower `max_display` |
| Old panel returns after re-running a cell | Saved buffer was restored; use `start=True` to begin fresh |
| `read_log()` says no log | Create the buffer with `log_file="name"` |
| Output scrolls instead of replacing | Output area is detached or using a front end without `clear_output` support |
