Metadata-Version: 2.4
Name: beast-logger
Version: 0.1.5
Summary: A package for advanced logging and visualization of Python objects, especially tensors.
Home-page: https://github.com/beast-logger
Author: qingxu.fu@alibaba-inc.com
Author-email: qingxu.fu@alibaba-inc.com
Project-URL: Bug Reports, https://github.com/beast-logger/issues
Project-URL: Source, https://github.com/beast-logger
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.7
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: loguru
Requires-Dist: rich
Requires-Dist: jieba
Requires-Dist: pydantic
Requires-Dist: fastapi
Requires-Dist: uvicorn
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license-file
Dynamic: project-url
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

<div align="center">

<img src="https://serve.gptacademic.cn/publish/auto/beastlogger/beast_banner.jpg" alt="Beast Logger Banner" width="100%" />

<br/>

<img src="https://serve.gptacademic.cn/publish/auto/beastlogger/beast_logo.jpg" alt="Beast Logger Logo" width="160" />

# Beast Logger

**Advanced Python Logging for Machine Learning**

[![PyPI version](https://img.shields.io/pypi/v/beast-logger.svg?color=00d4ff&labelColor=0d1117)](https://pypi.org/project/beast-logger/)
[![Python](https://img.shields.io/badge/python-3.7%2B-blue?color=8b5cf6&labelColor=0d1117)](https://pypi.org/project/beast-logger/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg?labelColor=0d1117)](https://opensource.org/licenses/MIT)
[![PyPI Downloads](https://img.shields.io/pypi/dm/beast-logger?color=f59e0b&labelColor=0d1117)](https://pypi.org/project/beast-logger/)

*Built for ML researchers who need structured, beautiful, and searchable logs — not walls of unreadable text.*

[快速开始](#-installation) · [API 文档](#-api-reference) · [Web 查看器](#-web-log-viewer) · [示例](#-examples)

</div>

---

## ✨ What is Beast Logger?

**Beast Logger** is a drop-in Python logging library purpose-built for Machine Learning workflows. It transforms raw Python data structures — lists, dicts, tensors, LLM token arrays — into rich, color-coded terminal tables and an interactive web log viewer.

Whether you're training a language model with SFT, RLHF, or GRPO, Beast Logger gives you a structured window into every step of your experiment — with zero overhead and maximum clarity.

### Key Capabilities

| Feature | Description |
|:--------|:------------|
| **Rich Terminal Tables** | Renders dicts, lists, nested structures as beautiful terminal widgets |
| **Web Log Viewer** | Browse, filter, and copy logs from a local web app at `localhost:8181` |
| **Tensor Logging** | Log PyTorch tensor shape, dtype, device, and value previews |
| **LLM Token Viewer** | Color-coded token-level visualization with hover tooltips |
| **Multi-Mod Routing** | Route logs to separate files per experiment module |
| **Multi-Language** | Optimized for English, Chinese, and many other languages |
| **Zero Config** | One import, one function call — it just works |

---

## 📦 Installation

```bash
pip install beast-logger
```

**From Aliyun Mirror (China):**
```bash
pip install beast-logger -i https://mirrors.aliyun.com/pypi/simple/
```

<details>
<summary>Build from source</summary>

```bash
# Clean previous builds
rm -rf build dist web_display_dist
rm -rf web_display/build web_display/dist
rm -rf beast_logger.egg-info

# Build web assets
cd web_display
nvm install 16 && nvm use 16
npm install
npm run build:all
cd ..

# Package and install
mkdir -p web_display_dist
mv web_display/build web_display_dist/build_pub
python setup.py sdist bdist_wheel
pip install dist/beast_logger-*.whl
```

</details>

---

## 🚀 Quick Start

```python
from beast_logger import register_logger, print_dict

# Initialize logging — declare which modules to log
register_logger(mods=["train"])

# Log any Python dictionary as a rich table
print_dict(
    {"epoch": 1, "loss": 0.342, "lr": 3e-4, "acc": 0.871},
    mod="train"
)
```

**Terminal output:**

```
╭────────────────────────────────────────────────╮
│ ┌──────────────────────┬─────────────────────┐ │
│ │ epoch                │ 1                   │ │
│ ├──────────────────────┼─────────────────────┤ │
│ │ loss                 │ 0.342               │ │
│ ├──────────────────────┼─────────────────────┤ │
│ │ lr                   │ 0.0003              │ │
│ ├──────────────────────┼─────────────────────┤ │
│ │ acc                  │ 0.871               │ │
│ └──────────────────────┴─────────────────────┘ │
╰────────────────────────────────────────────────╯
```

---

## 📋 API Reference

### `register_logger()`

Initialize the logger before logging anything.

```python
register_logger(
    mods=[],               # Modules logged to console + file
    non_console_mods=[],   # Modules logged to file only (silent)
    base_log_path="logs",  # Root directory for log files
    auto_clean_mods=[],    # Modules whose old logs are deleted on start
    rotation="100 MB"      # Max size per log file before rotation
)
```

> **Tip:** Use `mod="console"` in any print function to log to terminal only, without writing to any file.

---

### Core Logging Methods

#### `print_dict(dict_like, mod, ...)`

Log a flat dictionary as a two-column key-value table.

```python
from beast_logger import print_dict

print_dict({"loss": 0.25, "reward": 1.8, "kl": 0.03}, mod="rlhf")
```

```
╭────────────────────────────────────────────────╮
│ ┌──────────────────────┬─────────────────────┐ │
│ │ loss                 │ 0.25                │ │
│ ├──────────────────────┼─────────────────────┤ │
│ │ reward               │ 1.8                 │ │
│ ├──────────────────────┼─────────────────────┤ │
│ │ kl                   │ 0.03                │ │
│ └──────────────────────┴─────────────────────┘ │
╰────────────────────────────────────────────────╯
```

---

#### `print_list(list_like, mod, ...)`

Log a Python list as a single-column table.

```python
from beast_logger import print_list

print_list(["step_1", "step_2", "step_3"], mod="console")
```

---

#### `print_listofdict(list_of_dicts, mod, narrow=False, ...)`

Log a list of dictionaries as a row-wise table (each dict = one row).

```python
from beast_logger import print_listofdict

print_listofdict([
    {"model": "llama-7b",  "loss": 0.31, "acc": 0.82},
    {"model": "llama-13b", "loss": 0.24, "acc": 0.89},
    {"model": "llama-70b", "loss": 0.18, "acc": 0.94},
], mod="eval", narrow=True)
```

```
╭────────────────────────────────────────────────────────╮
│ ┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ │
│ ┃           ┃ model     ┃ loss     ┃ acc             ┃ │
│ ┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │
│ │ 0         │ llama-7b  │ 0.31     │ 0.82            │ │
│ ├───────────┼───────────┼──────────┼─────────────────┤ │
│ │ 1         │ llama-13b │ 0.24     │ 0.89            │ │
│ ├───────────┼───────────┼──────────┼─────────────────┤ │
│ │ 2         │ llama-70b │ 0.18     │ 0.94            │ │
│ └───────────┴───────────┴──────────┴─────────────────┘ │
╰────────────────────────────────────────────────────────╯
```

---

#### `print_dictofdict(dict_of_dicts, mod, header="", attach="", ...)`

Log a nested dictionary as a matrix table (outer keys = rows, inner keys = columns).

```python
from beast_logger import print_dictofdict

print_dictofdict(
    {
        "run_1": {"loss": 0.34, "reward": 1.2, "kl": 0.05},
        "run_2": {"loss": 0.28, "reward": 1.6, "kl": 0.03},
    },
    header="GRPO Training Step 100",
    mod="grpo",
    attach="Full config: lr=3e-4, batch=32, clip=0.2"  # Shown as copy button in web viewer
)
```

```
╭──────────────── GRPO Training Step 100 ─────────────────╮
│ ┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┓ │
│ ┃                      ┃ loss   ┃ reward  ┃ kl         ┃ │
│ ┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━┩ │
│ │ run_1                │ 0.34   │ 1.2     │ 0.05       │ │
│ ├──────────────────────┼────────┼─────────┼────────────┤ │
│ │ run_2                │ 0.28   │ 1.6     │ 0.03       │ │
│ └──────────────────────┴────────┴─────────┴────────────┘ │
╰─────────────────────────────────────────────────────────╯
```

---

### Tensor Logging Methods

> Requires `torch` to be installed.

#### `print_tensor(tensor, mod, ...)`

Log shape, dtype, device, and a value preview for a single tensor.

```python
import torch
from beast_logger import print_tensor

t = torch.randn(4, 512)
print_tensor(t, mod="debug")
```

#### `print_tensor_dict({name: tensor, ...}, mod, ...)`

Log a dictionary of tensors in a single compact table. Handles missing or malformed entries gracefully.

```python
from beast_logger import print_tensor_dict

print_tensor_dict(
    {"input_ids": input_ids, "attention_mask": mask, "labels": labels},
    mod="sft"
)
```

---

### LLM Token Logging

Log and visualize complex LLM token arrays with per-token metadata.

<div align="center">
    <img width="600" alt="Token Viewer" src="https://github.com/user-attachments/assets/26dcf109-236c-4b41-95bd-98d9b68434dc" />
</div>

```python
from beast_logger import register_logger, print_nested, NestedJsonItem, SeqItem

register_logger(mods=["rlhf"], base_log_path="./logs")

samples = {}
for i in range(5):
    samples[f"rollout.{i}"] = NestedJsonItem(
        item_id=f"sample_{i}",
        reward=round(1.2 + i * 0.1, 2),
        step=100,
        # Token-level visualization
        content=SeqItem(
            text=[f"Hello", f"world", f"<|im_end|>", f"Answer:", f"42"],
            title=[f"tok_{j}" for j in range(5)],   # Hover tooltip
            count=[str(j) for j in range(5)],        # Token index
            color=["blue", "blue", "red", "green", "green"]
        )
    )

print_nested(
    samples,
    main_content="RLHF Rollout Visualization",
    header="Step 100 — Batch 0",
    mod="rlhf",
    narrow=True,
    attach="Copy this entry to clipboard"
)
```

---

## 🌐 Web Log Viewer

Beast Logger ships with a built-in interactive web log viewer — no extra setup needed.

<div align="center">
    <img width="700" alt="Web Viewer" src="https://github.com/user-attachments/assets/5fa151d9-26e2-48ef-9565-ced714eb1617" />
</div>

### Start the Viewer

```bash
beast_logger_go
```

Then open **http://localhost:8181** in your browser.

### Features

- **Directory Browser** — Select any log directory (absolute path)
- **Module Filter** — Switch between different `mod` streams
- **One-Click Copy** — Copy any log entry with the `attach` field to clipboard
- **Token Viewer** — Interactive per-token hover details for LLM logs
- **Remote Access** — Set `beast_logger_WEB_SERVICE_URL` to serve logs over a network

### Remote Log Serving

```bash
export beast_logger_WEB_SERVICE_URL="http://your-server:8181/"
python your_training_script.py
```

Beast Logger will print the full URL to access your logs remotely.

---

## 🔧 Advanced Configuration

### Multiple Modules

Route different log streams to separate files:

```python
register_logger(
    mods=["train", "eval"],          # Console + file
    non_console_mods=["debug"],       # File only (silent)
    base_log_path="./experiment_001",
    auto_clean_mods=["debug"],        # Delete old debug logs on start
    rotation="50 MB"
)
```

Log files will be organized as:
```
experiment_001/
├── regular/
│   └── regular.log
├── train/
│   ├── train.log
│   └── train.json.log
├── eval/
│   ├── eval.log
│   └── eval.json.log
└── debug/
    ├── debug.log
    └── debug.json.log
```

### Change Log Path at Runtime

```python
from beast_logger import change_base_log_path

change_base_log_path("./experiment_002")
```

### Disable Console Colors

```bash
LOGURU_COLORIZE=NO python train.py
```

---

## 🧪 ML Training Integration

### SFT Training Loop

```python
from beast_logger import register_logger, print_dict, print_tensor_dict

register_logger(mods=["sft"], base_log_path="./logs/sft_run_001")

for step, batch in enumerate(dataloader):
    loss = model(batch)
    
    if step % 10 == 0:
        print_dict(
            {"step": step, "loss": loss.item(), "lr": scheduler.get_last_lr()[0]},
            mod="sft"
        )
    
    if step % 100 == 0:
        print_tensor_dict(
            {"input_ids": batch["input_ids"], "labels": batch["labels"]},
            mod="sft"
        )
```

### GRPO / RLHF Reward Logging

```python
from beast_logger import print_dictofdict

print_dictofdict(
    {
        f"sample_{i}": {
            "reward": rewards[i],
            "kl_div": kl_divs[i],
            "ref_logp": ref_logps[i],
        }
        for i in range(len(rewards))
    },
    header=f"GRPO Step {global_step}",
    mod="rlhf",
    attach=f"prompt={prompts[0][:80]}..."
)
```

---

## 📁 Project Structure

```
beast-logger/
├── beast_logger/
│   ├── __init__.py           # Public API exports
│   ├── register.py           # Logger initialization & mod routing
│   ├── print_basic.py        # print_dict, print_list, print_listofdict, print_dictofdict
│   ├── print_tensor.py       # print_tensor, print_tensor_dict
│   ├── print_nested.py       # print_nested, NestedJsonItem, SeqItem
│   ├── log_json.py           # JSON log writer
│   ├── serve_log_files.py    # File serving backend
│   └── web_launcher.py       # Web viewer launcher
├── web_display/              # React frontend for web log viewer
├── tests/                    # Test suite
├── requirements.txt
└── setup.py
```

---

## 📊 Dependencies

| Package | Purpose |
|:--------|:--------|
| `loguru` | Structured file logging with rotation |
| `rich` | Terminal table rendering |
| `jieba` | Chinese word segmentation |
| `pydantic` | Data validation for log entries |
| `fastapi` | Web viewer backend API |
| `uvicorn` | ASGI server for web viewer |

---

## 🤝 Contributing

1. Fork the repository
2. Create a feature branch: `git checkout -b feature/my-feature`
3. Run the test suite: `python -m pytest tests/`
4. Submit a Pull Request

For development setup, see [DEV.md](DEV.md).

---

## 📄 License

MIT License — see [LICENSE](LICENSE) for details.

---

<div align="center">

**Built with ❤️ at Alibaba Tongyi**

*Qingxu Fu · fuqingxu.fqx@alibaba-inc.com*

<img src="https://serve.gptacademic.cn/publish/auto/beastlogger/beast_logo.jpg" alt="Beast Logger" width="80" />

</div>
