Metadata-Version: 2.3
Name: pico-report
Version: 1.1.0
Summary: A Python package for uploading training metrics and checkpointing data to Pico backend databases
License: Apache 2.0
Author: Pico Team
Author-email: team@picolm.io
Requires-Python: >=3.10,<3.13
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Requires-Dist: click (>=8.1.7,<9.0.0)
Requires-Dist: pydantic (>=2.0.0,<3.0.0)
Requires-Dist: python-dotenv (>=1.0.0,<2.0.0)
Requires-Dist: requests (>=2.31.0,<3.0.0)
Description-Content-Type: text/markdown

# Pico Report

A Python package for uploading training metrics and checkpointing data to Pico backend databases. This package allows you to seamlessly integrate with your existing training pipelines while sending data to your private Pico dashboard.

## Installation

```bash
pip install pico-report
```

## Quick Start

First, set up your environment variables (recommended) or use direct configuration.

### Setup Environment Variables

```bash
# Copy the example file
cp .env.example .env

# Edit .env with your actual credentials (NEVER commit this file!)
# PICO_API_KEY=your_actual_api_key
# PICO_LAB_HASH=your_actual_lab_hash
```

### Using the Client

```python
from pico_report import PicoClient, ReporterConfig

# Method 1: Using environment variables (recommended - secure)
client = PicoClient()

# Method 2: Direct configuration (not recommended for production)
config = ReporterConfig(
    api_key="your-api-key",  # Required
    lab_hash="your-lab-hash",  # Required
    experiment_name="experiment-1"  # Optional
)
client = PicoClient(config=config)

# Log training metrics
client.log_metrics({
    "loss": 0.5,
    "accuracy": 0.85,
    "learning_rate": 0.001
}, step=100)

# Upload checkpoint data
client.upload_checkpoint_data({
    "model_state": "path/to/checkpoint",
    "optimizer_state": "path/to/optimizer",
    "epoch": 5
}, step=100)
```

## Configuration

### Environment Variables

Set the following **required** environment variables:

```bash
export PICO_API_KEY="your-api-key"
export PICO_LAB_HASH="your-lab-hash"
```

### Configuration File

Create a `.env` file in your project root:

```env
# Required
PICO_API_KEY=your-api-key
PICO_LAB_HASH=your-lab-hash

# Optional
PICO_EXPERIMENT_NAME=my-experiment
```

## High-Level Interface

For easier integration, use the `PicoReporter` class:

```python
from pico_report.integrations import PicoReporter

# lab_hash is required - provide it explicitly or via PICO_LAB_HASH environment variable
reporter = PicoReporter(
    lab_hash="my-lab-hash",  # Required
    experiment_name="transformer-training"  # Optional
)

# Setup experiment
reporter.setup_experiment(
    experiment_name="transformer-training",
    config_data={"lr": 0.001, "batch_size": 32},
    description="Training transformer model"
)

# Log training metrics
reporter.log_training_metrics({
    "loss": 0.5,
    "perplexity": 2.1
}, step=100)

# Log evaluation metrics
reporter.log_evaluation_metrics({
    "eval_loss": 0.45,
    "eval_accuracy": 0.87
}, step=100, prefix="validation")
```

## Integration with Existing Training Code

### PyTorch Lightning Integration

```python
import lightning as L
from pico_report.integrations import PicoReporter

class MyLightningModule(L.LightningModule):
    def __init__(self):
        super().__init__()
        # Requires PICO_API_KEY and PICO_LAB_HASH environment variables to be set
        self.pico_reporter = PicoReporter(
            experiment_name="lightning-training"
        )
    
    def training_step(self, batch, batch_idx):
        # Your training logic
        loss = self.compute_loss(batch)
        
        # Log to Pico
        if self.global_step % 10 == 0:
            self.pico_reporter.log_training_metrics({
                "train_loss": loss.item()
            }, step=self.global_step)
        
        return loss
    
    def validation_step(self, batch, batch_idx):
        # Your validation logic
        val_loss = self.compute_loss(batch)
        return {"val_loss": val_loss}
    
    def validation_epoch_end(self, outputs):
        avg_loss = torch.stack([x["val_loss"] for x in outputs]).mean()
        
        self.pico_reporter.log_evaluation_metrics({
            "val_loss": avg_loss.item()
        }, step=self.global_step)
```

### Direct Integration with Pico-Train

You can modify your existing pico-train setup to also send data to Pico backend:

```python
# In your training script
from pico_report.integrations import PicoReporter

# Initialize both wandb and pico reporter
wandb_logger = initialize_wandb(monitoring_config, checkpointing_config)
pico_reporter = PicoReporter(
    lab_hash=monitoring_config.pico.lab_hash,
    experiment_name=checkpointing_config.run_name
)

# In your training loop
for step, batch in enumerate(dataloader):
    # ... training logic ...
    
    # Log to both wandb and pico
    metrics = {"loss": loss.item(), "lr": lr}
    wandb_logger.log(metrics, step=step)
    pico_reporter.log_training_metrics(metrics, step=step)
```

## Configuration

The base URL should point to the report API endpoint. For local development:
```bash
export PICO_BASE_URL="http://localhost:3000/api/report"
```

For production:
```bash
export PICO_BASE_URL="https://picolabs.space/api/report"
```

## API Reference

### PicoClient

Main client for direct API interaction.

#### Methods

- `log_metrics(metrics, step, timestamp)`: Log training metrics
- `create_experiment(name, config_data, description)`: Create new experiment
- `list_experiments(limit, offset)`: List existing experiments

### PicoReporter

High-level interface for easier integration.

#### Methods

- `setup_experiment(name, config_data, description)`: Setup experiment
- `log_training_metrics(metrics, step, prefix)`: Log training metrics with prefix
- `log_evaluation_metrics(metrics, step, prefix)`: Log evaluation metrics with prefix
- `log_analysis_metrics(metric_name, metric_data, step, data_split, prefix)`: Log learning dynamics analysis metrics
- `log_system_metrics(**metrics)`: Log system performance metrics

## Error Handling

The package includes custom exceptions:

- `PicoReportError`: Base exception
- `PicoAuthError`: Authentication related errors
- `PicoUploadError`: Data upload errors
- `PicoConfigError`: Configuration errors

```python
from pico_report.exceptions import PicoAuthError, PicoUploadError

try:
    client.log_metrics(metrics, step=100)
except PicoAuthError:
    print("Authentication failed - check your API key")
except PicoUploadError as e:
    print(f"Upload failed: {e}")
```
