Best Practices¶

Guidelines for getting the best results with Pilz.

Data Preparation¶

Feature Types¶

Choose correct statistical types:

# Good examples

features:

  - name: status         # "active", "inactive" → categorial

    statistical: categorial

    type: string

  - name: age            # 0-100 → numerical

    statistical: numerical

    type: int

  - name: score          # 0.0-100.0 → numerical

    statistical: numerical

    type: float

Missing Values¶

Always handle missing values:

features:

  - name: income

    statistical: numerical

    type: float

    missing_value: 0  # Replace nulls with 0

Data Quality¶

• Check for outliers before training

• Ensure target column has no missing values

• Use consistent value formats (e.g., "Yes" vs "yes")

Training Settings¶

Starting Point¶

Begin with these safe defaults:

n: 1

max_depth: 10

n_dims: 1

n_cat: 5

Incrementally Increase Complexity¶

```mermaid flowchart TD START[Start] --> B["Baseline: n=1, n_dims=1"] B --> R{Results OK?} R -->|No| I["Increase n_dims=2"] I --> R2{Results OK?} R2 -->|No| M["Increase n=3-5"] M --> R3{Results OK?} R3 -->|No| D[Increase max_depth]

R -->|Yes| O[Optimize]
R2 -->|Yes| O
R3 -->|Yes| O

style START fill:#e0f0ff
style O fill:#ccffcc```

Parameter Guidelines¶

| Parameter | Low Value | High Value | Notes |

|-----------|-----------|------------|-------|

| n | 1 | 10+ | More trees = more accurate, slower |

| n_dims | 1 | 3 | Higher = more combinations, slower |

| n_cat | 3 | 10 | Higher = more bins, risk of overfitting |

| max_depth | 5 | 20 | Higher = deeper trees, risk of overfitting |

Model Selection¶

Bias-Variance Trade-off¶

```mermaid flowchart LR A[Start] --> B[Process] B --> C[End]

style A fill:#e0f0ff
style C fill:#ccffcc```

mermaid flowchart LR subgraph Underfitting U1[Simple model] --> U2[High bias] U2 --> U3[Low variance] end

subgraph Good
    G1[Balanced] --> G2[Balanced bias/variance]
end

subgraph Overfitting
    O1[Complex model] --> O2[Low bias]
    O2 --> O3[High variance]
end

style G1 fill:#ccffcc```

classes well - Domain knowledge suggests interactions - After establishing baseline

Evaluation¶

Test Set¶

• Always hold out test data
• Don't use training data for evaluation
• Use stratified sampling for class balance

Metrics¶

• AUC: Overall discrimination ability
• Accuracy: Simple but can be misleading for imbalanced data
• Per-class accuracy: Important for multi-class

Deployment¶

Model Export¶

• Keep models in version control
• Document settings used
• Test with sample predictions

SQL Production¶

• Test SQL in staging before production
• Monitor prediction distributions
• Set up alerts for unusual patterns

Performance Checklist¶

• [ ] Data cleaned and validated
• [ ] Correct feature types set
• [ ] Missing values handled
• [ ] Started with baseline settings
• [ ] Incremental tuning
• [ ] Test set properly separated
• [ ] AUC > 0.8 (or domain-specific)
• [ ] SQL tested in staging
• [ ] Model versioned