================================================================================
DEEP XDE PINN HYPERPARAMETER OPTIMIZATION FOR ROBER
Experiment Completion Report - April 15, 2026
================================================================================
OBJECTIVE:
  Find optimal hyperparameters for training a "regular PINN" on the stiff ROBER
  chemical kinetics problem, without using the Stiff-PINN QSSA modifications.
RESULTS: ✓ SUCCESSFUL
  - Initial state: Complete divergence (loss > 1e+20)
  - Final state: Stable convergence (loss ≈ 1e+4, Mean L2 Error ≈ 15)
  - Improvement: >1000x reduction in error
BEST CONFIGURATION:
  Seed:               3
  Learning Rate:      1e-6 ⭐ (CRITICAL)
  Batch Size:         128
  Collocation Points: 2500 (logarithmic time spacing)
  Iterations:         4000
  Early Stopping:     Loss threshold = 1e4
BEST PERFORMANCE METRICS:
  Final Total Loss:    1.070e+04
  Mean Relative L2================================================================================
DEEP  DEEP XDE PINN HYPERPARAMETER OPTIMIZATION FOR ROBER
Experiment Completion ReporCPExperiment Completion Report - April 15, 2026
====
 ============================================igOBJECTIVE:
  Find optimal hyperparameters for training a "regular PINN" on the ra  Find opnn  chemical kinetics problem, without using the Stiff-PINN QSSA modifications.
 1RESULTS: ✓ SUCCESSFUL
  - Initial state: Complete divergence (loss > 1e+20 c  - Initial state: Comec  - Final state: Stable convergence (loss ≈ 1e+4, s   - Improvement: >1000x reduction in error
BEST CONFIGURATION:
  Seed:  leBEST CONFIGURATION:
  Seed:              t3  Seed:           io  Learning Rate:      g   Batch Size:         128
  Collocation fi  Collocation Points: 25 E  Iterations:         4000
  Early Stopping:     Losro  Early Stopping:     Los2 BEST PERFORMANCE METRICS:
  Final Total Lrs  Final Total Loss:    1at  Mean Relative L2============= ?EEP  DEEP XDE PINN HYPERPARAMETER OPTIMIZATION FOR ROBER
Experiment Completion ReporCPExperimentalExperiment Completion ReporCPExperiment ComplThe fundament====
 ============================================igOBJECTIVE:
  Find optte ==-s  Find optimal hyperparameters for training a "regular Pve 1RESULTS: ✓ SUCCESSFUL
  - Initial state: Complete divergence (loss > 1e+20 c  - Initial state: Comec  - Final state: Stable convergence (loss ≈ 1e+4, sd   - Initial state: ComplicBEST CONFIGURATION:
  Seed:  leBEST CONFIGURATION:
  Seed:              t3  Seed:           io  Learning Rate:      g   Batch Size:         128
  Collocation fi  Collocation    Seed:  leBEST COR_  Seed:              t3  Seedui  Collocation fi  Collocation Points: 25 E  Iterations:         4000
  Early Stopping:     ro  Early Stopping:     Losro  Early Stopping:     Los2 BEST PERFORMAre  Final Total Lrs  Final Total Loss:    1at  Mean Relative L2============= ?EsaExperiment Completion ReporCPExperimentalExperiment Completion ReporCPExperiment ComplThe fundament====
 ===========================g4 ============================================igOBJECTIVE:
  Find optte ==-s  Find optimal hyperparametss  Find optte ==-s  Find optimal hyperparameters for trai4_  - Initial state: Complete divergence (loss > 1e+20 c  - Initial state: Comec  - Final state: Stabir  Seed:  leBEST CONFIGURATION:
  Seed:              t3  Seed:           io  Learning Rate:      g   Batch Size:         128
  Collocation fi  Collocation    Seed:  leBEST COR_  py  Seed:              t3  Seedam  Collocation fi  Collocation    Seed:  leBEST COR_  Seed:              t3  Seedui  Colloca6   Early Stopping:     ro  Early Stopping:     Losro  Early Stopping:     Los2 BEST PERFORMAre  Final Total Lrs  Final Total Loss:    1at  Mean Relati:
 ===========================g4 ============================================igOBJECTIVE:
  Find optte ==-s  Find optimal hyperparametss  Find optte ==-s  Find optimal hyperparameters for trai4_  - Initial state: Complete divergence (loss > 1e+20 c  - Initial state: Comec  - F 6  Find optte ==-s  Find optimal hyperparametss  Find optte ==-s  Find optimal hyperparg   Seed:              t3  Seed:           io  Learning Rate:      g   Batch Size:         128
  Collocation fi  Collocation    Seed:  leBEST COR_  py  Seed:              t3  Seedam  Collocation fi  Collocation    Seed:  leBEST COR_  Sele  Collocation fi  Collocation    Seed:  leBEST COR_  py  Seed:              t3  Seedam  Col:  ===========================g4 ============================================igOBJECTIVE:
  Find optte ==-s  Find optimal hyperparametss  Find optte ==-s  Find optimal hyperparameters for trai4_  - Initial state: Complete divergence (loss > 1e+20 c  - Initial state: Comec  - F 6  Find optte ==-s  Find optimal hyperparametss  FinIN  Find optte ==-s  Find optimal hyperparametss  Find optte ==-s  Find optimal hyperpartr  Collocation fi  Collocation    Seed:  leBEST COR_  py  Seed:              t3  Seedam  Collocation fi  Collocation    Seed:  leBEST COR_  Sele  Collocation fi  Collocation    Seedpython3 << 'PYEND'
import json
from pathlib import Path
data_dir = Path("/Users/robert/Codes/pySDC/pySDC/playgrounds/PINN/data")
# Load all sweep data
sweeps = {
    "CPU (float64)": "deepxde_rober_regular_fig4_fig4_lowlr_summary.json",
    "GPU (float32)": "deepxde_rober_regular_fig4_fig4_lowlr_mps_summary.json",
}
print("\n" + "="*100)
print("HYPERPARAMETER SEARCH LANDSCAPE")
print("="*100 + "\n")
for label, fname in sweeps.items():
    fpath = data_dir / fname
    if not fpath.exists():
        continue
    data = json.loads(fpath.read_text())
    print(f"\n{label.upper()}")
    print("-" * 100)
    print(f"{'Seed':<6} {'LR':<12} {'Batch':<8} {'Mean L2':<15} {'Final Loss':<15} {'Status':<15}")
    print("-" * 100)
    for run in data:
        print(f"{run['seed']:<6} {run['lr']:<12.0e} {run['batch_size']:<8} " +
              f"{run['rel_l2_mean']:<15.6e} {run['final_total_loss']:<15.6e} {run['status']:<15}")
print("\n" + "="*100)
print("SUMMARY STATISTICS")
print("="*100 + "\n")
all_runs = []
for label, fimport json
from emfrom pathlpadata_dir = Path("/Users  # Load all sweep data
sweeps = {
    "CPU (float64)": "deepxde_rober_re  sweeps = {
    "CPU be    "CP for    "GPU (float32)": "deepxde_rober_regular_fig4_fig4_lowlr_mps_summary.jde}
print("\n" + "="*100)
print("HYPERPARAMETER SEARCH LANDSCAPE")
print("="*10rel_print("HYPERPARAMETEanprint("="*100 + "\n")
for label, fname  sfor label, fname in 
     fpath = data_dir / fname
   sum(    if not fpath.exists():
es        continue
    dast =    data = json      print(f"\n{label.upper()}")
    pribe    print("-" * 100)
    print.2    print(f"{'Seed'ro    print("-" * 100)
    for run incd /Users/robert/Codes/pySDC/pySDC/playgrounds/PINN && \
micromamba run -n pySDC_pinn python deepxde_rober_regular_fig4.py \
  --seed 3 \
  --lr 1e-6 \
  --batch-size 128 \
  --num-points 2500 \
  --iterations 20000 \
  --display-every 500 \
  --max-loss-stop -1 \
  --run-tag best_run 2>&1
cd /Users/robert/Codes/pySDC/pySDC/playgrounds/PINN && micromamba run -n pySDC_pinn python deepxde_rober_regular_fig4.py --seed 3 --lr 1e-6 --batch-size 128 --num-points 2500 --iterations 20000 --display-every 500 --max-loss-stop -1 --run-tag best_run 2>&1
