<h1 id="Performance-Analysis">Performance Analysis</h1>

<h2 id="Setup">Setup</h2>

<h3 id="Imports">Imports</h3>

import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt

import sys
sys.path.append("..")

from my_project.dataset import AutomobileDataset

<h3 id="Constants">Constants</h3>

PROCESSED_TEST_FILE = "../data/processed/automobile_test"
MODEL_PATH = "../models/automobile_model.pth"

<h2 id="Load-model">Load model</h2>

test_dataset = AutomobileDataset(
    f"{PROCESSED_TEST_FILE}.parquet"
)

# We create the same model architecture and import the trained mdoel
input_size = test_dataset.X.shape[1] 

model = nn.Sequential(
    nn.Linear(input_size, 32),
    nn.ReLU(),
    nn.Linear(32, 1)
)

model.load_state_dict(
    torch.load(MODEL_PATH)
)

<All keys matched successfully>

<h2 id="Performance-Analysis">Performance Analysis</h2>

predicciones = []
reales = []

with torch.no_grad():
    for X, y in test_dataset:
        prediccion = model(X.unsqueeze(0)).item()
        predicciones.append(prediccion)
        reales.append(y.item())

predicciones = np.array(predicciones)
reales = np.array(reales)

errores = predicciones - reales
mae = np.mean(np.abs(errores))
rmse = np.sqrt(np.mean(errores ** 2))
r2 = 1 - np.sum(errores ** 2) / np.sum((reales - reales.mean()) ** 2)

print(f"MAE: {mae:.2f}")
print(f"RMSE: {rmse:.2f}")
print(f"R²: {r2:.3f}")

MAE: 5842.58
RMSE: 8224.33
R²: 0.583

<p>The model has MAE of €5,842.58 (predictions differ from the actual price by approximately €5,843). The RMSE is €8,224.33, higher than the MAE, meaning that there are some vehicles for which the prediction error is larger. The R squared of 0.583 indicates that the model explains approximately 58.3% of the variability in prices. Overall, the model shows moderate predictive performance, although it could be improved.</p>

ANALYSIS_IMAGES_FOLDER = "../reports/figures/model_analysis/"

plt.figure(figsize=(6, 6))
plt.scatter(reales, predicciones, alpha=0.6)
plt.plot(
    [reales.min(), reales.max()],
    [reales.min(), reales.max()],
    "r--"
)
plt.xlabel("Precio real")
plt.ylabel("Precio predicho")
plt.title("Predicciones frente a valores reales")

plt.savefig(
    f"{ANALYSIS_IMAGES_FOLDER}predictions.png",
    dpi=300,
    bbox_inches="tight"
)

plt.show()