Metadata-Version: 2.4
Name: trueloss
Version: 0.2.0.2
Summary: A Keras utility for measuring training loss and metrics in evaluation mode for clearer comparison with validation performance.
Home-page: https://github.com/trueloss/trueloss
Author: Siddique Abusaleh
Author-email: trueloss.py@gmail.com
License: MIT
Project-URL: Documentation, https://github.com/trueloss/trueloss#readme
Project-URL: Source Code, https://github.com/trueloss/trueloss
Project-URL: Issue Tracker, https://github.com/trueloss/trueloss/issues
Keywords: keras,tensorflow,machine learning,deep learning,training loss,validation loss,regularization,model evaluation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Natural Language :: English
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: tensorflow>=2.16
Requires-Dist: keras>=2.0
Requires-Dist: matplotlib>=3.0
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: project-url
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# trueloss

`trueloss` is a Python library designed to compute and plot training loss and metrics on the training data under evaluation conditions, allowing them to be compared more directly with validation performance in Keras models.

The library was created to address a recurring observation in machine learning experiments: in some training runs, the model reports a higher loss on the training data than on the validation data.

This can occur because training and validation losses are often computed under different conditions. During training, the reported loss may include the data loss, regularization penalties, and the effects of training-only model behaviour. Validation, by contrast, is performed in evaluation mode.

As a simplified example:

1. **Training objective**

**Training Loss = Data Loss + Regularization Loss**

For L2 regularization:

**Training Loss = Data Loss + λ‖θ‖²**

2. **Comparable training-data evaluation**

**trueloss = Data Loss**

Here, the loss is evaluated on the training data while the model is operating in evaluation mode.


As a result, the conventional training and validation curves may not always represent directly comparable quantities.

`trueloss` makes this distinction visible by measuring the model's performance on the training data using the same evaluation conditions applied to validation data. It does not replace the training loss reported by Keras. Instead, it adds a comparable loss and corresponding metrics to the existing Keras training history.

Version `0.2.0` adds compatibility with Keras 3 while preserving the existing `trueloss` workflow. It also extends support beyond classification to regression problems, including regression components used in applications such as bounding-box prediction in object detection.

## Features

* Compatible with Keras 3.
* Supports both classification and regression workflows.
* Computes training loss under evaluation conditions for clearer comparison with validation loss.
* Computes corresponding evaluation-mode values for a wider range of Keras metrics.
* Supports regression components used in applications such as object detection.
* Plots training, validation, and comparable training-data performance.
* Accepts the same `fit()` parameters and defaults as Keras `model.fit()`.
* Returns the same Keras `History` instance produced by `model.fit()`.
* Adds `base_loss` and corresponding `base_<metric_name>` values to the training history.
* Preserves the behaviour of the original Keras model and existing training workflow.

## Installation

Install `trueloss` from PyPI:

```bash
pip install trueloss
```


## Usage

### Basic Usage

Import TensorFlow and the `trueloss` class:

```python
import tensorflow as tf
from trueloss import trueloss
```

The examples below use Keras with the TensorFlow backend.

### Creating and Compiling a Model

Create and compile the Keras model as usual:

```python
model = tf.keras.models.Sequential([
    tf.keras.layers.Input(shape=(784,)),
    tf.keras.layers.Dense(64, activation="relu"),
    tf.keras.layers.Dense(10, activation="softmax"),
])

model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"],
)
```

### Training the Model with trueloss

Create a `trueloss` instance and call its `fit()` method using the same parameters that you would pass to `model.fit()`:

```python
true_loss_instance = trueloss(
    model=model,
    plot=True,
)

history = true_loss_instance.fit(
    x_train,
    y_train,
    validation_data=(x_val, y_val),
    epochs=10,
    batch_size=32,
    verbose=1,
)
```

The public training workflow remains unchanged:

```python
history = true_loss_instance.fit(...)
```

can be used in place of:

```python
history = model.fit(...)
```

All positional and keyword arguments are passed through using the Keras `fit()` interface.

## Training History

The object returned by `true_loss_instance.fit()` is the same Keras `History` instance returned by `model.fit()`.

For a model compiled with accuracy as a metric, the history includes:

```python
history.history["loss"]
history.history["accuracy"]

history.history["val_loss"]
history.history["val_accuracy"]

history.history["base_loss"]
history.history["base_accuracy"]
```

The additional entries represent performance on the training data measured under evaluation conditions.

For models using other metrics, `trueloss` adds the corresponding values using the `base_` prefix. For example:

```python
history.history["base_mae"]
history.history["base_mse"]
```

The exact metric names depend on the metrics supplied when compiling the Keras model.

## Important Notes

### Model Behaviour

The original model instance continues to behave normally after training through `trueloss`.

You can continue to use it for prediction:

```python
predictions = model.predict(x_test)
```

evaluation:

```python
results = model.evaluate(x_test, y_test)
```

saving:

```python
model.save("model.keras")
```

and any other standard Keras operation.

### Keras Integration

`trueloss.fit()` accepts the same training inputs and configuration parameters as `model.fit()`, including positional and keyword arguments.

Existing callbacks can still be passed normally:

```python
history = true_loss_instance.fit(
    x_train,
    y_train,
    validation_data=(x_val, y_val),
    epochs=10,
    callbacks=[early_stopping, model_checkpoint],
)
```

### Verbose Output

The normal Keras training output is preserved:

```python
history = true_loss_instance.fit(
    x_train,
    y_train,
    epochs=10,
    verbose=1,
)
```

### trueloss Is an Additional Signal

`base_loss` does not replace the loss reported by Keras.

The two values answer different questions:

* `loss` represents the loss reported during the optimization process.
* `base_loss` represents performance on the training data under evaluation conditions.

Keeping both values makes it easier to distinguish optimization behaviour from measured predictive performance.

## Regression Usage

Version `0.2.0` extends support beyond classification models.

A regression model can be compiled and trained in the same way:

```python
regression_model = tf.keras.models.Sequential([
    tf.keras.layers.Input(shape=(20,)),
    tf.keras.layers.Dense(64, activation="relu"),
    tf.keras.layers.Dense(32, activation="relu"),
    tf.keras.layers.Dense(1),
])

regression_model.compile(
    optimizer="adam",
    loss="mean_squared_error",
    metrics=["mean_absolute_error"],
)

true_loss_instance = trueloss(
    model=regression_model,
    plot=True,
)

history = true_loss_instance.fit(
    x_train,
    y_train,
    validation_data=(x_val, y_val),
    epochs=20,
    batch_size=32,
)
```

The resulting history includes values such as:

```python
history.history["loss"]
history.history["val_loss"]
history.history["base_loss"]

history.history["mean_absolute_error"]
history.history["val_mean_absolute_error"]
history.history["base_mean_absolute_error"]
```

The same approach can be used with regression components in more complex models, including bounding-box coordinate regression in object-detection workflows.

## Using a Dataset

Keras datasets and data loaders can be passed directly to `fit()`:

```python
train_dataset = tf.data.Dataset.from_tensor_slices(
    (x_train, y_train)
).batch(32)

validation_dataset = tf.data.Dataset.from_tensor_slices(
    (x_val, y_val)
).batch(32)

history = true_loss_instance.fit(
    train_dataset,
    validation_data=validation_dataset,
    epochs=10,
)
```

## Plotting Only

To plot an existing Keras history without fitting a model, initialize `trueloss` without a model and call `plot_fn()` directly:

```python
true_loss_instance = trueloss(plot=True)

true_loss_instance.plot_fn(history)
```

## Complete Classification Example

```python
import tensorflow as tf
from trueloss import trueloss


# Load and preprocess the data
(x_train, y_train), (x_val, y_val) = (
    tf.keras.datasets.mnist.load_data()
)

x_train = x_train.astype("float32") / 255.0
x_val = x_val.astype("float32") / 255.0


# Create and compile the model
model = tf.keras.models.Sequential([
    tf.keras.layers.Input(shape=(28, 28)),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation="relu"),
    tf.keras.layers.Dropout(0.2),
    tf.keras.layers.Dense(10, activation="softmax"),
])

model.compile(
    optimizer="adam",
    loss="sparse_categorical_crossentropy",
    metrics=["accuracy"],
)


# Initialize trueloss
true_loss_instance = trueloss(
    model=model,
    plot=True,
)


# Train the model
history = true_loss_instance.fit(
    x_train,
    y_train,
    validation_data=(x_val, y_val),
    epochs=10,
    batch_size=32,
    verbose=1,
)


# Access the additional values
print(history.history["base_loss"])
print(history.history["base_accuracy"])
```

You can also view the [example notebook](https://github.com/trueloss/trueloss/blob/main/tests/example.ipynb).

## Project Links

* [PyPI](https://pypi.org/project/trueloss/)
* [Source code](https://github.com/trueloss/trueloss)
* [Issue tracker](https://github.com/trueloss/trueloss/issues)
* [Example notebook](https://github.com/trueloss/trueloss/blob/main/tests/example.ipynb)

## Contributing

Contributions are welcome.

To suggest an improvement, report unexpected behaviour, or contribute a fix, open an issue or submit a pull request through the GitHub repository.

Technical reviews, edge cases, and compatibility reports from research and production workflows are particularly valuable.

## License

This project is licensed under the MIT License. See the [LICENSE](https://github.com/trueloss/trueloss/blob/main/LICENSE) file for details.

## Contact

For questions, feedback, or support, use the [GitHub issue tracker](https://github.com/trueloss/trueloss/issues) or contact Siddique Abusaleh at `trueloss.py@gmail.com`.

## Acknowledgements

`trueloss` is built for Keras and TensorFlow workflows. Thanks to the contributors and maintainers of these projects for their work on the machine-learning ecosystem.

## Citation

If you use `trueloss` in research, please consider citing it:

```bibtex
@misc{trueloss2026,
  author       = {Siddique Abusaleh},
  title        = {trueloss: Comparable Training Loss and Metrics for Keras Models},
  year         = {2026},
  publisher    = {GitHub},
  journal      = {GitHub repository},
  howpublished = {\url{https://github.com/trueloss/trueloss}}
}
```

By using `trueloss`, you can preserve the standard Keras training workflow while gaining a clearer view of model performance on the training data.
