Metadata-Version: 2.4
Name: mono-dense-keras
Version: 0.1.1
Summary: Monotonic Dense Layer implemented in Keras
Home-page: https://github.com/airtai/mono-dense-keras
Author: AIRT Technologies d.o.o.
Author-email: info@airt.ai
License: Creative Commons License
Project-URL: Bug Tracker, https://github.com/airtai/mono-dense-keras/issues
Project-URL: CI, https://github.com/airtai/mono-dense-keras/actions
Project-URL: Documentation, https://mono-dense-keras.airt.ai/
Keywords: tensorflow monotone monotonic dense layer nbdev nbdev-mkdocs jupyter notebook python
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: Natural Language :: English
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: License :: Free for non-commercial use
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: tensorflow<2.16,>=2.10.0
Provides-Extra: dev
Requires-Dist: keras-tuner[bayesian]==1.3.5; extra == "dev"
Requires-Dist: nbdev-mkdocs==0.6.1; extra == "dev"
Requires-Dist: nbdev==2.3.12; extra == "dev"
Requires-Dist: fastcore==1.7.20; extra == "dev"
Requires-Dist: pytest==7.3.1; extra == "dev"
Requires-Dist: pandas>=1.3.5; extra == "dev"
Requires-Dist: nbqa==1.7.0; extra == "dev"
Requires-Dist: black==23.3.0; extra == "dev"
Requires-Dist: isort==5.12.0; extra == "dev"
Requires-Dist: matplotlib==3.7.1; extra == "dev"
Requires-Dist: seaborn==0.12.2; extra == "dev"
Requires-Dist: mypy==1.3.0; extra == "dev"
Requires-Dist: bandit==1.7.5; extra == "dev"
Requires-Dist: semgrep==1.23.0; extra == "dev"
Requires-Dist: tqdm; extra == "dev"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: license-file
Dynamic: project-url
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# Monotonic Dense Layer


<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->

## ⚠️ **Deprecation Notice** ⚠️

This package is no longer maintained and has been **replaced** by
`monotonic-nn` ([PyPi](https://pypi.org/project/monotonic-nn/) \|
[GitHub](https://github.com/airtai/monotonic-nn) \|
[Docs](https://monotonic.airt.ai/latest/))

Please update your dependencies accordingly.

------------------------------------------------------------------------

This Python library implements Monotonic Dense Layer as described in
Davor Runje, Sharath M. Shankaranarayana, “Constrained Monotonic Neural
Networks” \[[PDF](https://arxiv.org/pdf/2205.11775.pdf)\].

If you use this library, please cite:

``` title="bibtex"
@inproceedings{runje2023,
  title={Constrained Monotonic Neural Networks},
  author={Davor Runje and Sharath M. Shankaranarayana},
  booktitle={Proceedings of the 40th {International Conference on Machine Learning}},
  year={2023}
}
```

This package contains an implementation of our Monotonic Dense Layer
[`MonoDense`](https://airtai.github.io/mono-dense-keras/latest/api/mono_dense_keras/MonoDense/#mono_dense_keras.MonoDense)
(Constrained Monotonic Fully Connected Layer). Below is the figure from
the paper for reference.

In the code, the variable `monotonicity_indicator` corresponds to **t**
in the figure and parameters `is_convex`, `is_concave` and
`activation_weights` are used to calculate the activation selector **s**
as follows:

- if `is_convex` or `is_concave` is **True**, then the activation
  selector **s** will be (`units`, 0, 0) and (0, `units`, 0),
  respecively.

- if both `is_convex` or `is_concave` is **False**, then the
  `activation_weights` represent ratios between $\breve{s}$, $\hat{s}$
  and $\tilde{s}$, respecively. E.g. if `activation_weights = (2, 2, 1)`
  and `units = 10`, then

$$
(\breve{s}, \hat{s}, \tilde{s}) = (4, 4, 2)
$$

![mono-dense-layer-diagram](https://github.com/airtai/mono-dense-keras/raw/main/nbs/images/mono-dense-layer-diagram.png)

## Install

``` sh
pip install mono-dense-keras
```

## How to use

In this example, we’ll assume we have a simple dataset with three inputs
values $x_1$, $x_2$ and $x_3$ sampled from the normal distribution,
while the output value $y$ is calculated according to the following
formula before adding Gaussian noise to it:

$y = x_1^3 + \sin\left(\frac{x_2}{2 \pi}\right) + e^{-x_3}$

<style type="text/css">
</style>

| x0        | x1        | x2        | y         |
|-----------|-----------|-----------|-----------|
| 0.304717  | -1.039984 | 0.750451  | 0.234541  |
| 0.940565  | -1.951035 | -1.302180 | 4.199094  |
| 0.127840  | -0.316243 | -0.016801 | 0.834086  |
| -0.853044 | 0.879398  | 0.777792  | -0.093359 |
| 0.066031  | 1.127241  | 0.467509  | 0.780875  |

Now, we’ll use the
[`MonoDense`](https://airtai.github.io/mono-dense-keras/latest/api/mono_dense_keras/MonoDense/#mono_dense_keras.MonoDense)
layer instead of `Dense` layer to build a simple monotonic network. By
default, the
[`MonoDense`](https://airtai.github.io/mono-dense-keras/latest/api/mono_dense_keras/MonoDense/#mono_dense_keras.MonoDense)
layer assumes the output of the layer is monotonically increasing with
all inputs. This assumtion is always true for all layers except possibly
the first one. For the first layer, we use `monotonicity_indicator` to
specify which input parameters are monotonic and to specify are they
increasingly or decreasingly monotonic:

- set 1 for increasingly monotonic parameter,

- set -1 for decreasingly monotonic parameter, and

- set 0 otherwise.

In our case, the `monotonicity_indicator` is `[1, 0, -1]` because $y$
is: - monotonically increasing w.r.t. $x_1$
$\left(\frac{\partial y}{x_1} = 3 {x_1}^2 \geq 0\right)$, and

- monotonically decreasing w.r.t. $x_3$
  $\left(\frac{\partial y}{x_3} = - e^{-x_2} \leq 0\right)$.

``` python
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense, Input

from mono_dense_keras import MonoDense

model = Sequential()

model.add(Input(shape=(3,)))
monotonicity_indicator = [1, 0, -1]
model.add(
    MonoDense(128, activation="elu", monotonicity_indicator=monotonicity_indicator)
)
model.add(MonoDense(128, activation="elu"))
model.add(MonoDense(1))

model.summary()
```

    Model: "sequential_7"
    _________________________________________________________________
     Layer (type)                Output Shape              Param #   
    =================================================================
     mono_dense_21 (MonoDense)   (None, 128)               512       
                                                                     
     mono_dense_22 (MonoDense)   (None, 128)               16512     
                                                                     
     mono_dense_23 (MonoDense)   (None, 1)                 129       
                                                                     
    =================================================================
    Total params: 17,153
    Trainable params: 17,153
    Non-trainable params: 0
    _________________________________________________________________

Now we can train the model as usual using `Model.fit`:

``` python
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.optimizers.schedules import ExponentialDecay

lr_schedule = ExponentialDecay(
    initial_learning_rate=0.01,
    decay_steps=10_000 // 32,
    decay_rate=0.9,
)
optimizer = Adam(learning_rate=lr_schedule)
model.compile(optimizer=optimizer, loss="mse")

model.fit(
    x=x_train, y=y_train, batch_size=32, validation_data=(x_val, y_val), epochs=10
)
```

    Epoch 1/10
    313/313 [==============================] - 2s 5ms/step - loss: 9.6909 - val_loss: 6.3050
    Epoch 2/10
    313/313 [==============================] - 1s 4ms/step - loss: 4.1970 - val_loss: 2.0028
    Epoch 3/10
    313/313 [==============================] - 1s 4ms/step - loss: 1.7086 - val_loss: 1.0551
    Epoch 4/10
    313/313 [==============================] - 1s 4ms/step - loss: 0.9906 - val_loss: 0.5927
    Epoch 5/10
    313/313 [==============================] - 1s 4ms/step - loss: 0.6411 - val_loss: 0.1694
    Epoch 6/10
    313/313 [==============================] - 1s 4ms/step - loss: 0.6686 - val_loss: 1.7604
    Epoch 7/10
    313/313 [==============================] - 1s 4ms/step - loss: 0.6464 - val_loss: 0.1079
    Epoch 8/10
    313/313 [==============================] - 1s 4ms/step - loss: 0.4570 - val_loss: 0.1365
    Epoch 9/10
    313/313 [==============================] - 1s 4ms/step - loss: 0.2945 - val_loss: 0.0664
    Epoch 10/10
    313/313 [==============================] - 1s 4ms/step - loss: 0.2095 - val_loss: 0.0849

    <keras.callbacks.History>

## License

<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Creative Commons Licence" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br />This
work is licensed under a
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative
Commons Attribution-NonCommercial-ShareAlike 4.0 International
License</a>.

You are free to: - Share — copy and redistribute the material in any
medium or format

- Adapt — remix, transform, and build upon the material

The licensor cannot revoke these freedoms as long as you follow the
license terms.

Under the following terms: - Attribution — You must give appropriate
credit, provide a link to the license, and indicate if changes were
made. You may do so in any reasonable manner, but not in any way that
suggests the licensor endorses you or your use.

- NonCommercial — You may not use the material for commercial purposes.

- ShareAlike — If you remix, transform, or build upon the material, you
  must distribute your contributions under the same license as the
  original.

- No additional restrictions — You may not apply legal terms or
  technological measures that legally restrict others from doing
  anything the license permits.
