# RMSprop { #phasic.svgd.RMSprop }

```python
phasic.svgd.RMSprop(learning_rate=0.001, decay=0.99, epsilon=1e-08)
```

RMSprop optimizer for SVGD.

RMSprop adapts the learning rate for each parameter by dividing by an
exponentially decaying average of squared gradients. This helps with
non-stationary objectives and noisy gradients.

## Parameters {.doc-section .doc-section-parameters}

<code>[**learning_rate**]{.parameter-name} [:]{.parameter-annotation-sep} [[float](`float`) or [StepSizeSchedule](`phasic.svgd.StepSizeSchedule`)]{.parameter-annotation} [ = ]{.parameter-default-sep} [0.001]{.parameter-default}</code>

:   Base learning rate. Can be a schedule for learning rate decay.

<code>[**decay**]{.parameter-name} [:]{.parameter-annotation-sep} [[float](`float`) or [StepSizeSchedule](`phasic.svgd.StepSizeSchedule`)]{.parameter-annotation} [ = ]{.parameter-default-sep} [0.99]{.parameter-default}</code>

:   Decay rate for the moving average of squared gradients. Higher values give longer memory. Can be a schedule.

<code>[**epsilon**]{.parameter-name} [:]{.parameter-annotation-sep} [[float](`float`)]{.parameter-annotation} [ = ]{.parameter-default-sep} [1e-8]{.parameter-default}</code>

:   Small constant for numerical stability.

## Attributes {.doc-section .doc-section-attributes}

<code>[**v**]{.parameter-name} [:]{.parameter-annotation-sep} [[array](`array`) or None]{.parameter-annotation}</code>

:   Moving average of squared gradients, shape (n_particles, theta_dim)

## Examples {.doc-section .doc-section-examples}

```python
>>> from phasic import SVGD, RMSprop
>>>
>>> optimizer = RMSprop(learning_rate=0.001, decay=0.99)
>>> svgd = SVGD(
...     model=model,
...     observed_data=observations,
...     theta_dim=2,
...     optimizer=optimizer
... )
```

## References {.doc-section .doc-section-references}

Hinton, G. (2012). Lecture 6.5 - RMSprop. Coursera: Neural Networks for
Machine Learning.

## Notes {.doc-section .doc-section-notes}

Update rule: v = decay * v + (1 - decay) * gradient²; params += lr * gradient / (√v + ε)

## Methods

| Name | Description |
| --- | --- |
| [reset](#phasic.svgd.RMSprop.reset) | Reset optimizer state for given particle shape. |
| [step](#phasic.svgd.RMSprop.step) | Compute RMSprop update. |

### reset { #phasic.svgd.RMSprop.reset }

```python
phasic.svgd.RMSprop.reset(shape)
```

Reset optimizer state for given particle shape.

### step { #phasic.svgd.RMSprop.step }

```python
phasic.svgd.RMSprop.step(phi, particles=None)
```

Compute RMSprop update.

#### Parameters {.doc-section .doc-section-parameters}

<code>[**phi**]{.parameter-name} [:]{.parameter-annotation-sep} [[array](`array`)([n_particles](`n_particles`), [theta_dim](`theta_dim`))]{.parameter-annotation}</code>

:   SVGD gradient direction.

<code>[**particles**]{.parameter-name} [:]{.parameter-annotation-sep} [[array](`array`)([n_particles](`n_particles`), [theta_dim](`theta_dim`))]{.parameter-annotation} [ = ]{.parameter-default-sep} [None]{.parameter-default}</code>

:   Current particle positions. Not used by RMSprop.

#### Returns {.doc-section .doc-section-returns}

<code>[**update**]{.parameter-name} [:]{.parameter-annotation-sep} [[array](`array`)([n_particles](`n_particles`), [theta_dim](`theta_dim`))]{.parameter-annotation}</code>

:   Scaled update to add to particles.