snn.nn — Flat namespace

snn.nn collects everything into one flat import so you never have to remember which sub-module a class lives in.

from snn.nn import Dense, ReLU, Adam, CCE, Sequential

It also ships NNFS-style aliases — prefixed class names (Layer_Dense, Activation_ReLU, Loss_CCE, Optimizer_Adam …) so code from the Neural Networks from Scratch book ports without any renaming.

---

Quick start

from snn.nn import Dense, ReLU, Softmax, Adam, CCE, Sequential, to_categorical
import numpy as np

# Build a model using plain class names
model = Sequential([
    Dense(128),
    ReLU(),
    Dense(64),
    ReLU(),
    Dense(10),
    Softmax(),
])

model.compile(Adam(learning_rate=1e-3), CCE(), metrics=['accuracy'])
model.fit(X_train, y_train, epochs=30, batch_size=64)

---

NNFS-style manual loop

from snn.nn import Layer_Dense, Activation_ReLU, Activation_Softmax
from snn.nn import Loss_CCE, Optimizer_Adam
import numpy as np

dense1  = Layer_Dense(784, 128)
relu    = Activation_ReLU()
dense2  = Layer_Dense(128, 10)
softmax = Activation_Softmax()
loss_fn = Loss_CCE()
opt     = Optimizer_Adam(learning_rate=0.001)

for epoch in range(100):
    # Forward
    out = dense1.forward(X_train)
    out = relu.forward(out)
    out = dense2.forward(out)
    out = softmax.forward(out)
    loss = loss_fn.forward(out, y_train)

    # Backward
    grad = loss_fn.backward(out, y_train)
    grad = softmax.backward(grad)
    grad = dense2.backward(grad)
    grad = relu.backward(grad)
    dense1.backward(grad)

    # Update
    opt.update(dense1)
    opt.update(dense2)

---

API reference

snn.nn — Everything in one flat namespace.

Inspired by the “Neural Networks from Scratch” (NNFS) book style, this module collects every building block into a single importable namespace so you never have to remember which sub-module something lives in.

Quick start:

from snn.nn import Dense, ReLU, Softmax, Adam, CCE, Sequential

model = Sequential([
    Dense(128),
    ReLU(),       # ← same class you'd use standalone!
    Dense(64),
    ReLU(),
    Dense(10),
    Softmax(),
])
model.compile(Adam(learning_rate=1e-3), CCE())
model.fit(X_train, y_train, epochs=20)

NNFS-style aliases are also provided so code from the book ports directly:

from snn.nn import Layer_Dense, Activation_ReLU, Loss_CCE, Optimizer_Adam

dense   = Layer_Dense(128, 64)
relu    = Activation_ReLU()
loss_fn = Loss_CCE()
opt     = Optimizer_Adam(learning_rate=0.05)

# Manual training loop (NNFS style)
for epoch in range(100):
    out   = dense.forward(X)
    out   = relu.forward(out)
    loss  = loss_fn.forward(out, y)
    grad  = loss_fn.backward(out, y)
    grad  = relu.backward(grad)
    dense.backward(grad)
    opt.apply_gradients(dense.params, dense.grads)

class snn.nn.Activation

Bases: object

Base class for all activation functions.

forward(x)

backward(grad)

class snn.nn.Linear

Bases: Activation

Identity / no-op activation. f(x) = x.

forward(x)

backward(grad)

class snn.nn.ReLU

Bases: Activation

Rectified Linear Unit. f(x) = max(0, x).

forward(x)

backward(grad)

class snn.nn.LeakyReLU(alpha=0.01)

Bases: Activation

Leaky ReLU. f(x) = x if x > 0 else alpha*x.

Parameters:

alpha (float) – Slope for negative inputs (default 0.01).

forward(x)

backward(grad)

class snn.nn.ELU(alpha=1.0)

Bases: Activation

Exponential Linear Unit. f(x) = x if x > 0 else alpha*(exp(x)-1).

Parameters:

alpha (float) – Scale for negative saturation region (default 1.0).

forward(x)

backward(grad)

class snn.nn.SELU

Bases: Activation

Scaled ELU — self-normalising activation for deep networks.

forward(x)

backward(grad)

class snn.nn.Sigmoid

Bases: Activation

Logistic sigmoid. f(x) = 1 / (1 + exp(-x)).

forward(x)

backward(grad)

class snn.nn.Tanh

Bases: Activation

Hyperbolic tangent. f(x) = tanh(x).

forward(x)

backward(grad)

class snn.nn.Softmax

Bases: Activation

Softmax over the last axis. Typically used on the output layer for multi-class classification.

forward(x)

backward(grad)

class snn.nn.Softplus

Bases: Activation

Smooth approximation of ReLU. f(x) = log(1 + exp(x)).

forward(x)

backward(grad)

class snn.nn.Swish

Bases: Activation

Swish / SiLU. f(x) = x * sigmoid(x). Smooth, non-monotonic.

forward(x)

backward(grad)

class snn.nn.Mish

Bases: Activation

Mish. f(x) = x * tanh(softplus(x)). Self-regularising, smooth.

forward(x)

backward(grad)

class snn.nn.GELU

Bases: Activation

Gaussian Error Linear Unit.

Uses the tanh approximation: f(x) ≈ 0.5 * x * (1 + tanh(√(2/π) * (x + 0.044715 * x³)))

The standard choice in modern Transformers (BERT, GPT). Smooth and non-monotonic; outperforms ReLU/ELU on many regression and NLP tasks.

forward(x)

backward(grad)

class snn.nn.PReLU(alpha=0.25)

Bases: Activation

Parametric ReLU with a learnable negative slope.

f(x) = x if x ≥ 0 f(x) = alpha * x if x < 0

alpha is updated by the optimiser just like any weight — pass the layer that contains this activation through a Sequential model and alpha will be trained automatically.

Parameters:

alpha (float) – Initial slope for the negative region (default 0.25).

forward(x)

backward(grad)

property params

property grads

class snn.nn.Sine

Bases: Activation

Sine activation for SIREN (Sinusoidal Representation Networks).

f(x) = sin(x)

Designed for learning smooth, periodic, and polynomial-like functions. A network with sine activations can exactly represent derivatives of itself and naturally approximates functions like y = x², y = sin(x), or any smooth manifold.

Initialisation note: use kernel_initializer="siren_uniform" (or scale weights by 1/fan_in for hidden layers) for best convergence. As a quick start, glorot_uniform works for shallow networks.

forward(x)

backward(grad)

class snn.nn.Hardswish

Bases: Activation

Hard Swish activation (MobileNetV3).

f(x) = x * ReLU6(x + 3) / 6

Piecewise-smooth approximation of Swish. Efficient for deployment while retaining the non-monotonic character of Swish.

forward(x)

backward(grad)

class snn.nn.BentIdentity

Bases: Activation

Bent Identity activation.

f(x) = (√(x² + 1) − 1) / 2 + x

Smooth everywhere, non-saturating, and non-linear at every point. The derivative is always positive (≥ 1) so it avoids the dying-neuron problem while still introducing the curvature needed to fit polynomial or quadratic data. Good default for regression tasks.

forward(x)

backward(grad)

class snn.nn.Squareplus(b=4.0)

Bases: Activation

Squareplus — smooth, monotonic ReLU alternative.

f(x) = (x + √(x² + b)) / 2

Always positive and differentiable, with quadratic behaviour near the origin controlled by b. Approaches ReLU as b → 0.

Parameters:

b (float) – Curvature at origin (default 4.0). Larger values = smoother transition.

forward(x)

backward(grad)

class snn.nn.ReLU6

Bases: Activation

ReLU capped at 6. f(x) = min(max(0, x), 6).

Used in MobileNet and other efficient architectures. The cap at 6 improves fixed-point quantisation precision.

forward(x)

backward(grad)

class snn.nn.Hardsigmoid

Bases: Activation

Piecewise-linear sigmoid approximation.

f(x) = clip((x + 3) / 6, 0, 1)

Computationally cheaper than the exact sigmoid; commonly used in quantisation-friendly networks (MobileNetV3).

forward(x)

backward(grad)

class snn.nn.LogSoftmax

Bases: Activation

Log-softmax. f(x) = x − log(Σ exp(x)).

Numerically stable; used together with negative log-likelihood loss (equivalent to CrossEntropy = NLLLoss(LogSoftmax(x))).

forward(x)

backward(grad)

class snn.nn.Sparsemax

Bases: Activation

Sparsemax — differentiable sparse softmax (Martins & Astudillo 2016).

Like softmax but returns a sparse probability distribution: many outputs are exactly 0. Useful for attention mechanisms where hard, interpretable focus is desired.

f(x) = max(0, x − τ(x)) where τ is the threshold that makes outputs sum to 1.

forward(x)

backward(grad)

class snn.nn.CELU(alpha=1.0)

Bases: Activation

Continuously Differentiable ELU (Barron 2017).

f(x) = x if x >= 0 else alpha*(exp(x/alpha) - 1)

Unlike ELU, CELU is continuously differentiable at x=0 for any alpha.

Parameters:

alpha (float) – Scale for the negative branch (default 1.0).

forward(x)

backward(grad)

class snn.nn.Softsign

Bases: Activation

Softsign activation. f(x) = x / (1 + |x|).

A smooth, bounded alternative to Tanh with heavier tails. Saturates more slowly, which can help with gradient flow.

forward(x)

backward(grad)

class snn.nn.Tanhshrink

Bases: Activation

Tanhshrink activation. f(x) = x - tanh(x).

A soft-thresholding function that shrinks values toward zero. Used in sparse autoencoders and shrinkage estimators.

forward(x)

backward(grad)

snn.nn.get_activation(identifier)

Return an Activation instance from a string or object.

Parameters:

identifier (str, Activation, or None) – None or "linear" → Linear.

Raises:

ValueError – If the string key is not recognised.

class snn.nn.Layer(trainable=True, name=None)

Bases: object

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

get_config()

count_params()

set_param(key, val)

Set a single parameter by name.

Override in composite layers (e.g. TransformerBlock) to route prefixed keys to the correct sub-layer. The default implementation calls setattr(self, key, val) when the attribute exists.

class snn.nn.Dense(units, activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', name=None)

Bases: Layer

Fully-connected (dense) layer.

Parameters:

• units (int) – Number of output neurons.

• activation (str or Activation, optional) – Activation function applied after the linear transform.

• use_bias (bool) – Whether to include a bias vector.

• kernel_initializer (str or callable) – Initializer for the weight matrix W.

• bias_initializer (str or callable) – Initializer for the bias vector b.

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

get_config()

class snn.nn.Conv1D(filters, kernel_size, stride=1, padding='valid', activation=None, use_bias=True, kernel_initializer='glorot_uniform', name=None)

Bases: Layer

1-D Convolution layer (channels-last: batch, length, channels).

Slides a bank of filters kernels of width kernel_size along the time/sequence axis. Useful for local pattern detection in sequences (text, audio, time-series).

Parameters:

• filters (int) – Number of output channels (kernels).

• kernel_size (int) – Width of each convolutional kernel.

• stride (int) – Step between successive windows (default 1).

• padding (str) – "valid" — no padding, output shorter than input. "same" — zero-pad so output length equals input length (only when stride=1).

• activation (str or Activation) – Applied after the convolution.

• use_bias (bool) – Whether to include a bias term.

• kernel_initializer (str) – Weight initialiser (default "glorot_uniform").

• shape (Input / output)

• --------------------

• ``(batch (→) –

• length

• in_channels)``

• ``(batch –

• out_length

• filters)``

• 1. (where out_length = (length + 2*pad - kernel_size) // stride +)

Examples

>>> conv = Conv1D(filters=32, kernel_size=3, activation='relu')
>>> x = np.random.randn(4, 50, 8)   # (batch, seq, channels)
>>> out = conv.forward(x)           # (4, 48, 32) with valid padding

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

get_config()

class snn.nn.Conv2D(filters, kernel_size=3, stride=1, padding='valid', activation=None, use_bias=True, kernel_initializer='he_normal', bias_initializer='zeros', name=None)

Bases: Layer

2D Convolutional layer.

Expects input shape: (N, H, W, C_in) — channels-last convention.

Parameters:

• filters (int) – Number of output filters (channels).

• kernel_size (int or (int, int)) – Size of the convolution kernel.

• stride (int) – Stride of the convolution.

• padding (str or int) – ‘same’, ‘valid’, or an explicit integer pad.

• activation (str or Activation) – Applied after convolution + bias.

• use_bias (bool)

• kernel_initializer (str or callable)

• bias_initializer (str or callable)

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

get_config()

class snn.nn.MaxPooling1D(pool_size=2, stride=None, name=None)

Bases: Layer

1-D Max Pooling over the time/sequence axis.

Parameters:

• pool_size (int) – Size of the pooling window (default 2).

• stride (int) – Step between windows (default equals pool_size).

• shape (Input / output)

• --------------------

• ``(batch (channels)`` →) –

• length

• ``(batch –

• out_length

• channels)``

forward(x, training=False)

backward(grad)

class snn.nn.AveragePooling1D(pool_size=2, stride=None, name=None)

Bases: Layer

1-D Average Pooling over the time/sequence axis.

Parameters:

• pool_size (int) – Width of each pooling window (default 2).

• stride (int) – Step between windows (default equals pool_size).

forward(x, training=False)

backward(grad)

class snn.nn.MaxPooling2D(pool_size=2, stride=None, name=None)

Bases: Layer

2D Max Pooling layer (channels-last: N, H, W, C).

forward(x, training=False)

backward(grad)

class snn.nn.AveragePooling2D(pool_size=2, stride=None, name=None)

Bases: Layer

2D Average Pooling layer (channels-last: N, H, W, C).

forward(x, training=False)

backward(grad)

class snn.nn.GlobalAveragePooling1D(trainable=True, name=None)

Bases: Layer

Global Average Pooling over the time/sequence axis.

Reduces (batch, seq_len, features) to (batch, features) by averaging over seq_len. Standard way to collapse a Transformer or LSTM output into a fixed-size representation for classification.

Examples

>>> gap = GlobalAveragePooling1D()
>>> x = np.random.randn(8, 20, 64)   # (batch, seq, features)
>>> out = gap.forward(x)             # (8, 64)

forward(x, training=False)

backward(grad)

class snn.nn.GlobalMaxPooling1D(trainable=True, name=None)

Bases: Layer

Global Max Pooling over the time/sequence axis.

Reduces (batch, seq_len, features) to (batch, features) by taking the maximum over seq_len.

Examples

>>> gmp = GlobalMaxPooling1D()
>>> x = np.random.randn(8, 20, 64)
>>> out = gmp.forward(x)    # (8, 64)

forward(x, training=False)

backward(grad)

class snn.nn.GlobalAveragePooling2D(trainable=True, name=None)

Bases: Layer

Global Average Pooling — reduces (N, H, W, C) to (N, C).

forward(x, training=False)

backward(grad)

class snn.nn.GlobalMaxPooling2D(trainable=True, name=None)

Bases: Layer

Global Max Pooling — reduces (N, H, W, C) to (N, C).

forward(x, training=False)

backward(grad)

class snn.nn.Dropout(rate=0.5, seed=None, name=None)

Bases: Layer

Dropout regularization layer.

During training, randomly sets a fraction rate of inputs to zero and scales the remaining values by 1 / (1 - rate) (inverted dropout). During inference, the layer is a pass-through.

Parameters:

• rate (float) – Fraction of units to drop (0 <= rate < 1).

• seed (int, optional) – Random seed for reproducibility.

forward(x, training=False)

backward(grad)

get_config()

class snn.nn.SpatialDropout2D(rate=0.5, seed=None, name=None)

Bases: Layer

Spatial Dropout for 2D feature maps (N, H, W, C).

Drops entire feature maps (channels) instead of individual elements.

Parameters:

• rate (float) – Fraction of feature maps to drop.

• seed (int, optional)

forward(x, training=False)

backward(grad)

get_config()

class snn.nn.BatchNormalization(axis=-1, momentum=0.99, epsilon=1e-05, center=True, scale=True, name=None)

Bases: Layer

Batch Normalization layer.

Normalizes activations over the batch dimension. During training uses batch statistics; during inference uses running (exponential moving average) statistics.

Parameters:

• axis (int) – Axis to normalize (usually the features/channels axis). For Dense output (N, F) use axis=-1. For Conv2D output (N, H, W, C) use axis=-1.

• momentum (float) – Momentum for the moving average (typically 0.99).

• epsilon (float) – Small constant for numerical stability.

• center (bool) – If True, add learnable beta offset.

• scale (bool) – If True, multiply by learnable gamma scale.

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

get_config()

class snn.nn.LayerNormalization(epsilon=1e-06, center=True, scale=True, name=None)

Bases: Layer

Layer Normalization — normalizes over the last axis (feature axis).

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

class snn.nn.Flatten(trainable=True, name=None)

Bases: Layer

Flattens input to (N, -1). Keeps the batch dimension.

forward(x, training=False)

backward(grad)

class snn.nn.Reshape(target_shape, name=None)

Bases: Layer

Reshapes input (excluding batch dim) to a target shape.

Parameters:

target_shape (tuple) – New shape for each sample (not including batch size).

forward(x, training=False)

backward(grad)

get_config()

class snn.nn.SimpleRNN(units, activation='tanh', return_sequences=False, return_state=False, kernel_initializer='glorot_uniform', recurrent_initializer='orthogonal', bias_initializer='zeros', name=None)

Bases: Layer

Vanilla (Elman) Recurrent Neural Network cell.

Parameters:

• units (int) – Number of hidden units.

• activation (str or Activation) – Activation applied to the hidden state (default: ‘tanh’).

• return_sequences (bool) – If True, return output at every time step; else only the last.

• return_state (bool) – If True, return (output, last_hidden_state) tuple.

• kernel_initializer (str or callable)

• recurrent_initializer (str or callable)

• bias_initializer (str or callable)

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

class snn.nn.LSTM(units, return_sequences=False, return_state=False, kernel_initializer='glorot_uniform', recurrent_initializer='orthogonal', bias_initializer='zeros', name=None)

Bases: Layer

Long Short-Term Memory (LSTM) layer.

Implements the standard LSTM equations:

f_t = sigmoid(x_t @ Wf + h_{t-1} @ Uf + bf) i_t = sigmoid(x_t @ Wi + h_{t-1} @ Ui + bi) g_t = tanh(x_t @ Wg + h_{t-1} @ Ug + bg) o_t = sigmoid(x_t @ Wo + h_{t-1} @ Uo + bo) c_t = f_t * c_{t-1} + i_t * g_t h_t = o_t * tanh(c_t)

Parameters:

• units (int)

• return_sequences (bool)

• return_state (bool)

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

class snn.nn.GRU(units, return_sequences=False, return_state=False, kernel_initializer='glorot_uniform', bias_initializer='zeros', name=None)

Bases: Layer

Gated Recurrent Unit (GRU) layer.

Equations:

z_t = sigmoid(x_t @ Wz + h_{t-1} @ Uz + bz) # update gate r_t = sigmoid(x_t @ Wr + h_{t-1} @ Ur + br) # reset gate n_t = tanh(x_t @ Wn + (r_t * h_{t-1}) @ Un + bn) # new gate h_t = (1 - z_t) * n_t + z_t * h_{t-1}

Parameters:

• units (int)

• return_sequences (bool)

• return_state (bool)

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

class snn.nn.Embedding(vocab_size, embed_dim, embeddings_initializer='random_normal', name=None)

Bases: Layer

Token embedding layer.

Maps integer token indices to dense float vectors. This is the standard first layer for any NLP or sequence model.

Parameters:

• vocab_size (int) – Number of unique tokens (size of the vocabulary).

• embed_dim (int) – Dimensionality of the embedding vectors.

• embeddings_initializer (str) – Initialiser for the embedding matrix (default "random_normal").

• shape (Output)

• -----------

• ``(batch –

• ``[0 (seq_len)`` — integer token indices in) –

• vocab_size)``.

• shape

• ------------

• ``(batch –

• seq_len

• vectors. (embed_dim)`` — dense embedding)

Examples

>>> emb = Embedding(vocab_size=1000, embed_dim=64)
>>> x = np.array([[1, 5, 23, 0], [4, 2, 9, 7]])   # (2, 4) integer tokens
>>> out = emb.forward(x)   # (2, 4, 64)

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

count_params()

get_config()

class snn.nn.PositionalEncoding(max_seq_len=512, name=None)

Bases: Layer

Sinusoidal Positional Encoding (Vaswani et al. 2017).

Adds a fixed, non-learned position signal to a sequence embedding so that the model can distinguish token order. The encoding is defined by:

PE[pos, 2i]   = sin(pos / 10000^(2i/d))
PE[pos, 2i+1] = cos(pos / 10000^(2i/d))

This layer has no trainable parameters — it just adds the PE matrix to the incoming embedding and passes the gradient straight through.

Parameters:

• max_seq_len (int) – Maximum sequence length to pre-compute (default 512).

• shape (Input / output)

• --------------------

• ``(batch –

• seq_len

• in-place). (embed_dim)`` → same shape (PE added)

Examples

>>> pe = PositionalEncoding(max_seq_len=128)
>>> x = np.random.randn(4, 20, 64)    # (batch, seq, embed_dim)
>>> out = pe.forward(x)               # (4, 20, 64) — PE added

forward(x, training=False)

backward(grad)

class snn.nn.MultiHeadAttention(embed_dim, num_heads, dropout_rate=0.0, name=None)

Bases: Layer

Multi-Head Self-Attention (Vaswani et al. 2017).

Computes scaled dot-product attention in parallel over num_heads heads, then projects the concatenated result back to embed_dim.

Attention(Q, K, V) = softmax(Q·Kᵀ / √d_k) · V

Each head operates on a head_dim = embed_dim // num_heads subspace.

Parameters:

• embed_dim (int) – Total embedding dimensionality (must be divisible by num_heads).

• num_heads (int) – Number of parallel attention heads.

• dropout_rate (float) – Dropout applied to the attention weight matrix during training.

• shape (Output)

• -----------

• ``(batch –

• seq_len

• embed_dim)``

• shape

• ------------

• ``(batch –

• seq_len

• embed_dim)``

Examples

>>> attn = MultiHeadAttention(embed_dim=64, num_heads=4)
>>> x = np.random.randn(2, 10, 64)
>>> out = attn.forward(x)   # (2, 10, 64)

build(input_shape)

forward(x, training=False, mask=None)

backward(grad)

property params

property grads

class snn.nn.TransformerBlock(embed_dim, num_heads, ff_dim, dropout_rate=0.1, activation='gelu', name=None)

Bases: Layer

Transformer encoder block (Pre-LayerNorm variant).

Architecture per block:

x → LayerNorm → MultiHeadAttention → Dropout → + x (residual)
  → LayerNorm → FFN(ff_dim) → Dropout → + x (residual)

The FFN consists of two Dense layers: Dense(ff_dim, activation) → Dense(embed_dim).

Pre-LN (applied before each sub-layer) is more numerically stable than Post-LN and typically requires no learning-rate warmup.

Parameters:

• embed_dim (int) – Model dimensionality.

• num_heads (int) – Number of attention heads.

• ff_dim (int) – Hidden size of the feed-forward network (usually 2–4 × embed_dim).

• dropout_rate (float) – Dropout applied inside attention and after each sub-layer.

• activation (str) – Activation used in the FFN hidden layer (default "gelu").

• shape (Input / output)

• --------------------

• ``(batch –

• seq_len

• shape. (embed_dim)`` → same)

Examples

Simple language classifier:

model = Sequential([
    Embedding(vocab_size=5000, embed_dim=64),
    PositionalEncoding(),
    TransformerBlock(embed_dim=64, num_heads=4, ff_dim=128),
    GlobalAveragePooling1D(),
    Dense(2, activation="softmax"),
])

forward(x, training=False)

backward(grad)

property params

property grads

set_param(key, val)

Route a prefixed parameter update to the correct sub-layer.

get_config()

class snn.nn.GLU(units, use_bias=True, kernel_initializer='glorot_uniform', name=None)

Bases: Layer

Gated Linear Unit (Dauphin et al. 2017).

Projects the input to 2 × units, splits into two halves (a, b), and returns a ⊙ σ(b):

z = x @ W + bias          # shape (..., 2*units)
a, b = z[..., :units], z[..., units:]
output = a * sigmoid(b)   # shape (..., units)

This halves the effective hidden size while adding a learnable gating mechanism. Widely used in CNNs and early Transformer variants.

Parameters:

• units (int) – Output dimensionality (half of the internal projection width).

• use_bias (bool) – Whether to add a bias term (default True).

• kernel_initializer (str) – Weight initialiser (default "glorot_uniform").

Examples

>>> glu = GLU(units=32)
>>> x = np.random.randn(4, 20, 64)
>>> out = glu.forward(x)   # (4, 20, 32)

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

get_config()

class snn.nn.SwiGLU(units, use_bias=True, kernel_initializer='glorot_uniform', name=None)

Bases: Layer

SwiGLU — Swish-Gated Linear Unit (Shazeer 2020).

Like GLU but replaces the sigmoid gate with Swish (SiLU):

z = x @ W + bias
a, b = z[..., :units], z[..., units:]
output = a ⊙ swish(b)    where swish(b) = b * σ(b)

SwiGLU is the standard FFN activation in LLaMA, PaLM, Mistral, and Gemma. It outperforms ReLU and GELU on most language tasks despite having 50% more parameters than a plain Dense layer (offset by removing one bias in the original papers).

Parameters:

• units (int) – Output dimensionality.

• use_bias (bool) – Whether to include a bias (default True).

Examples

Replace a standard FFN sub-layer:

# Standard
Dense(ff_dim, activation="gelu") → Dense(embed_dim)

# SwiGLU drop-in (ff_dim must be 2/3 of original for equal params)
SwiGLU(units=ff_dim) → Dense(embed_dim)

build(input_shape)

forward(x, training=False)

backward(grad)

property params

property grads

get_config()

class snn.nn.Add(trainable=True, name=None)

Bases: Layer

Element-wise addition of multiple inputs.

Used in residual / skip-connection architectures. All inputs must have the same shape.

In Sequential context, wrap it in a Residual block. In GraphModel context, call it with a list of Tensor nodes:

h1 = Dense(64, 'relu')(x)
h2 = Dense(64, 'relu')(x)
merged = Add()([h1, h2])

Input / output shape

List of arrays, each (batch, …) → same shape.

forward(inputs, training=False)

backward(grad)

class snn.nn.Concatenate(axis=-1, name=None)

Bases: Layer

Concatenate multiple inputs along one axis.

# GraphModel usage
h1 = Dense(32, 'relu')(x)
h2 = Dense(32, 'relu')(x)
merged = Concatenate(axis=-1)([h1, h2])   # shape (..., 64)

Parameters:

axis (int) – Axis along which to concatenate (default -1).

forward(inputs, training=False)

backward(grad)

class snn.nn.Residual(layers, activation='relu', shortcut=None, name=None)

Bases: Layer

Residual (skip-connection) block — for use inside Sequential.

Wraps a sequence of sub-layers and adds a skip connection:

output = activation(sub_layers(x) + shortcut(x))

If the input and output feature sizes differ, a learnable linear projection is built lazily and used as the shortcut.

Parameters:

• layers (list of Layer) – The main (non-shortcut) path.

• activation (str) – Activation applied after the residual sum (default "relu"). Pass None or "linear" to skip the activation.

• shortcut (Layer, optional) – Explicit shortcut layer. Auto-built if input/output dims differ.

Examples

Basic residual block inside Sequential:

model = Sequential([
    Dense(64, activation='relu'),
    Residual([
        Dense(64, activation='relu'),
        Dense(64),              # no activation — applied after sum
    ]),
    Dense(10, activation='softmax'),
])

Two blocks of different sizes (auto-projection):

model = Sequential([
    Dense(128, activation='relu'),
    Residual([Dense(64, activation='relu'), Dense(64)]),
    # shortcut Dense(64) auto-added because 128 ≠ 64
    Dense(10, activation='softmax'),
])

forward(x, training=False)

backward(grad)

property params

property grads

set_param(key, val)

Set a single parameter by name.

Override in composite layers (e.g. TransformerBlock) to route prefixed keys to the correct sub-layer. The default implementation calls setattr(self, key, val) when the attribute exists.

count_params()

class snn.nn.TimeDistributed(layer, name=None)

Bases: Layer

Apply a layer independently to each time step of a sequence.

Reshapes (batch, time, features) → (batch*time, features), runs the wrapped layer, then reshapes back. This lets you apply any layer (Dense, Conv1D, …) to every position in a sequence without writing a Python loop.

Parameters:

• layer (Layer) – The layer to apply at each time step.

• shape (Input / output)

• --------------------

• ``(batch (in_features)`` →) –

• time

• ``(batch –

• time

• out_features)``

Examples

>>> td = TimeDistributed(Dense(32, activation='relu'))
>>> x = np.random.randn(8, 20, 64)
>>> out = td.forward(x)    # (8, 20, 32)

Typical use — dense classification head over RNN output:

model = Sequential([
    LSTM(64, return_sequences=True),         # (batch, seq, 64)
    TimeDistributed(Dense(32, 'relu')),      # (batch, seq, 32)
    TimeDistributed(Dense(vocab_size, 'softmax')),
])

forward(x, training=False)

backward(grad)

property params

property grads

set_param(key, val)

Set a single parameter by name.

Override in composite layers (e.g. TransformerBlock) to route prefixed keys to the correct sub-layer. The default implementation calls setattr(self, key, val) when the attribute exists.

count_params()

class snn.nn.Loss

Bases: object

forward(y_pred, y_true)

backward(y_pred, y_true)

class snn.nn.MeanSquaredError

Bases: Loss

forward(y_pred, y_true)

backward(y_pred, y_true)

class snn.nn.MeanAbsoluteError

Bases: Loss

forward(y_pred, y_true)

backward(y_pred, y_true)

class snn.nn.HuberLoss(delta=1.0)

Bases: Loss

forward(y_pred, y_true)

backward(y_pred, y_true)

class snn.nn.BinaryCrossentropy(from_logits=False, eps=1e-07)

Bases: Loss

forward(y_pred, y_true)

backward(y_pred, y_true)

class snn.nn.CategoricalCrossentropy(from_logits=False, eps=1e-07)

Bases: Loss

forward(y_pred, y_true)

backward(y_pred, y_true)

class snn.nn.SparseCategoricalCrossentropy(from_logits=False, eps=1e-07)

Bases: Loss

forward(y_pred, y_true)

backward(y_pred, y_true)

class snn.nn.KLDivergence(eps=1e-07)

Bases: Loss

forward(y_pred, y_true)

backward(y_pred, y_true)

snn.nn.MSE

alias of MeanSquaredError

snn.nn.MAE

alias of MeanAbsoluteError

snn.nn.Huber

alias of HuberLoss

snn.nn.BCE

alias of BinaryCrossentropy

snn.nn.CCE

alias of CategoricalCrossentropy

snn.nn.SparseCCE

alias of SparseCategoricalCrossentropy

snn.nn.KLD

alias of KLDivergence

snn.nn.get_loss(identifier)

Return a Loss instance from a string, instance, or dict.

Parameters:

identifier (str, Loss, or dict) –

• String key — "mse", "categorical_crossentropy", …

• Loss instance — returned unchanged.

• Config dict — {"name": "huber", "delta": 2.0}. Any key accepted by the loss constructor may appear; "name" (or "class") selects the class.

Raises:

ValueError – Unknown string or dict name.

class snn.nn.Optimizer(learning_rate=0.01)

Bases: object

apply_gradients(params, grads)

get_config()

class snn.nn.SGD(learning_rate=0.01, momentum=0.0, nesterov=False, weight_decay=0.0)

Bases: Optimizer

Stochastic Gradient Descent with optional momentum and Nesterov acceleration.

apply_gradients(params, grads)

get_config()

class snn.nn.Adam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, weight_decay=0.0, amsgrad=False)

Bases: Optimizer

Adam optimizer (Adaptive Moment Estimation).

apply_gradients(params, grads)

get_config()

class snn.nn.AdamW(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, weight_decay=0.01)

Bases: Adam

AdamW — Adam with decoupled weight decay regularization.

apply_gradients(params, grads)

class snn.nn.RMSprop(learning_rate=0.001, rho=0.9, epsilon=1e-08, momentum=0.0, weight_decay=0.0, centered=False)

Bases: Optimizer

RMSprop optimizer.

apply_gradients(params, grads)

get_config()

class snn.nn.Adagrad(learning_rate=0.01, epsilon=1e-08, initial_accumulator_value=0.1, weight_decay=0.0)

Bases: Optimizer

Adagrad optimizer — adapts learning rates using accumulated squared gradients.

apply_gradients(params, grads)

get_config()

class snn.nn.Adadelta(learning_rate=1.0, rho=0.95, epsilon=1e-07, weight_decay=0.0)

Bases: Optimizer

Adadelta optimizer — adapts learning rates without a global learning rate.

apply_gradients(params, grads)

get_config()

class snn.nn.Nadam(learning_rate=0.002, beta_1=0.9, beta_2=0.999, epsilon=1e-08, weight_decay=0.0)

Bases: Optimizer

Nadam — Nesterov-accelerated Adaptive Moment Estimation.

Combines Adam’s adaptive learning rates with Nesterov momentum, giving faster convergence than standard Adam in many settings.

f(θ) update rule:

m_t  = β₁ · m_{t-1} + (1 − β₁) · g_t
v_t  = β₂ · v_{t-1} + (1 − β₂) · g_t²
m̂    = β₁ · m_t / (1−β₁^{t+1}) + (1−β₁) · g_t / (1−β₁^t)
θ_t  = θ_{t-1} − lr · m̂ / (√v̂ + ε)

Parameters:

• learning_rate (float) – Step size (default 2e-3).

• beta_1 (float) – Exponential decay for first moment (default 0.9).

• beta_2 (float) – Exponential decay for second moment (default 0.999).

• epsilon (float) – Numerical stability constant (default 1e-8).

• weight_decay (float) – L2 regularisation coefficient (default 0.0).

apply_gradients(params, grads)

get_config()

class snn.nn.RAdam(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, weight_decay=0.0)

Bases: Optimizer

RAdam — Rectified Adam.

Fixes the variance issue in Adam’s early training phase by computing the effective sample size of the second moment estimate. When the approximation is not reliable (small t), it falls back to an SGD-with- momentum step; otherwise it applies the full variance-corrected Adam update. This removes the need for a warmup schedule.

Reference: Liu et al. (2019) “On the Variance of the Adaptive Learning Rate and Beyond”. https://arxiv.org/abs/1908.03265

Parameters:

• learning_rate (float) – Step size (default 1e-3).

• beta_1 (float) – First-moment decay (default 0.9).

• beta_2 (float) – Second-moment decay (default 0.999).

• epsilon (float) – Numerical stability constant (default 1e-8).

• weight_decay (float) – L2 regularisation coefficient (default 0.0).

apply_gradients(params, grads)

get_config()

class snn.nn.Lion(learning_rate=0.0001, beta_1=0.9, beta_2=0.99, weight_decay=0.0)

Bases: Optimizer

Lion — EvoLved Sign Momentum.

Uses only the sign of the update vector, making every parameter receive an update of exactly ±learning_rate per step. This gives Lion 3–10× lower memory usage than Adam (one momentum buffer instead of two) and competitive or better performance on image and language tasks.

Update rule:

c_t  = β₁ · m_{t-1} + (1 − β₁) · g_t
θ_t  = θ_{t-1} − lr · (sign(c_t) + λ · θ_{t-1})
m_t  = β₂ · m_{t-1} + (1 − β₂) · g_t

Note: Lion typically needs a 3–10× smaller learning rate and 3–10× larger weight decay than Adam. A good starting point is lr=1e-4, weight_decay=1e-2.

Reference: Chen et al. (2023) “Symbolic Discovery of Optimization Algorithms”. https://arxiv.org/abs/2302.06675

Parameters:

• learning_rate (float) – Step size (default 1e-4).

• beta_1 (float) – Coefficient for computing the update direction (default 0.9).

• beta_2 (float) – Coefficient for maintaining the momentum buffer (default 0.99).

• weight_decay (float) – Decoupled weight decay (default 0.0).

apply_gradients(params, grads)

get_config()

class snn.nn.LAMB(learning_rate=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-06, weight_decay=0.0, clip_ratio=10.0)

Bases: Optimizer

LAMB — Layer-wise Adaptive Moments optimizer (Ginsburg et al. 2019).

Extends Adam with a layer-wise trust ratio that scales the update by ‖θ‖ / ‖adam_update‖. This allows training with very large batch sizes (e.g. 65 536) without learning-rate tuning, and is the standard optimizer for BERT-style pretraining.

Update rule:

m_t = β₁·m_{t-1} + (1−β₁)·g_t
v_t = β₂·v_{t-1} + (1−β₂)·g_t²
m̂   = m_t/(1−β₁^t)   v̂ = v_t/(1−β₂^t)
u   = m̂/(√v̂ + ε) + λ·θ
r   = clip(‖θ‖/‖u‖, 0, clip_ratio)
θ   = θ − lr·r·u

Note

LAMB is designed for large batch training. For small batches (< 512) Adam or AdamW will generally perform equally well.

Parameters:

• learning_rate (float) – Base step size (default 1e-3).

• beta_1 (float) – Moment decay rates (default 0.9, 0.999).

• beta_2 (float) – Moment decay rates (default 0.9, 0.999).

• epsilon (float) – Numerical stability (default 1e-6).

• weight_decay (float) – L2 regularisation coefficient λ (default 0.0).

• clip_ratio (float) – Upper bound for the trust ratio (default 10.0).

apply_gradients(params, grads)

get_config()

class snn.nn.Lookahead(optimizer, k=5, alpha=0.5)

Bases: Optimizer

Lookahead optimizer wrapper (Zhang et al. 2019).

Wraps any base optimizer and adds a slow-weights outer loop:

• Inner loop — the base optimizer updates “fast weights” for k steps as usual.

• Outer update — after every k inner steps, the slow weights interpolate toward the fast weights:

  θ_slow ← θ_slow + α · (θ_fast − θ_slow)
  θ_fast ← θ_slow
  

This stabilises training across a wide range of learning rates and often improves generalisation with minimal overhead.

Parameters:

• optimizer (Optimizer) – Any snn optimizer instance (Adam, SGD, Nadam, …).

• k (int) – Number of inner (fast) steps before each slow update (default 5).

• alpha (float) – Slow-weights interpolation coefficient (default 0.5).

Examples

>>> from snn.optimizers import Adam, Lookahead
>>> opt = Lookahead(Adam(learning_rate=1e-3), k=5, alpha=0.5)
>>> model.compile(opt, "categorical_crossentropy")

property learning_rate

property weight_decay

apply_gradients(params, grads)

get_config()

class snn.nn.Adan(learning_rate=0.001, beta_1=0.98, beta_2=0.92, beta_3=0.99, epsilon=1e-08, weight_decay=0.02)

Bases: Optimizer

Adan — Adaptive Nesterov Momentum Algorithm (Xie et al. 2022).

Uses three exponential moving averages — of the gradient, the gradient difference, and a combined Nesterov-like term — to get fast convergence on non-convex problems. Achieves competitive or better results than Adam on image classification and NLP benchmarks.

Update rule:

dk  = gk − g_{k-1}           (gradient difference)
m1  = β₁·m1 + (1−β₁)·gk
m2  = β₂·m2 + (1−β₂)·dk
m3  = β₃·m3 + (1−β₃)·(gk + (1−β₂)·dk)²
η   = lr / (√m3 + ε)
θ   = (1 + λ·lr)⁻¹ · (θ − η · (m1 + (1−β₂)·m2))

Parameters:

• learning_rate (float) – Step size (default 1e-3).

• beta_1 (float) – Decay for first moment (default 0.98).

• beta_2 (float) – Decay for gradient difference (default 0.92).

• beta_3 (float) – Decay for second-order moment (default 0.99).

• epsilon (float) – Numerical stability (default 1e-8).

• weight_decay (float) – Decoupled weight decay λ (default 0.02).

apply_gradients(params, grads)

get_config()

snn.nn.get_optimizer(identifier)

Return an Optimizer instance from a string, instance, or dict.

Parameters:

identifier (str, Optimizer, or dict) –

• String key — "adam", "sgd", …

• Optimizer instance — returned unchanged.

• Config dict — {"name": "adam", "learning_rate": 1e-3}. Any key accepted by the optimizer constructor may appear; "name" (or "class") selects the class.

Raises:

ValueError – Unknown string or dict name.

class snn.nn.Sequential(layers=None)

Bases: object

Sequential model — a linear stack of layers.

Build, compile, then train:

>>> model = Sequential([
...     Dense(64, activation='relu'),
...     Dropout(0.3),
...     Dense(10, activation='softmax'),
... ])
>>> model.compile(
...     optimizer='adam',
...     loss='categorical_crossentropy',
...     metrics=['accuracy'],
... )
>>> history = model.fit(X_train, y_train, epochs=20, batch_size=32,
...                     validation_data=(X_val, y_val))
>>> y_pred = model.predict(X_test)
>>> results = model.evaluate(X_test, y_test)

add(layer)

Append a layer to the model.

Parameters:

layer (Layer) – Any snn.layers.base.Layer subclass.

Returns:

Allows chaining: model.add(Dense(64)).add(Dense(10)).

Return type:

self

compile(optimizer='adam', loss='mse', metrics=None, *, learning_rate=None, weight_decay=None, clip_norm=None, from_logits=False, verbose=False)

Configure the model for training.

Parameters:

• optimizer (str, Optimizer, or dict) –

  Weight-update rule. Three forms accepted:

  • String shorthand — "adam", "sgd", "adamw", …

  • Instance — Adam(learning_rate=3e-4)

  • Config dict — {"name": "adam", "learning_rate": 1e-3} Any key recognised by the optimizer constructor can appear in the dict; the "name" (or "class") key selects the optimizer class.

• loss (str, Loss, or dict) –

  Objective function. Three forms accepted:

  • String shorthand — "mse", "categorical_crossentropy", …

  • Instance — HuberLoss(delta=2.0)

  • Config dict — {"name": "huber", "delta": 2.0}

• metrics (str or list, optional) – Metric(s) tracked per epoch. Accepts a single string or a list of strings / callables.

• learning_rate (float, optional) – Shortcut to override the optimizer’s learning rate without constructing an instance manually. Applied after the optimizer is built from optimizer.

• weight_decay (float, optional) – Shortcut to set weight_decay on the optimizer (if supported).

• clip_norm (float, optional) – Clip the global gradient L2 norm to this value before each optimizer step. Equivalent to using Trainer(model, clip_grad_norm=clip_norm) but available directly on the model.

• from_logits (bool) – If True, the loss receives raw (unnormalised) scores. Passed automatically to BinaryCrossentropy and CategoricalCrossentropy when constructed from a string or dict. Has no effect when you pass a Loss instance directly.

• verbose (bool) – If True, prints a compile-summary table showing the chosen optimizer, loss, metrics, and any extra settings.

Return type:

self

Examples

Minimal:

>>> model.compile("adam", "mse")

Learning-rate shortcut (no need to import Adam):

>>> model.compile("adam", "categorical_crossentropy",
...               metrics=["accuracy"], learning_rate=3e-4)

Dict config — all hyperparameters in one place:

>>> model.compile(
...     optimizer={"name": "adamw", "learning_rate": 1e-3,
...                "weight_decay": 1e-4},
...     loss={"name": "huber", "delta": 2.0},
...     metrics=["r2"],
... )

With gradient clipping (no Trainer needed):

>>> model.compile("adam", "mse", clip_norm=1.0)

Logit input to loss (last Dense has no activation):

>>> model.compile("adam", "categorical_crossentropy",
...               from_logits=True)

get_compile_config()

Return a plain-dict snapshot of the current compile settings.

Useful for logging, serialising experiments, or re-compiling a new model with the same settings.

Returns:

Keys: "optimizer", "loss", "metrics", "learning_rate", "weight_decay", "clip_norm".

Return type:

dict

fit(x, y, epochs=10, batch_size=32, validation_data=None, shuffle=True, verbose=1, callbacks=None)

Train the model.

Parameters:

• x (ndarray) – Input data of shape (N, ...).

• y (ndarray) – Target data of shape (N, ...).

• epochs (int) – Number of full passes over the dataset.

• batch_size (int) – Number of samples per gradient update.

• validation_data (tuple, optional) – (X_val, y_val) evaluated at the end of each epoch.

• shuffle (bool) – Whether to shuffle the training data each epoch.

• verbose (int) – 0 = silent, 1 = one line per epoch.

• callbacks (list, optional) – Callables invoked at the end of each epoch as callback(epoch, history).

Returns:

Training history. Keys are "loss", each metric name, and "val_*" equivalents when validation_data is provided.

Return type:

dict

predict(x, batch_size=None)

Run inference on x.

Parameters:

• x (ndarray)

• batch_size (int, optional) – If given, runs inference in mini-batches (saves memory for large datasets).

Returns:

Predictions of shape (N, output_units).

Return type:

ndarray

evaluate(x, y, batch_size=None, verbose=1)

Compute loss and metrics on (x, y).

Parameters:

• x (ndarray)

• y (ndarray)

• batch_size (int, optional)

• verbose (int)

Returns:

{"loss": float, metric_name: float, …}

Return type:

dict

summary()

Print a table of layers, output shapes, and parameter counts.

get_weights()

Return a list of per-layer weight dicts (copies).

set_weights(weights)

Restore weights from the format returned by get_weights().

save_weights(path)

Save all trainable weights to a NumPy .npz file.

Parameters:

path (str) – Filename prefix (the .npz extension is added automatically).

load_weights(path)

Load weights from a .npz file written by save_weights().

Parameters:

path (str) – Path to the .npz file (with or without extension).

class snn.nn.Model

Bases: object

Subclassable model class — define call() and backward() yourself.

All Layer instances assigned as attributes (or stored in lists/tuples of attributes) are auto-discovered for parameter collection and optimizer updates. The compile / fit / predict / evaluate API is identical to Sequential.

Examples

Basic usage (functionally identical to Sequential):

class MyNet(Model):
    def __init__(self):
        super().__init__()
        self.fc1 = Dense(128, activation='relu')
        self.bn   = BatchNormalization()
        self.fc2  = Dense(10, activation='softmax')

    def call(self, x, training=False):
        x = self.fc1(x, training=training)
        x = self.bn(x, training=training)
        return self.fc2(x, training=training)

    def backward(self, grad):
        grad = self.fc2.backward(grad)
        grad = self.bn.backward(grad)
        return self.fc1.backward(grad)

model = MyNet()
model.compile('adam', 'categorical_crossentropy', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=20)

Residual block (skip connections):

class ResBlock(Model):
    def __init__(self, units):
        super().__init__()
        self.fc1 = Dense(units, activation='relu')
        self.fc2 = Dense(units)                 # no activation here
        self.proj = Dense(units)                # shortcut projection

    def call(self, x, training=False):
        shortcut = self.proj(x, training=training)
        h = self.fc1(x, training=training)
        h = self.fc2(h, training=training)
        self._pre_act = h + shortcut
        return np.maximum(0, self._pre_act)     # ReLU after residual

    def backward(self, grad):
        d = grad * (self._pre_act > 0)          # ReLU gate
        d_proj = self.proj.backward(d)
        d = self.fc2.backward(d)
        d = self.fc1.backward(d)
        return d + d_proj

call(x, training=False)

Forward pass. Override in subclasses.

backward(grad)

Backward pass. Override in subclasses.

compile(optimizer='adam', loss='mse', metrics=None, *, learning_rate=None, weight_decay=None, clip_norm=None, from_logits=False, verbose=False)

Configure the model — identical interface to Sequential.

Parameters:

• optimizer (str, Optimizer, or dict)

• loss (str, Loss, or dict)

• metrics (str or list, optional)

• learning_rate (float, optional)

• weight_decay (float, optional)

• clip_norm (float, optional)

• from_logits (bool)

• verbose (bool)

fit(x, y, epochs=10, batch_size=32, validation_data=None, shuffle=True, verbose=1, callbacks=None)

Train the model. Identical interface to Sequential.fit.

predict(x, batch_size=None)

Run inference on x. Returns predictions as a NumPy array.

evaluate(x, y, batch_size=None, verbose=1)

Compute loss and metrics on (x, y).

summary()

Print all discovered Layer types and parameter counts.

save_weights(path)

Save all trainable weights to a .npz file.

load_weights(path)

Load weights from a .npz file written by save_weights().

get_compile_config()

class snn.nn.GraphModel(inputs, outputs)

Bases: object

Functional-API model: build arbitrary computation graphs with skip connections, multi-input layers, and shared weights.

Build a graph by connecting layers to Input nodes, then pass the terminal node(s) to GraphModel. The model traces the DAG, sorts it topologically, and handles forward / backward automatically.

Examples

Simple feedforward (equivalent to Sequential):

from snn.model import Input, GraphModel
from snn.layers import Dense

x   = Input(shape=(784,))
h   = Dense(256, activation='relu')(x)
h   = Dense(128, activation='relu')(h)
out = Dense(10, activation='softmax')(h)

model = GraphModel(inputs=x, outputs=out)
model.compile('adam', 'categorical_crossentropy', metrics=['accuracy'])
model.fit(X_train, y_train, epochs=20)

Residual / skip connection:

x      = Input(shape=(128,))
skip   = Dense(64)(x)                        # shortcut
h      = Dense(64, activation='relu')(x)
h      = Dense(64)(h)                        # same size as skip
merged = Add()([h, skip])                    # element-wise add
out    = Dense(10, activation='softmax')(merged)

model = GraphModel(inputs=x, outputs=out)

Multi-branch (text + metadata fusion):

text_in  = Input(shape=(300,))              # e.g. mean-pooled GloVe
meta_in  = Input(shape=(10,))               # numerical features

h_text   = Dense(128, 'relu')(text_in)
h_meta   = Dense(32, 'relu')(meta_in)
merged   = Concatenate()([h_text, h_meta])  # (batch, 160)
out      = Dense(5, 'softmax')(merged)

model = GraphModel(inputs=[text_in, meta_in], outputs=out)
model.fit([X_text, X_meta], y, epochs=10)

Parameters:

• inputs (Input or list of Input) – Entry-point node(s).

• outputs (_Tensor or list of _Tensor) – Terminal node(s) whose values are returned by predict.

compile(optimizer='adam', loss='mse', metrics=None, *, learning_rate=None, weight_decay=None, clip_norm=None, from_logits=False, verbose=False)

Configure the model. Identical interface to Sequential.

Parameters:

• optimizer (str, Optimizer, or dict)

• loss (str, Loss, or dict)

• metrics (str or list, optional)

• learning_rate (float, optional)

• weight_decay (float, optional)

• clip_norm (float, optional)

• from_logits (bool)

• verbose (bool)

fit(x, y, epochs=10, batch_size=32, validation_data=None, shuffle=True, verbose=1, callbacks=None)

Train the model. Identical interface to Sequential.fit.

predict(x, batch_size=None)

Run inference on x.

evaluate(x, y, batch_size=None, verbose=1)

Compute loss and metrics on (x, y).

summary()

Print all graph nodes and parameter counts.

save_weights(path)

Save all weights to a .npz file.

load_weights(path)

Load weights from a .npz file.

class snn.nn.Input(shape)

Bases: _Tensor

Entry-point node for the GraphModel functional API.

Parameters:

shape (tuple) – Shape of a single sample (without the batch dimension).

Examples

>>> x = Input(shape=(784,))
>>> h = Dense(128, activation='relu')(x)
>>> out = Dense(10, activation='softmax')(h)
>>> model = GraphModel(inputs=x, outputs=out)

snn.nn.to_categorical(y, num_classes=None)

Convert integer class labels to one-hot vectors.

snn.nn.train_test_split(x, y, test_size=0.2, shuffle=True, seed=None)

Split arrays into train and test sets.

class snn.nn.EarlyStopping(monitor='val_loss', patience=5, min_delta=0.0, mode='auto', restore_best_weights=False, verbose=1)

Bases: object

Stop training when a monitored metric has stopped improving.

Parameters:

• monitor (str) – Name of the metric to watch (e.g. ‘val_loss’).

• patience (int) – Number of epochs with no improvement before stopping.

• min_delta (float) – Minimum change to qualify as an improvement.

• mode (str) – ‘min’ for loss-like metrics, ‘max’ for accuracy-like.

• restore_best_weights (bool) – If True, restore the best weights when training stops.

property stopped

class snn.nn.ReduceLROnPlateau(monitor='val_loss', factor=0.5, patience=3, min_lr=1e-06, verbose=1)

Bases: object

Reduce learning rate when a metric has stopped improving.

Parameters:

• monitor (str)

• factor (float) – Factor by which the learning rate will be reduced.

• patience (int)

• min_lr (float) – Lower bound for the learning rate.

• verbose (int)

class snn.nn.Trainer(model, gradient_accumulation_steps=1, mixed_precision=False, clip_grad_norm=None, checkpoints=None, verbose=1)

Bases: object

Advanced training loop for a Sequential model.

Extends the built-in model.fit() with:

• Gradient accumulation — accumulate gradients over N micro-batches before a weight update. Useful for effective large batch sizes when memory is limited.

• Mixed-precision simulation — casts activations/gradients to float16 and immediately back to float64 during the forward/ backward pass, simulating the precision loss of hardware FP16 without requiring actual hardware support.

• Multi-metric checkpointing — track any number of metrics and save the best weights for each independently.

• Callbacks — plug in EarlyStopping, ReduceLROnPlateau, or any callable (epoch, history, model, optimizer) → None.

Parameters:

• model (Sequential) – Compiled Sequential model.

• gradient_accumulation_steps (int) – Number of micro-batches to accumulate before calling optimizer.apply_gradients. Effective batch size = batch_size × gradient_accumulation_steps.

• mixed_precision (bool) – Simulate FP16 mixed precision.

• clip_grad_norm (float or None) – If set, clip the global gradient norm to this value before each optimizer step.

• checkpoints (list[Checkpoint] or None) – One or more Checkpoint objects. Each monitors its own metric.

• verbose (int) – 0 = silent, 1 = one line per epoch, 2 = one line per step.

Examples

>>> from snn.trainer import Trainer, Checkpoint
>>> ckpt = Checkpoint(monitor='val_loss', save_path='best_model')
>>> trainer = Trainer(
...     model,
...     gradient_accumulation_steps=4,
...     mixed_precision=True,
...     clip_grad_norm=1.0,
...     checkpoints=[ckpt],
... )
>>> history = trainer.fit(
...     X_train, y_train,
...     epochs=50,
...     batch_size=32,
...     validation_data=(X_val, y_val),
... )
>>> ckpt.restore(model)   # reload best weights

fit(x, y, epochs=10, batch_size=32, validation_data=None, shuffle=True, callbacks=None)

Train the model.

Parameters:

• x (ndarray) – Input features, shape (N, ...).

• y (ndarray) – Targets, shape (N, ...).

• epochs (int) – Total number of passes over the dataset.

• batch_size (int) – Micro-batch size (before accumulation).

• validation_data (tuple (x_val, y_val), optional) – If provided, compute validation metrics each epoch.

• shuffle (bool) – Shuffle training data each epoch.

• callbacks (list, optional) – List of callables invoked as cb(epoch, history, model, optimizer).

Returns:

Training history.

Return type:

dict

class snn.nn.Checkpoint(monitor='val_loss', mode='auto', save_path='checkpoint', verbose=1)

Bases: object

Saves and restores model weights when a monitored metric improves.

Parameters:

• monitor (str) – Name of the metric to watch (e.g. 'val_loss', 'val_accuracy').

• mode (str) – 'min' for loss-like metrics, 'max' for accuracy-like metrics, 'auto' to infer from the metric name.

• save_path (str, optional) – Path prefix for the .npz file. Defaults to 'checkpoint'.

• verbose (int) – Verbosity level (0 = silent, 1 = print on save).

update(current, model)

restore(model)

save_to_disk(model)

snn.nn.accuracy(y_pred, y_true)

Classification accuracy.

snn.nn.binary_accuracy(y_pred, y_true, threshold=0.5)

snn.nn.categorical_accuracy(y_pred, y_true)

snn.nn.precision(y_pred, y_true, threshold=0.5)

snn.nn.recall(y_pred, y_true, threshold=0.5)

snn.nn.f1_score(y_pred, y_true, threshold=0.5)

snn.nn.r2_score(y_pred, y_true)

snn.nn.confusion_matrix(y_pred, y_true, n_classes=None)

snn.nn.get_metric(identifier)

snn.nn.Layer_Dense

alias of Dense

snn.nn.Layer_Dropout

alias of Dropout

snn.nn.Layer_Flatten

alias of Flatten

snn.nn.Layer_Reshape

alias of Reshape

snn.nn.Layer_Conv2D

alias of Conv2D

snn.nn.Layer_Conv1D

alias of Conv1D

snn.nn.Layer_MaxPooling2D

alias of MaxPooling2D

snn.nn.Layer_AveragePooling2D

alias of AveragePooling2D

snn.nn.Layer_MaxPooling1D

alias of MaxPooling1D

snn.nn.Layer_AveragePooling1D

alias of AveragePooling1D

snn.nn.Layer_LSTM

alias of LSTM

snn.nn.Layer_GRU

alias of GRU

snn.nn.Layer_SimpleRNN

alias of SimpleRNN

snn.nn.Layer_Embedding

alias of Embedding

snn.nn.Layer_BatchNorm

alias of BatchNormalization

snn.nn.Layer_LayerNorm

alias of LayerNormalization

snn.nn.Layer_Residual

alias of Residual

snn.nn.Layer_TimeDistributed

alias of TimeDistributed

snn.nn.Layer_TransformerBlock

alias of TransformerBlock

snn.nn.Activation_Linear

alias of Linear

snn.nn.Activation_ReLU

alias of ReLU

snn.nn.Activation_LeakyReLU

alias of LeakyReLU

snn.nn.Activation_ELU

alias of ELU

snn.nn.Activation_SELU

alias of SELU

snn.nn.Activation_Sigmoid

alias of Sigmoid

snn.nn.Activation_Tanh

alias of Tanh

snn.nn.Activation_Softmax

alias of Softmax

snn.nn.Activation_Softplus

alias of Softplus

snn.nn.Activation_Swish

alias of Swish

snn.nn.Activation_Mish

alias of Mish

snn.nn.Activation_GELU

alias of GELU

snn.nn.Activation_PReLU

alias of PReLU

snn.nn.Activation_Sine

alias of Sine

snn.nn.Activation_Hardswish

alias of Hardswish

snn.nn.Activation_ReLU6

alias of ReLU6

snn.nn.Activation_Hardsigmoid

alias of Hardsigmoid

snn.nn.Activation_LogSoftmax

alias of LogSoftmax

snn.nn.Activation_Sparsemax

alias of Sparsemax

snn.nn.Activation_CELU

alias of CELU

snn.nn.Activation_Softsign

alias of Softsign

snn.nn.Activation_Tanhshrink

alias of Tanhshrink

snn.nn.Loss_MSE

alias of MeanSquaredError

snn.nn.Loss_MAE

alias of MeanAbsoluteError

snn.nn.Loss_Huber

alias of HuberLoss

snn.nn.Loss_BCE

alias of BinaryCrossentropy

snn.nn.Loss_CCE

alias of CategoricalCrossentropy

snn.nn.Loss_SparseCCE

alias of SparseCategoricalCrossentropy

snn.nn.Loss_KLD

alias of KLDivergence

snn.nn.Optimizer_SGD

alias of SGD

snn.nn.Optimizer_Adam

alias of Adam

snn.nn.Optimizer_AdamW

alias of AdamW

snn.nn.Optimizer_RMSprop

alias of RMSprop

snn.nn.Optimizer_Adagrad

alias of Adagrad

snn.nn.Optimizer_Adadelta

alias of Adadelta

snn.nn.Optimizer_Nadam

alias of Nadam

snn.nn.Optimizer_RAdam

alias of RAdam

snn.nn.Optimizer_Lion

alias of Lion

snn.nn.Optimizer_LAMB

alias of LAMB

snn.nn.Optimizer_Lookahead

alias of Lookahead

snn.nn.Optimizer_Adan

alias of Adan