Metadata-Version: 2.4
Name: spingalett
Version: 1.6.1
Summary: Neural networks in C23 from Python: training, graphs, ONNX and PyTorch import, INT8 to INT2 deployment
License-Expression: MIT
Project-URL: Homepage, https://github.com/pka-human/Spingalett
Keywords: neural networks,deep learning,inference,quantization,onnx,microcontrollers
Classifier: Development Status :: 5 - Production/Stable
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: C
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.20

# Spingalett for Python

Thin `ctypes` bindings over the Spingalett shared library: no compiler is needed to install them,
only NumPy and `libspingalett` (`.so` / `.dylib` / `.dll`). The wheels on PyPI and on the GitHub
releases carry the library inside (Linux x86-64 and AArch64 with glibc 2.28 or newer, Windows
x86-64, macOS 11 and newer, for any Python 3), with OpenMP:

```bash
pip install spingalett
```

From a source checkout, build the library and point the bindings at it:

```bash
cmake -S . -B Build -DCMAKE_BUILD_TYPE=Release -DBUILD_WITH_OPENMP=ON
cmake --build Build --parallel            # produces Bin/libspingalett.so
pip install ./Bindings/Python
export SPINGALETT_LIBRARY=$PWD/Bin/libspingalett.so   # or put Bin/ on the library path
```

Without `SPINGALETT_LIBRARY` the package looks inside itself (wheels), then in the repository's
`Bin/` directory (when imported from a checkout), then on the system library path. The package is
typed (`py.typed`): editors and type checkers read its annotations.

```python
import numpy as np
import spingalett as sg

x = np.array([[0, 0], [0, 1], [1, 0], [1, 1]], dtype=np.float32)
y = np.array([[0], [1], [1], [0]], dtype=np.float32)

sg.set_compute_mode(sg.ComputeMode.OPENBLAS)   # falls back to single-threaded if unavailable
sg.seed(42)

layers = [
    sg.Layer(2),
    sg.Layer(16, sg.Activation.TANH, sg.Init.XAVIER, dropout=0.1),
    sg.Layer(1, sg.Activation.SIGMOID, sg.Init.XAVIER),
]
with sg.Network(sg.Loss.MSE, layers) as net:
    net.train(x, y, epochs=3000, optimizer=sg.Optimizer.ADAMW, learning_rate=0.02,
              weight_decay=1e-4, lr_scheduler=sg.WarmupCosine(warmup_epochs=100),
              callback=lambda net, progress: progress.train_loss < 1e-4, callback_interval=100)
    print(net.forward(x))          # one row per sample
    net.save("xor", precision=sg.Precision.FP16)

# validation, early stopping and the best epoch's weights
x_train, y_train = sg.load_idx("train-images-idx3-ubyte", "train-labels-idx1-ubyte", num_classes=10)
with sg.Network(sg.Loss.CROSS_ENTROPY, [784, sg.Layer(128, sg.Activation.RELU, sg.Init.HE),
                                        sg.Layer(10, sg.Activation.SOFTMAX, sg.Init.XAVIER)]) as net:
    result = net.train(x_train[:55000], y_train[:55000], epochs=30, strategy=sg.Strategy.MINI_BATCH,
                       batch_size=128, optimizer=sg.Optimizer.ADAMW, learning_rate=1e-3,
                       validation_data=(x_train[55000:], y_train[55000:]), monitor=sg.Monitor.VAL_ACCURACY,
                       early_stopping_patience=3, restore_best_weights=True)
    print(result.status.name, result.best_epoch, net.evaluate(x_train[55000:], y_train[55000:]))

# a convolutional network: images as rows of height x width x channels floats (channels last)
cnn = sg.Network(sg.Loss.CROSS_ENTROPY, [
    sg.Input(28, 28, 1),
    sg.Conv2D(32, 3, padding=1), sg.MaxPool2D(2),             # ReLU and He initialization by default
    sg.Conv2D(64, 3, padding=1), sg.MaxPool2D(2),
    sg.Layer(128, sg.Activation.RELU, sg.Init.HE, dropout=0.3),
    sg.Layer(10, sg.Activation.SOFTMAX, sg.Init.XAVIER),
])
cnn.train(x_train[:55000], y_train[:55000], epochs=2, strategy=sg.Strategy.MINI_BATCH, batch_size=128,
          optimizer=sg.Optimizer.ADAMW, learning_rate=1e-3)
print([(l.type.name, l.shape) for l in cnn.layers], cnn.get_weights(0).shape)   # filters x 3 x 3 x 1

# batch normalization, a depthwise-separable block and image augmentation on CIFAR-10
x_cifar, y_cifar = sg.load_cifar([f"cifar-10-batches-bin/data_batch_{i}.bin" for i in range(1, 6)])
bn = sg.Network(sg.Loss.CROSS_ENTROPY, [
    sg.Input(32, 32, 3),
    sg.Conv2D(32, 3, padding=1, activation=sg.Activation.NONE), sg.BatchNorm(sg.Activation.RELU),
    sg.Conv2D(32, 3, padding=1, groups=32, activation=sg.Activation.NONE), sg.BatchNorm(sg.Activation.RELU),
    sg.Conv2D(64, 1, activation=sg.Activation.NONE), sg.BatchNorm(sg.Activation.RELU), sg.MaxPool2D(2),
    sg.Layer(10, sg.Activation.SOFTMAX, sg.Init.XAVIER),
])
bn.train(x_cifar, y_cifar, epochs=5, strategy=sg.Strategy.MINI_BATCH, batch_size=128,
         optimizer=sg.Optimizer.ADAMW, learning_rate=2e-3, augment_shift=4, augment_flip=True)

# a residual block: inputs= names earlier layers, add_add / add_concat combine them
res = sg.Network(sg.Loss.CROSS_ENTROPY)
x = res.add_input(32, 32, 3).add_conv2d(16, 3, padding=1, activation=sg.Activation.NONE) \
       .add_batch_norm(sg.Activation.RELU).last
res.add_conv2d(16, 3, padding=1, activation=sg.Activation.NONE).add_batch_norm(sg.Activation.RELU)
res.add_conv2d(16, 3, padding=1, activation=sg.Activation.NONE).add_batch_norm()
res.add_add([x, -1], activation=sg.Activation.RELU)       # -1: the layer before
res.add_global_avg_pool().add_layer(10, sg.Activation.SOFTMAX, sg.Init.XAVIER)

# a U-Net: transposed convolutions up, concatenated with the maps on the way down; a sigmoid a pixel
unet = sg.Network(sg.Loss.CROSS_ENTROPY)
e = unet.add_input(64, 64, 3).add_conv2d(16, 3, padding=1).last                  # 64 x 64 x 16
unet.add_max_pool2d(2).add_conv2d(32, 3, padding=1)                                 # 32 x 32 x 32
u = unet.add_conv_transpose2d(16, 2, stride=2).last                                 # 64 x 64 x 16
unet.add_concat([e, u]).add_layer_norm(sg.Activation.RELU)                          # 64 x 64 x 32
unet.add_upsample(2, sg.Upsample.BILINEAR).add_conv2d(1, 1, activation=sg.Activation.SIGMOID)

# PyTorch: a module through ONNX in memory, or a state dict into a network of the same layers
import torch
model = torch.nn.Sequential(torch.nn.Conv2d(3, 8, 3, padding=1), torch.nn.ReLU(), torch.nn.Flatten(),
                            torch.nn.Linear(8 * 32 * 32, 10))
net = sg.Network.from_torch(model, torch.rand(1, 3, 32, 32))     # takes (32, 32, 3) channels-last images
same = sg.Network(sg.Loss.MSE, [sg.Input(32, 32, 3), sg.Conv2D(8, 3, padding=1), sg.Layer(10, sg.Activation.NONE)])
same.load_pytorch(model.state_dict())                              # or "model.pt", "model.safetensors"
onnx = sg.Network.from_onnx("model.onnx")
# transformers too (GPT-2, LLaMA, BERT-like, nn.TransformerEncoderLayer): token ids in, as rows of floats
lm = sg.Network.from_torch(my_gpt, torch.randint(0, 50257, (1, 256)))   # attention, RMS norm, SwiGLU ... as layers
logits = lm.forward(tokens.astype(np.float32))      # tokens: (samples, 256) ids; logits: (samples, 256 x 50257)

# any ONNX model as it is, as a graph of operators (NCHW images and all), or a PyTorch module through torch.onnx
graph = sg.Graph.load("model.onnx")                       # or a .slett file of format 11
print(graph.inputs, graph.outputs, graph.operators)
outs = graph.run({"image": np.zeros((1, 3, 224, 224), np.float32)})       # a dict of arrays, by output name
vision = sg.Graph.from_torch(model, torch.rand(1, 3, 32, 32), input_names=["image"])
vision.save("model.slett")

# a pretrained language model and its tokenizer, from a Hugging Face directory or a GGUF file, read by the C library
lm = sg.Network.from_pretrained("models/SmolLM2-135M", context=512)       # or "SmolLM2-135M.Q4_K_M.gguf"
tok = sg.Tokenizer("models/SmolLM2-135M")
prompt = tok.encode("<|im_start|>user\nWhat is a tokenizer?<|im_end|>\n<|im_start|>assistant\n", special=True)
print(tok.decode(lm.generate(prompt, 64, temperature=0.7, top_p=0.9, stop=[tok.id("<|im_end|>")])))
# a GGUF file as it is stored (its quantized types as block precisions), run by a session
with sg.Model.load("Llama-3.2-1B-Instruct-Q4_K_M.gguf", context=4096) as model, model.session() as s:
    tok = sg.Tokenizer("Llama-3.2-1B-Instruct-Q4_K_M.gguf")
    print(tok.decode(s.generate([tok.encode("The capital of France is", add_special=True)], 32)[0]))

# a language model: GPT-2 from Hugging Face's weights (or nanoGPT's), trained further on a token file, sampled
from transformers import GPT2LMHeadModel
gpt = sg.gpt2_from_state_dict(GPT2LMHeadModel.from_pretrained("gpt2").state_dict(), heads=12, context=256)
sg.set_compute_mode(sg.ComputeMode.CUDA); sg.set_gpu_precision(sg.Precision.BFLOAT16)
gpt.train_from_tokens("train.bin", epochs=1, batch_size=8, optimizer=sg.Optimizer.ADAMW, learning_rate=3e-4)
print(gpt.generate([15496, 11, 314], 50, temperature=0.8, top_k=40))

# or one of the library's own: a LLaMA-like model, rotary embeddings and grouped queries, a SwiGLU MLP
lm = sg.Network(sg.Loss.SPARSE_CROSS_ENTROPY).add_layer(256).add_embedding(32000, 512)   # windows of 256 tokens
x = lm.last
lm.add_rms_norm().add_linear(512 + 2 * 128).add_attention(heads=8, kv_heads=2, rope_theta=10000.0, causal=True)
x = lm.add_linear(512).add_add([x, lm.last]).last
n = lm.add_rms_norm().last
g = lm.add_linear(1376, activation=sg.Activation.SILU).last
lm.add_linear(1376, inputs=n).add_multiply([g, lm.last]).add_linear(512).add_add([x, lm.last])
lm.add_rms_norm().add_linear(32000)

# its deployment model a token at a time with a key-value cache: four continuations of one prompt together
with lm.to_model(sg.Precision.INT8) as model, model.session(sequences=4) as s:
    logits = s.feed([1, 450, 4996])                 # the prompt's tokens at once: the next token's logits
    for b in range(1, 4): s.copy(0, b)
    print(s.generate([[int(logits.argmax())]] * 4, 50, temperature=0.8, top_k=40, seed=1))

with sg.Network.load("xor.slett") as net:
    print(net.topology, net.forward([1, 0]))
    # deployment: a read-only INT8 model with integer kernels, and a C header for firmware
    with net.to_model(sg.Precision.INT8) as model:
        print(model.predict([1, 0]), model.layers, model.size)
    net.export_c_header("xor_model.h", "xor_model", sg.Precision.INT8)
```

| API | Notes |
|---|---|
| `Network(loss, layers)` / `add_layer(...)` | first layer is the input layer; `layers` may mix `Layer`, `Input`, `Conv2D`, `ConvTranspose2D`, `MaxPool2D`, `AvgPool2D`, `Upsample2D`, `BatchNorm`, `LayerNorm`, `Add`, `Concat`, `GlobalAvgPool`, `Linear`, `Embedding`, `Attention`, `RMSNorm`, `Multiply` and plain widths |
| `add_embedding(vocabulary, size, positions=False)`, `add_linear(neurons, activation=NONE)`, `add_attention(heads=1, kv_heads=0, causal=False, rope_theta=0)`, `add_rms_norm(epsilon=1e-5)`, `add_multiply(inputs)` | transformers: a window of token ids (the input layer, `add_layer(tokens)`) as vectors, a linear map of each position, multi-head attention over packed queries, keys and values (grouped-query, causal, rotary), RMS normalization, element-wise products (SwiGLU); activations `GELU`, `GELU_TANH`, `SILU`; `Loss.SPARSE_CROSS_ENTROPY` takes `target_size` class indices a sample |
| `train_from_tokens(path, token_bytes=0, stride=0, offset=0, shuffle=True, ...)`, `generate(prompt, count, temperature=0, top_k=0, top_p=0, seed=0, stop=())` | a language model on the windows of a token file (nanoGPT's `.bin`, llm.c's), through the C reader; the continuation of a prompt of token ids, a token at a time |
| `gpt2_from_state_dict(state, heads, context=None, activation=None)`, `llama_from_state_dict(state, heads, kv_heads=None, context=2048, rope_theta=10000, epsilon=1e-6, window=0)` | networks of PyTorch checkpoints: GPT-2 (Hugging Face's or nanoGPT's state dict) and LLaMA's form in Hugging Face's names (LLaMA, Mistral, Qwen2); the weights copied, the head tied to the embedding (`add_linear(..., tied=1)`) when the checkpoint ties them; the networks train, generate and save as any other |
| `add_linear(neurons, ..., tied=e)`, `Linear(..., tied=e)` | a linear layer computing with embedding `e`'s table (tied input and output embeddings); `layer(i).tied`, `model.layers[i].tied` name it |
| `add_attention(..., ignore_padding=True)`, `Attention(..., ignore_padding=True)` | the keys of padding (tokens of -1 in the inputs, which an embedding reads as zeros and the sparse loss as no target) left out: sequences of different lengths in one batch |
| `add_attention(..., window=w)`, `Attention(..., window=w)` | a sliding window (Mistral's): each cell attends to the keys less than `w` cells away, with `causal` the cell and the `w - 1` before it; `layer(i).window`, `model.layers[i].window` give it |
| `train(..., recompute=True)` | activation recomputation (checkpointing) on the CPU: the outputs of the layers several read (a transformer's residual stream) kept, the others computed again a block at a time in the backward pass; the same weights bit for bit, a third less memory for about one more forward pass |
| `add_attention(..., inputs=[q, kv])`, `add_embedding(..., token_offset=o, token_count=n)` | cross-attention (an encoder-decoder's): the queries of layer `q` over the keys and values of layer `kv`; an embedding of the input's values `o` to `o + n` (an encoder's and a decoder's tokens side by side in one input, the second embedding reading `inputs=[0]`) |
| `Network.from_pretrained(path, context=0)` | a language model read by the C library: a Hugging Face model directory (`config.json`, safetensors weights, sharded or not; `llama`, `mistral`, `qwen2`, `gpt2`) or a GGUF file (`llama`, `qwen2`, `gpt2`; weights dequantized from F32, F16, BF16, Q8_0, Q4_0, Q4_1, Q5_0, Q5_1, Q2_K to Q6_K, IQ4_NL, IQ4_XS); the same networks as the two functions above, without PyTorch |
| `Tokenizer(path or bytes)`: `encode(text, special=False, add_special=False)`, `decode(ids, skip_special=False)`, `token(id)`, `id(token)`, `vocabulary`, `bos`, `eos`, `pad`, `unk`, `added` | a language model's tokenizer in C (a `tokenizer.json`, a model directory or a GGUF file): byte-level BPE, tiktoken's encodings and SentencePiece's BPE, the ids and texts of Hugging Face's `tokenizers`; `special` turns special tokens written in the text into their ids, `add_special` adds the template's (LLaMA's BOS) |
| `inputs=` on every `add_*`, `last`, `len(net)` | a layer reads the one before it, or the earlier layers `inputs` names (indices; negative ones count back from the new layer); `last` is the index of the layer added last |
| `add_add(inputs, activation=NONE, dropout=0)`, `add_concat(inputs, ...)`, `add_global_avg_pool(dropout=0, inputs=None)` | the sum of layers of one shape (residual connections), layers side by side along the channels, the mean of each channel |
| `Network.from_onnx(path or bytes)`, `Network.from_torch(module, example_input)` | ONNX models (and PyTorch modules, exported to ONNX in memory) as networks with channels-last inputs: transpose NCHW images with `x.transpose(0, 2, 3, 1)`; transformers (an `example_input` of token ids `[1, T]`, or of vectors `[1, T, C]`) with their embeddings, normalizations, attention (grouped-query, causal, rotary) and gated MLPs as the library's layers, token ids given as floats |
| `load_pytorch(state_dict, path or bytes, modules=None)` | PyTorch weights (`state_dict()`, `torch.save` or safetensors files) into a network of the same layers, reordered for channels-last data; the network is unchanged on error |
| `add_input(h, w, c)`, `add_conv2d(filters, kernel, stride=1, padding=0, activation=RELU, init=HE, dropout=0, groups=1)`, `add_max_pool2d(kernel, stride=0, padding=0)`, `add_avg_pool2d(...)` | convolution (grouped or depthwise with `groups`) and pooling over channels-last tensors; pooling's stride defaults to the kernel size |
| `add_batch_norm(activation=NONE, epsilon=1e-5, momentum=0.1, dropout=0)` | per-channel normalization of the previous layer: batch statistics while training, running averages otherwise; `get_running_statistics(i)` / `set_running_statistics(i, mean, var)` |
| `add_conv_transpose2d(filters, kernel, stride=1, padding=0, output_padding=0, activation=RELU, init=HE, dropout=0, groups=1)`, `add_upsample(factor=2, mode=Upsample.NEAREST)`, `add_layer_norm(activation=NONE, epsilon=1e-5)` | transposed convolution ((in - 1) stride - 2 padding + kernel + output_padding cells an axis), nearest or bilinear upsampling (PyTorch's `align_corners=False`), normalization of each cell over its channels (a dense layer's outputs are one cell) |
| `layers`, `layer(i)`, `topology` | `LayerDescription(type, shape, outputs, activation, dropout, kernel, stride, padding, weight_count, bias_count, groups, epsilon, momentum, inputs)` per layer |
| `forward(x)` | 1-D input -> vector, 2-D batch -> matrix (one batched `predict()` call; results are copies) |
| `train(x, y, config=None, validation_data=None, **overrides)` | fields of `TrainConfig` (e.g. `epochs`, `strategy`, `batch_size`, `shuffle`, `early_stopping_patience`, `restore_best_weights`, `augment_shift`, `augment_flip`, `label_smoothing`); returns a `TrainResult`; `callback(network, progress)` gets a `Progress`; exceptions raised in callbacks stop training and are re-raised |
| `train_from_generator(fn, samples_per_epoch=0, validation_data=None, ...)` | `fn(inputs, targets)` fills the given arrays and returns the number of rows; 0 ends the epoch |
| `evaluate(x, y)` | `Metrics(loss, accuracy)` |
| `Trainer(net, max_batch)` | `forward(x)`, `backward(y)`, `backward_output_grads(dl_dout)` for custom losses, `step(optimizer=..., learning_rate=...)`, `zero_grad()`, `train_on_batch(x, y, ...)`; gradients via `net.get_weight_gradients(i)` / `get_bias_gradients(i)`; on the GPU when made with `ComputeMode.VULKAN` or `ComputeMode.CUDA` |
| `load_idx(images, labels, num_classes=0)`, `load_cifar(paths, num_classes=10)`, `load_csv(path, target_columns=1, num_classes=0)` | return `(inputs, targets)` float32 arrays; CIFAR images as 32 x 32 x 3, channels last |
| `save_dataset(path, x, y, input_encoding=AUTO, target_encoding=AUTO, compress=True, shape=None, class_names=None, target_name=None, extra_targets=None)`, `load_dataset(path, target_set=0)`, `dataset_info(path)` | `.slettd` data set files, optionally with the input shape, class names and further sets of targets (`extra_targets`: dicts with `"targets"` and optionally `"name"`, `"class_names"`, `"encoding"`); `dataset_info` returns the counts, encodings, `shape` and `target_sets` |
| `train_from_file(path, shuffle=True, in_memory=False, prefetch=True, target_set=0, ...)` | trains on a `.slettd` file through the C reader: streamed with a few chunks in memory, or with `in_memory=True` decoded once and kept in its compact form (a byte per 8-bit value); `prefetch=False` keeps decoding off a background thread |
| `CosineDecay`, `LinearWarmup`, `StepDecay`, `WarmupCosine`, `LinearDecay`, `WarmupLinear` | built-in schedules; any `fn(epoch, total, initial_lr)` works too; with `lr_per_step=True` they count optimizer steps instead of epochs (`warmup_epochs` too) |
| `accumulation_steps=k` (`TrainConfig`) | gradient accumulation: an optimizer step after every `k` mini-batches of `batch_size` samples, with the mean gradient of their samples (the memory of one mini-batch; the bits of one batch of `k x batch_size` samples computed in chunks of `batch_size`) |
| `get_weights(i)`, `set_weights(i, w)`, `get_biases(i)`, `set_biases(i, b)` | weights `i` feed layer `i + 1`: shape `(out, in)` for a dense layer, `(filters, kernel_h, kernel_w, in_channels / groups)` for a convolution or a transposed one, gamma `(channels,)` for batch and layer normalization (beta are its biases), empty for pooling and upsampling |
| `save(path, precision, save_optimizer)`, `Network.load(path)` | `.slett` files, shared with the C API |
| `save_checkpoint(path)`, `Network.load_checkpoint(path)` | checkpoints of training runs: the network, its optimizer state and the run's state as its last epoch left it (seeds, Adam's corrections, epochs and steps, schedule, plateau, best epoch and its parameters, the reader's place). The training call after `load_checkpoint`, given the run's arguments again (`epochs` counting all the run's epochs), runs the epochs left to the bits of a run that never stopped, its reader (`train_from_file`, `train_from_tokens`) resumed where the run's stood. Autosaves in FLOAT32 with the optimizer state are checkpoints; checkpoints load as models too |
| `to_bytes(precision, save_optimizer=False)`, `Network.from_bytes(data)` | the same files as `bytes` |
| `to_model(precision=INT8)` | a `Model`: read-only, computes in its precision (integer kernels for INT8, INT4, INT2) |
| `Model.load(path, context=0)`, `Model.from_bytes(data)`, `model.to_bytes()` | models from and to `.slett` files of any version; `load()` opens a language model's GGUF file too (`llama`, `qwen2`, `qwen3`, `gemma3`, `gpt2`), its tensors as the file stores them (the quantized types as the block precisions `Q8_0` to `IQ4_XS`), with a window of `context` tokens (0: the model's, 4096 at most) |
| `model.predict(x)` / `model(x)`, `model.evaluate(x, y)` | 1-D input -> vector, 2-D batch -> matrix; `Metrics(loss, accuracy)` |
| `model.layers`, `input_size`, `output_size`, `size`, `workspace_size` | `LayerInfo(inputs, outputs, activation, precision, type, shape, input_shape, kernel, stride, padding, groups, epsilon, input_layers)` per layer (normalizations after a dense or convolution layer are folded into it); image and C workspace bytes |
| `export_c_header(path, name, precision=INT8)` | the model as a C header for the standalone engine (`Spingalett.Inference.h`) |
| `model.session(max_length=0, sequences=1, gpu_memory=0)` / `Session(model, ...)` | a causal language model run a token at a time with a key-value cache (`gpu_memory`: bytes of the GPU's memory it may take, in a GPU mode; a model of GGUF's types it does not hold splits with the CPU): `feed(tokens, sequence=0, all=False)` (the next token's logits, or every fed token's), `feed_many([(sequence, tokens), ...])` (several sequences in one pass), `rewind(length, sequence=0)`, `copy(source, target)`, `length(sequence=0)`, `vocabulary`, `generate(prompts, count, temperature=0, top_k=0, top_p=0, seed=0, stop=())` (sequences advanced together, sequence b drawing with seed + b); the logits are `predict()`'s for each position bit for bit; `Network.generate()` runs one on the CPU |
| `Graph.load(path)`, `Graph.from_bytes(data)`, `Graph.from_torch(module, *inputs, input_names=None, opset=18, **export_args)` | a graph of operators (docs/Graphs.md): an ONNX file (external data files beside it) or a `.slett` file of format 11, or a module exported by `torch.onnx` (its dynamo exporter) in memory; constants folded and chains fused into the library's kernels when it opens |
| `graph.inputs`, `graph.outputs`, `graph.operators`, `graph.run(inputs, outputs=None)`, `graph.save(path)` | `TensorInfo(name, dtype, shape)` with a symbol's name (`"batch"`) or `None` where no number gives a size; the nodes' operators after the fusions; `run` takes a dict of arrays by name (or a list in the inputs' order; float64 becomes float32) and returns copies of the outputs by name (all, or those named); the graph computes on the threads of `set_compute_mode(ComputeMode.OPENMP)`, the same bits on any number |
| `set_compute_mode`, `set_num_threads`, `seed`, `set_verbose`, `set_log_level`, `set_log_callback` | process-wide settings; `ComputeMode.CUDA` (NVIDIA) and `ComputeMode.VULKAN` train and predict on the GPU |
| `DeviceData(values)` | a data set in the GPU's memory (rows of float32, an array `(n, ...)` flattened to a row a sample), copied there once; `train()`, `validation_data`, `forward()` and `evaluate()` take it in place of an array (inputs, targets or both), and with the GPU modes gather, augment and smooth their batches on the device; `len(d)`, `d.shape`, `d.numpy(first=0, count=None)`, `close()` or a `with` block |
| `set_gpu_precision(Precision.BFLOAT16)`, `get_gpu_precision()` | the GPU's matrix products in bfloat16 on its matrix units, or `FLOAT32` (the default); returns whether the GPU multiplies in that precision |
| `cpu_kernels()`, `cuda_device()`, `gpu_device()`, `library_version()`, `library_path()` | the matrix kernels in use (`"AVX-512"`, `"AVX2"`, ...), the GPUs `CUDA` and `VULKAN` use (`None` without one), the loaded library |

`uint8` arrays stand for 8-bit images: value q means q / 255 in `train()`, `forward()`,
`predict()`, `evaluate()` and `save_dataset()`. `train()` keeps such inputs as bytes, a quarter of
the memory of float32, and converts them a batch at a time, with the same result as training on
`x / 255` in float32.

Library errors raise `SpingalettError`, whose `code` is an `ErrorCode`. The bindings check on import
that the library can serve them (`library_version()`), because they mirror its struct layouts: the
same major version and at least their own minor one (before 1.0, the same major.minor version). A
`Network` is not thread-safe: use one per thread. A `Model` is read-only and can be shared between
threads; a `Session` is used by one thread at a time, and any number of them may share a model. A `Graph` runs one
call at a time (calls from several threads wait for each other).
