Metadata-Version: 2.5
Name: topologies
Version: 0.0.2
Summary: A PyTorch framework for quick, convenient experiments with different neural-network architectures
Project-URL: Homepage, https://topologies.hackwhiz.net
Project-URL: Documentation, https://topologies.hackwhiz.net
Project-URL: Repository, https://github.com/yvsazh/topologies
Project-URL: Issues, https://github.com/yvsazh/topologies/issues
Author: Yaroslav Sazhchenko
License: Apache-2.0
License-File: LICENSE
Keywords: architectures,deep-learning,neural-networks,pytorch,transformer
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27
Requires-Dist: numpy>=1.23
Requires-Dist: pydantic>=2.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: tomli>=2.0; python_version < '3.11'
Requires-Dist: torch<2.13,>=2.2
Provides-Extra: all
Requires-Dist: datasets>=2.14; extra == 'all'
Requires-Dist: matplotlib>=3.7; extra == 'all'
Requires-Dist: pillow>=9; extra == 'all'
Requires-Dist: psutil>=5.9; extra == 'all'
Requires-Dist: pyarrow>=12; extra == 'all'
Requires-Dist: tensorboard>=2.12; extra == 'all'
Requires-Dist: tokenizers>=0.15; extra == 'all'
Requires-Dist: torchvision>=0.17; extra == 'all'
Provides-Extra: dev
Requires-Dist: build; extra == 'dev'
Requires-Dist: matplotlib>=3.7; extra == 'dev'
Requires-Dist: pytest-timeout>=2.3; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff==0.16.9; extra == 'dev'
Requires-Dist: twine; extra == 'dev'
Provides-Extra: introspection
Requires-Dist: matplotlib>=3.7; extra == 'introspection'
Provides-Extra: nlp
Requires-Dist: datasets>=2.14; extra == 'nlp'
Requires-Dist: tokenizers>=0.15; extra == 'nlp'
Provides-Extra: optimization
Requires-Dist: torchao==0.18.0; extra == 'optimization'
Provides-Extra: qt
Requires-Dist: pyside6>=6.5; extra == 'qt'
Provides-Extra: synth-parquet
Requires-Dist: pyarrow>=12; extra == 'synth-parquet'
Provides-Extra: system
Requires-Dist: psutil>=5.9; extra == 'system'
Provides-Extra: tensorboard
Requires-Dist: tensorboard>=2.12; extra == 'tensorboard'
Provides-Extra: vision
Requires-Dist: pillow>=9; extra == 'vision'
Requires-Dist: torchvision>=0.17; extra == 'vision'
Provides-Extra: visualization
Requires-Dist: matplotlib>=3.7; extra == 'visualization'
Description-Content-Type: text/markdown

# topologies

A PyTorch toolbox for building, training, comparing and inspecting neural-network
architectures — plus a catalog of small reference implementations (GPT, LLaMA, ViT,
ResNet, BERT, Mamba, ...).

> **Early alpha (v0.0.1).** The toolbox — training engine, data utilities, tokenizers,
> model constructor, generation, evaluation, experiments, introspection — is tested.
> Catalog models in `topologies.models` / `topologies.layers` are experimental
> references: importable, trainable and smoke-tested, but not validated against the papers.
> APIs may change between releases.

## Install

```bash
pip install topologies                 # core
pip install "topologies[all]"          # + vision, nlp, tensorboard, visualization, ...
```

Python 3.10–3.13, PyTorch `>=2.2,<2.13`.

## Quickstart

Train a tiny language model on any text corpus:

```python
from topologies import Trainer, build, data, generate, train_tokenizer
from topologies.nn import TransformerLM

docs = data.read_documents("corpus.txt")
train_docs, val_docs = data.train_val_split(docs, 0.1)
tok = train_tokenizer(train_docs, kind="bpe", vocab_size=2048)
model = build(TransformerLM, "tiny", vocab_size=tok.vocab_size, max_seq=128)

trainer = Trainer(model, data.lm_dataset(train_docs, tok, 128),
                  data.lm_dataset(val_docs, tok, 128),
                  tokenizer=tok, batch_size=32, max_epochs=20, lr=1e-3)
trainer.fit()
trainer.save_pretrained("artifacts/tiny-lm")
print(generate(model, "Once upon a time", tokenizer=tok, max_new_tokens=100, top_k=40))
```

Compare many configurations with one declaration — the library runs every trial
(resumably), then ranks and explains the results:

```python
from topologies import Experiment
from topologies.experiments import LanguageModeling, Variant

exp = Experiment("arch-x-tokenizer",
                 task=LanguageModeling(docs, val_fraction=0.1, block_size=128),
                 axes={"model": [TransformerLM,
                                 Variant("llama-ish", TransformerLM, pos="rope",
                                         norm="rms", activation="swiglu")],
                       "tokenizer": ["char", "bpe:1k", "bpe:4k"]},
                 target_params="1M", train_for="2 epochs", seeds=2)
exp.run().show()
```

## What's inside

- `topologies.nn` — composable stacks, presets and parameter budgets
- `Trainer`, `generate`, evaluation, optimization and quantization helpers
- `topologies.data`, dataset catalog, tokenizers
- `Experiment` — sweeps, leaderboards, pivots, importance analysis
- `topologies.introspection` / `topologies.visualization` — look inside trained models
- `topologies` CLI — `topologies --help`

## Links

- Documentation: <https://topologies.hackwhiz.net>
- Repository and issues: <https://github.com/yvsazh/topologies>

## License

Apache-2.0
