Metadata-Version: 2.4
Name: riftco-transformer
Version: 0.3.0
Summary: Dependency-free staged Python and C++ Transformer framework
Author: Quang T Nguyen
License-Expression: Apache-2.0
License-File: LICENSE
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: C++
Classifier: Typing :: Typed
Project-URL: Homepage, https://github.com/quangng2000/riftco-transformer
Project-URL: Issues, https://github.com/quangng2000/riftco-transformer/issues
Project-URL: Repository, https://github.com/quangng2000/riftco-transformer
Requires-Python: >=3.10
Description-Content-Type: text/markdown

# riftco-transformer Python distribution

This package is the typed, runtime-dependency-free-by-default `ctypes` interface to
`libriftco_transformer_c`, plus explicit data preparation, pretraining,
post-training, experiment, artifact, generation, and local-serving modules.
A platform wheel carries both the Python modules and its native C ABI library;
users do not install the native framework separately.

## Install

After a release has been published to PyPI:

```bash
python3 -m pip install riftco-transformer
python3 -c "from riftco_transformer import Context; print(Context().backend)"
```

`riftco-transformer` is the installable distribution name; Python code imports
the stable `riftco_transformer` package.

The wheel stores `libriftco_transformer_c.so`,
`libriftco_transformer_c.dylib`, or `riftco_transformer_c.dll` under
`riftco_transformer/.libs`. It has no third-party runtime dependencies and needs
no compiler or environment variable after installation. Initial binary wheels
cover Linux `x86_64` and `aarch64` for both glibc (`manylinux`) and musl
(`musllinux`), macOS `x86_64` and `arm64`, and Windows `AMD64`. CPU is
available on every supported platform; the macOS wheels also include Metal.
Standard wheels recognize the stable `cuda` and `tpu` backend names but compile
their unavailable stubs, so installing one does not add CUDA or `libtpu`
runtime dependencies.

From a source checkout, install at the repository root with:

```bash
python3 -m pip install .
```

That source build compiles the C++20 implementation, so it needs a supported
native compiler and platform SDK. `RIFTCO_TRANSFORMER_LIBRARY` remains an advanced
development override for selecting a particular local native build; released
wheels do not require it.

CUDA is an explicit source-build option. It needs CUDA Toolkit 12 or newer, a
compatible NVIDIA driver, and an NVIDIA GPU:

```bash
CMAKE_ARGS="-DRIFTCO_TRANSFORMER_ENABLE_CUDA=ON" \
  python3 -m pip install .
```

The CUDA backend provides managed tensor storage plus native GPU matmul,
materialized/Flash attention, attention gradients, and paged decode. Other
framework capabilities remain functional through synchronous audited reference
paths over that storage. Full fine-tuning, LoRA, held-out generalization
evaluation, artifact creation, and serving can all use `backend="cuda"`, but
this is not a claim that every operation is GPU-accelerated.

TPU is a separate experimental source-build option for Linux x86-64 Cloud TPU
VMs. It requires Google's external `libtpu.so` at runtime:

```bash
export RIFTCO_TRANSFORMER_TPU_LIBRARY=/absolute/path/to/libtpu.so
CMAKE_ARGS="-DRIFTCO_TRANSFORMER_ENABLE_TPU=ON" \
  python3 -m pip install .
```

The TPU backend compiles batched matmul, materialized attention and its
gradients, and paged decode through the PJRT C API and StableHLO. Flash
attention and the other capabilities use synchronous audited reference paths
over host-mirrored storage, so full fine-tuning, LoRA, evaluation, and serving
are functionally wired without implying an end-to-end TPU speedup. Real Cloud
TPU validation is still pending. Default installs never load `libtpu`.

The Python package follows the framework release version (`0.3.0` here), while
the native C ABI has its own compatibility version (`2.2`). The client accepts
the same ABI major and an equal or newer additive minor, and rejects older or
breaking ABIs before use.

Backend names are `"cpu"`, `"metal"`, `"cuda"`, and `"tpu"`. High-level
configurations also accept `"auto"`, which prefers TPU when available, then
CUDA, Metal, and CPU. An explicitly requested unavailable backend raises an
error instead of silently changing the experiment backend.

## Release automation

`.github/workflows/release.yml` builds and verifies the source distribution and
self-contained platform wheels. `workflow_dispatch` is verification-only.
Pushing a `v<version>` tag creates a GitHub Release after the artifacts pass.
It also publishes to PyPI only when the repository variable
`PUBLISH_TO_PYPI` is `true`.

PyPI publication uses Trusted Publishing rather than a stored API token. The
publisher configuration is project `riftco-transformer`, owner `quangng2000`,
repository `riftco-transformer`, workflow `release.yml`, and environment `pypi`.
The project is licensed under Apache-2.0. Do not enable publication until the
Trusted Publisher is configured.

## Selectable tokenizers

`Tokenizer` offers interchangeable byte and byte-pair-encoding strategies:

```python
from riftco_transformer import Tokenizer

with Tokenizer(
    "Hello, café 🙂 Hello again.",
    method="bpe",
    vocabulary_size=272,
    minimum_pair_frequency=2,
) as tokenizer:
    token_ids = tokenizer.encode("café 🙂")
    assert tokenizer.decode(token_ids) == "café 🙂"
    print(tokenizer.method, tokenizer.vocab_size)
    print(tokenizer.vocabulary)
```

The BPE vocabulary begins with all 256 single-byte tokens and appends learned
pair pieces, so unseen bytes remain encodable. `vocabulary_size` is a maximum:
learning can stop earlier when no pair reaches `minimum_pair_frequency`.
Repeated pair counts are resolved deterministically.

For backward compatibility, `Tokenizer(corpus)` selects the corpus-derived
byte method. It assigns IDs by sorted unsigned byte value:

```python
with Tokenizer(b"cab\ncab") as tokenizer:
    assert tokenizer.method == "byte"
    assert tokenizer.vocabulary_bytes == b"\nabc"
```

`Tokenizer` accepts a `str` corpus, encoded as UTF-8, or a
`bytes`/`bytearray`/`memoryview` corpus. `encode()` and `decode()` are strict
UTF-8 conveniences. Use `encode_bytes()` and `decode_bytes()` for arbitrary
binary data, including embedded NUL bytes. `vocabulary` returns a tuple of
byte pieces for either method; `vocabulary_bytes` is the byte-only
compatibility property.

## End-to-end training

An end-to-end BPE training step uses only public Python objects:

```python
from riftco_transformer import (
    Adam,
    DecoderOnlyTransformer,
    Tokenizer,
    TransformerConfig,
    cross_entropy,
)

corpus = "hello hello hello"
with Tokenizer(
    corpus,
    method="bpe",
    vocabulary_size=272,
) as tokenizer:
    encoded = tokenizer.encode(corpus)
    tokens = [encoded[:-1]]
    targets = [encoded[1:]]

    config = TransformerConfig(
        vocabulary_size=tokenizer.vocab_size,
        maximum_context=len(tokens[0]),
        model_width=16,
        head_count=4,
        block_count=1,
        feed_forward_width=32,
    )
    with DecoderOnlyTransformer(
        config,
        attention="flash",
        activation_checkpointing="block",
    ).to("cpu") as model:
        with model.parameters() as parameters:
            with Adam(parameters) as optimizer:
                with cross_entropy(model(tokens), targets) as loss:
                    loss_value = loss.item()
                    loss.backward()
                    stats = optimizer.step()
                print(loss_value, stats.gradient_norm)
```

Change `.to("cpu")` to `.to("metal")` on systems with the Metal backend,
`.to("cuda")` in a CUDA-enabled source build, or `.to("tpu")` in a TPU-enabled
Cloud TPU build. Computation graphs are single-use: build a fresh forward/loss
graph for each training step.
`attention="flash"` selects the dependency-free exact memory-linear
full-sequence forward/backward implementation; omit it to keep the
`"materialized"` default. The Flash path saves `[batch, heads, time]` row
maxima and exponential sums and reconstructs probabilities during backward,
rather than saving `[batch, heads, time, time]` probabilities. The explicit
probability-returning diagnostic remains materialized. This selector does not
change incremental serving, and no speedup is assumed without measuring the
target workload.

`activation_checkpointing="block"` retains only transformer-block boundaries
and replays each block during backward. Omit it for the `"disabled"` default.
This reduces retained activation graph state at the cost of another block
forward calculation during backward. It composes with FlashAttention and
LoRA, but does not affect model artifacts or incremental decode.

On Metal, Flash working storage is proportional to the per-head width and
must fit the device's threadgroup-memory limit. The native runtime preflights
the complete forward/backward path before starting the forward pass. If the
device rejects a very wide head, use more heads or select
`attention="materialized"`.

## Hugging Face data and rank experiments

The `riftco_transformer.data` package is also dependency-free. Its default
transport uses `urllib` to read bounded pages from the official Hugging Face
Dataset Viewer API, while adapters convert TinyStories, Dolly 15K, and
HH-RLHF into stage-specific files. Preparation removes exact duplicates,
assigns records to deterministic content-hash splits, and writes an atomic
directory with a provenance manifest and SHA-256 file digests.

From the framework directory:

```bash
PYTHONPATH="$PWD/python" \
python3 examples/python/prepare_huggingface_data.py \
  --preset dolly \
  --output data/external/huggingface/dolly-lora-v1 \
  --limit 2000 \
  --seed lora-v1
```

`HF_TOKEN` is an optional environment variable, never a CLI argument.
Prepared downloads under `data/external/` are ignored by Git. The Dolly
adapter maps `instruction` plus optional `context` into `prompt`, preserves
`response`, and retains `category`. TinyStories becomes plain text. HH-RLHF
remains chosen/rejected preference data and is not accepted by the current SFT
pipeline.

The `riftco_transformer.experiments` package compares LoRA ranks from the same
immutable base:

```python
from riftco_transformer.artifacts import ModelBundle
from riftco_transformer.experiments import (
    LoraRankExperimentConfig,
    compare_lora_ranks,
    load_prepared_instruction_splits,
)

base = ModelBundle.load("results/stages/tinystories_pretrained.rift")
splits = load_prepared_instruction_splits(
    "data/external/huggingface/dolly-lora-v1"
)
comparison = compare_lora_ranks(
    base,
    splits,
    LoraRankExperimentConfig(
        ranks=(1, 2, 4, 8),
        alpha_over_rank=2.0,
        steps=20,
        backend="cpu",
    ),
)
print(comparison.best_rank, comparison.selected_test.loss)
```

Every rank shares data fingerprints, seeds, sampler, optimizer controls,
LoRA targets, and `alpha / rank`. Validation selects the winner; held-out test
evaluation begins only after selection. The objective is still
full-sequence causal SFT, not response-only loss. All adapters are merged
before persistence, so ranks have the same serving topology and inference
timings are only smoke measurements. The CLI atomically publishes a new,
complete output directory and embeds the verified prepared-data manifest plus
its SHA-256 in `comparison.json`.

`compare_fine_tuning()` generalizes the same held-out protocol to fixed full
fine-tuning recipes and LoRA rank groups. It exhaustively evaluates train and
validation to calculate a comparable generalization gap, then evaluates test
only for the fixed full recipe and validation-selected LoRA rank. The example
CLI is:

```bash
python3 examples/python/compare_fine_tuning.py \
  --base results/stages/tinystories_pretrained.rift \
  --data data/external/huggingface/dolly-lora-v1 \
  --output results/experiments/full-vs-lora \
  --full-learning-rate 0.001 \
  --lora-learning-rate 0.005 \
  --backend cpu
```

The resulting test split is consumed for a final method comparison and must be
retired before further tuning.

See
`docs/DATASETS_AND_LORA_EXPERIMENTS.md` in the framework repository for
license links, exact TinyStories train/validation commands, sample-size
guidance, provenance details, and the CLI rank workflow.

## Staged pipeline

The high-level modules make the stage boundaries explicit:

```python
from riftco_transformer.artifacts import ModelBundle
from riftco_transformer import LoraConfig
from riftco_transformer.post_training import (
    PostTrainingConfig,
    post_train_jsonl,
)
from riftco_transformer.pretraining import PretrainingConfig, pretrain_file
from riftco_transformer.serving import ServingConfig, serve_model

base = pretrain_file(
    "data/pretraining/tiny_corpus.txt",
    PretrainingConfig(
        steps=20,
        backend="cpu",
        attention="flash",
        activation_checkpointing="block",
    ),
)
base.bundle.save("results/stages/tiny_pretrained.rift")

restored = ModelBundle.load("results/stages/tiny_pretrained.rift")
assistant = post_train_jsonl(
    restored,
    "data/post_training/tiny_instructions.jsonl",
    PostTrainingConfig(
        steps=10,
        backend="cpu",
        attention="flash",
        activation_checkpointing="block",
        fine_tuning_method="lora",
        lora=LoraConfig(rank=4, alpha=8.0),
    ),
)
assistant.bundle.save("results/stages/tiny_post_trained.rift")

serve_model(
    "results/stages/tiny_post_trained.rift",
    host="127.0.0.1",
    port=8000,
    config=ServingConfig(
        backend="cpu",
        kv_cache="paged",
        kv_cache_block_size=16,
    ),
)
```

The HTTP adapter serves a dependency-free browser chat at `/` and keeps
`POST /v1/generate` as the stable JSON generation endpoint. Each chat message
is formatted with `PlainChatFormatter` as one independent single-turn SFT
prompt; the visual transcript is not added to the model context. Custom
formatter templates are not persisted in the current artifact format.
`GET /health` reports the selected backend, context and vocabulary sizes, and
active KV-cache strategy.

`ModelBundle` persists the exact byte/BPE tokenizer definition, model
configuration, named parameter shapes, float32 weights, checksums, stage
metadata, and parent artifact ID. It is an immutable inference or warm-start
artifact, not a resumable training checkpoint: Adam moments, optimizer step,
data position, and random-generator state are deliberately not included yet.
LoRA post-training optimizes only adapter factors, then merges them before
capturing this ordinary serving-ready bundle. Full-parameter post-training
remains the default; adapter-only persistence is not part of `ModelBundle`.

The first post-training objective is explicitly
`full_sequence_causal_sft`: it applies causal cross-entropy to the complete
formatted prompt/response sequence. Response-only loss masking is a future
extension.

## Incremental generation

Native models use the current ABI 2.2 `DecodeSession` surface instead of
rerunning the full-sequence training forward for every generated token.
`TextGenerator` creates a request-local session, prefills the
prompt one token at a time, and then performs one-token decode:

```python
from riftco_transformer.artifacts import ModelBundle
from riftco_transformer.serving import TextGenerator

bundle = ModelBundle.load("results/stages/tiny_post_trained.rift")
with bundle.instantiate("cpu") as runtime:
    result = TextGenerator(
        runtime.model,
        runtime.tokenizer,
        kv_cache="paged",
        kv_cache_block_size=16,
    ).generate("Tensor:", max_new_tokens=32)
    print(result.text)
```

Paged caching is the default; use `kv_cache="contiguous"` for the reference
strategy. CPU, Metal, CUDA, and TPU have backend-owned paged-decode
implementations; TPU stages the request through PJRT from host-mirrored
storage. When the learned
absolute-position context fills, `TextGenerator`
resets the cache and replays the retained suffix from position zero. A raw
session exposes the lower-level step/reset contract:

```python
with bundle.instantiate("cpu") as runtime:
    with runtime.model.decode_session(
        cache="paged",
        block_size=16,
    ) as session:
        for token in runtime.tokenizer.encode("Tensor:"):
            logits = session.step(token)
```

The raw session does not tokenize, sample, or implement rollover. Paged
storage also does not yet imply continuous batching, a request scheduler, or
prefix sharing. Full-sequence `attention="flash"` does not alter this path:
serving prefill is still token-at-a-time and decode remains paged.

See `docs/PIPELINE.md`, `docs/SERVING.md`, `docs/TOKENIZATION.md`, and
`docs/BACKENDS_AND_PYTHON.md` in the framework repository for the complete
workflow, lifecycle, backend, and error-handling contracts.

## Package layout

The physical package mirrors the runtime boundaries:

```text
riftco_transformer/
├── native/          # stable C ABI bindings
├── artifacts/       # ModelBundle persistence
├── data/            # external dataset adapters and preparation
├── experiments/     # controlled full/LoRA and rank comparisons
├── training/        # shared batches, metrics, and trainer
├── pretraining/     # next-token pretraining stage
├── post_training/   # supervised continuation stage
└── serving/         # generation, model service, and HTTP
```

The package root re-exports the public low-level API. The breaking rename
installs only `riftco_transformer`; no legacy package-name alias is provided.

## License

Copyright 2026 Quang T Nguyen. Licensed under the Apache License 2.0. The full
license text is included in the source distribution and every wheel.
