Metadata-Version: 2.4
Name: arclm
Version: 0.1.1
Summary: A compact PyTorch language-model training and fine-tuning library.
Author: ArcLM Contributors
License-Expression: MIT
Project-URL: Homepage, https://github.com/ahmad-al-dibo/arclm
Project-URL: Documentation, https://github.com/ahmad-al-dibo/arclm#readme
Project-URL: Source, https://github.com/ahmad-al-dibo/arclm
Project-URL: Issues, https://github.com/ahmad-al-dibo/arclm/issues
Keywords: language-model,transformer,pytorch,training,fine-tuning,nlp
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch<3,>=2.1
Requires-Dist: numpy<3,>=1.24
Requires-Dist: sentencepiece<0.3,>=0.2
Requires-Dist: transformers<6,>=4.38
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: build<2,>=1.2; extra == "dev"
Requires-Dist: twine<7,>=5; extra == "dev"
Provides-Extra: web
Requires-Dist: flask<4,>=3; extra == "web"
Dynamic: license-file

# ArcLM

ArcLM is a small PyTorch framework for developers who want a readable language-model training stack without giving up the pieces that matter in real projects: data preparation, tokenizer handling, checkpointing, fine-tuning, and inference.

It is designed for local experiments, compact domain models, teaching, and framework-style workflows where you want to understand and control the training loop.

## Why Use ArcLM

* Train compact causal language models from plain text
* Fine-tune ArcLM checkpoints on new domain data
* Load ArcLM checkpoints, raw PyTorch state dicts, safetensors files, and Hugging Face sources through one loader API
* Save resumable checkpoints by epoch or every N training batches
* Keep tokenizer metadata inside checkpoints for safer reloads
* Use a high-level `train_model()` API or drop down to `Config`, `build_model()`, and `Trainer`
* Run inference with a simple `load_model()` and `predict()` flow

ArcLM is not trying to replace large production training systems. It is a practical framework for building, testing, and adapting small transformer language models with code that stays easy to inspect.

## Installation

```bash
pip install arclm
```

ArcLM requires Python 3.9+ and PyTorch.

## Quick Start

Train a model from text:

```python
from arclm import train_model

result = train_model(
    mode="pretrain",
    data="data/data.txt",
    output="models/arclm.pth",
    tokenizer_type="word",
    max_vocab=2000,
    embed_dim=64,
    num_blocks=2,
    block_size=32,
    batch_size=8,
    num_epochs=3,
    checkpoint_batch_interval=100,
)

print(result.model_path)
```

Generate text from the saved checkpoint:

```python
from arclm import load_model

model = load_model("models/arclm.pth")
print(model.predict("machine learning", max_new_tokens=50, top_p=0.9))
```

## Fine-Tuning

```python
from arclm import train_model

result = train_model(
    mode="finetune",
    checkpoint="models/arclm.pth",
    data="data/domain_text.txt",
    output="models/arclm_domain.pth",
    num_epochs=2,
    learning_rate=2e-5,
    freeze_backbone=True,
    checkpoint_batch_interval=50,
)
```

For native ArcLM checkpoints, tokenizer compatibility is checked before fine-tuning so token IDs are not silently mixed up.

## Lower-Level Control

Use the framework pieces directly when you need custom training logic:

```python
import torch
from arclm import (
    Config,
    build_model,
    build_trainer,
    create_checkpoint_callback,
    prepare_data,
)

config = Config(
    data_path="data/data.txt",
    model_path="models/arclm.pth",
    tokenizer_type="sentencepiece",
    max_vocab=8000,
    embed_dim=128,
    num_blocks=4,
    block_size=128,
    batch_size=32,
    num_epochs=5,
    learning_rate=3e-4,
    validation_split=0.1,
    checkpoint_batch_interval=100,
    device="cuda" if torch.cuda.is_available() else "cpu",
)

data = prepare_data(config)
config.vocab_size = data.vocab_size

model = build_model(config, data.vocab_size)
trainer = build_trainer(model, config)

trainer.train(
    data.train_loader,
    config.num_epochs,
    val_loader=data.val_loader,
    early_stopping_patience=2,
    min_delta=1e-4,
    checkpoint_callback=create_checkpoint_callback(config, data.tokenizer, data.vocab_size),
)
```

## Checkpointing

ArcLM checkpoints include model weights, optimizer state, config, training history, tokenizer mappings, and tokenizer metadata.

Useful settings:

* `checkpoint_batch_interval=100` saves every 100 training batches
* `checkpoint_interval=1` saves every completed epoch
* `early_stopping_patience=2` stops training when validation loss stops improving

Checkpoint writes are atomic: ArcLM writes to a temporary file first and replaces the checkpoint only after the save succeeds.

## External Model Loading

ArcLM can normalize several checkpoint formats before training:

* ArcLM checkpoints saved by `Trainer.save`
* raw PyTorch state dict files: `.pth`, `.pt`, `.bin`, `.ckpt`
* `.safetensors` files when `safetensors` is installed
* Hugging Face model folders or model IDs when `transformers` is installed

```python
from arclm import adapt_for_training, load_external_model

loaded = load_external_model("external/model.pth")
bundle = adapt_for_training(loaded)

print(loaded.source_type)
print(bundle.config)
```

## Project Layout

```text
arclm/      Framework source code
docs/       Usage guides and references
examples/   Example scripts
tests/      Test suite
```

## Documentation

See [docs/USAGE.md](docs/USAGE.md) for a longer guide covering training, loading checkpoints, fine-tuning, and external sources.

## License

MIT
