Metadata-Version: 2.4
Name: unsloth-candle
Version: 2026.4.1
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Dist: torch>=2.4.0
Requires-Dist: transformers>=4.44.0
Requires-Dist: huggingface-hub>=0.23.0
Requires-Dist: safetensors>=0.4.0
Requires-Dist: llama-cpp-python>=0.3.1
Requires-Dist: patchelf>=0.17.2.4
Requires-Dist: trl>=0.12.0
Requires-Dist: unsloth-candle[vision,audio] ; extra == 'all'
Requires-Dist: librosa ; extra == 'audio'
Requires-Dist: soundfile ; extra == 'audio'
Requires-Dist: pillow ; extra == 'vision'
Requires-Dist: torchvision ; extra == 'vision'
Provides-Extra: all
Provides-Extra: audio
Provides-Extra: vision
License-File: LICENSE
Summary: High-performance LLM fine-tuning library built with Candle and Rust.
Keywords: llm,finetuning,lora,qlora,candle,transformers,rust
Author-email: Unsloth Communtiy <mashriram@gmail.com>
License: Apache-2.0
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/unslothai/unsloth-candle
Project-URL: Issues, https://github.com/unslothai/unsloth-candle/issues
Project-URL: Repository, https://github.com/unslothai/unsloth-candle

# 🚀 Unsloth-Candle

<div align="center">

**High-performance LLM fine-tuning library built with Rust 🦀 and Candle.**

[![PyPI version](https://img.shields.io/pypi/v/unsloth-candle.svg)](https://pypi.org/project/unsloth-candle/)
[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE)
[![Unsloth](https://img.shields.io/badge/Inspiration-Unsloth-orange.svg)](https://github.com/unslothai/unsloth)

</div>

---

**Unsloth-Candle** brings the blazing fast performance of [Unsloth](https://unsloth.ai) to the [Candle](https://github.com/huggingface/candle) ecosystem. By leveraging optimized Rust kernels and efficient memory management, it enables 2x faster training and 70% less memory usage compared to standard implementations.

## ✨ Core Advantages

*   **Zero Learning Curve**: 1:1 API compatibility with Unsloth's Python interface.
*   **Hardware Optimized**: Native support for **CUDA**, **Metal** (Apple Silicon), and **AVX/Neon** (CPU).
*   **Memory Efficient**: Native **4-bit NF4** quantization and gradient checkpointing.
*   **Unified Support**: One engine for Llama 3.2, Mistral, Qwen 2.5, DeepSeek-V3, and more.

## 📦 Installation

### Via Pip (Recommended)

```bash
pip install unsloth-candle
```

### Build from Source

```bash
git clone https://github.com/unslothai/unsloth-candle.git
cd unsloth-candle
pip install -e .
```

To enable GPU acceleration:
*   **CUDA**: `pip install -e . --features cuda`
*   **Metal**: `pip install -e . --features metal`

## 🛠 Usage

### 1. Load Model & Tokenizer
```python
from unsloth_candle import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/Llama-3.2-1B-Instruct",
    max_seq_length = 2048,
    load_in_4bit = True,
)
```

### 2. Apply LoRA/DoRA
```python
model = FastLanguageModel.get_peft_model(
    model,
    r = 16,
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_alpha = 16,
    use_gradient_checkpointing = True,
    use_dora = False, # Set to True for DoRA
)
```

### 3. Fine-tuning with SFTTrainer
```python
from unsloth_candle import SFTTrainer, SFTConfig

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    dataset_text_field = "text",
    max_seq_length = 2048,
    args = SFTConfig(
        max_steps = 60,
        learning_rate = 2e-4,
        logging_steps = 1,
    ),
)
trainer.train()
```

### 4. Save & Export
```python
# Save as merged HF weights
model.save_pretrained_merged("output_hf", tokenizer)

# Save as GGUF (for Ollama/llama.cpp)
model.save_pretrained_gguf("output_gguf", tokenizer, quantization_type="q4_k_m")
```

## 🗺️ Model Catalog

| Model | Architecture | 4-bit | LoRA | DoRA |
| :--- | :--- | :---: | :---: | :---: |
| **Llama 3.2** | LlamaForCausalLM | ✅ | ✅ | ✅ |
| **Mistral Nemo** | MistralForCausalLM | ✅ | ✅ | ✅ |
| **Qwen 2.5** | Qwen2ForCausalLM | ✅ | ✅ | ✅ |
| **DeepSeek V3** | DeepSeekV3 (MLA) | ✅ | ✅ | ✅ |
| **Gemma 3** | Gemma3 (GeGLU) | ✅ | ✅ | ✅ |
| **Phi 4** | Phi4 | ✅ | ✅ | ✅ |

## 📜 License

Licensed under the [Apache License, Version 2.0](LICENSE).

---
<div align="center">
Built with 💖 by the Unsloth Community and Antigravity.
</div>

