Metadata-Version: 2.4
Name: tinyport
Version: 0.1.0
Summary: Run big AI models on small hardware. Integer Descent guided model compression.
Home-page: https://github.com/tinyport-ai/tinyport
Author: TinyPort
License: MIT
Project-URL: Homepage, https://tinyport.ai
Project-URL: Repository, https://github.com/tinyport-ai/tinyport
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Requires-Dist: numpy>=1.24
Provides-Extra: torch
Requires-Dist: torch>=2.0; extra == "torch"
Provides-Extra: hub
Requires-Dist: huggingface_hub>=0.20; extra == "hub"
Requires-Dist: safetensors>=0.4; extra == "hub"
Provides-Extra: full
Requires-Dist: torch>=2.0; extra == "full"
Requires-Dist: huggingface_hub>=0.20; extra == "full"
Requires-Dist: safetensors>=0.4; extra == "full"
Dynamic: author
Dynamic: home-page
Dynamic: requires-python

# TinyPort

**Run big AI models on small hardware.**

TinyPort compresses AI models from HuggingFace or local files so they can run on phones, Raspberry Pi, laptops — any device with a processor.

It uses **Integer Descent** — an information-theoretic framework that measures how much real information each layer of a neural network is actually carrying. Layers with low information density (high "carry pressure") get compressed aggressively. Information-dense layers keep their precision. The result: smaller models that behave like larger ones.

## Install

```bash
pip install tinyport
```

With HuggingFace support:
```bash
pip install tinyport[full]
```

## Quickstart

### Python

```python
from tinyport import load_model, compress

# Load from HuggingFace Hub
weights = load_model("openai/whisper-tiny")

# Compress for Raspberry Pi 5
compressed = compress(weights, target="rpi5")

print(compressed.summary())
# Model:       openai/whisper-tiny
# Original:    151.0 MB (FP32)
# Compressed:  37.8 MB (4.0x smaller)
# Quality:     Good (INT4 quality, ~3-5% accuracy impact with ID-guidance)
```

### CLI

```bash
# Analyze a model
tinyport analyze --model openai/whisper-tiny

# Compress for a phone
tinyport compress --model openai/whisper-tiny --target phone

# List all hardware targets
tinyport targets
```

## Hardware Targets

| Target | Device | Bits |
|--------|--------|------|
| `server` | Cloud GPU / data center | INT8 |
| `desktop` | Gaming PC with GPU | INT6 |
| `laptop` | MacBook / laptop | INT5 |
| `phone` | Android / iPhone | INT4 |
| `rpi5` | Raspberry Pi 5 | INT4 |
| `rpi4` | Raspberry Pi 4 | INT4 |
| `rpi_zero` | Raspberry Pi Zero 2W | INT4 (tiny models only) |

## How It Works

Standard quantization (like bitsandbytes, GPTQ) applies the same bit width to every layer. TinyPort profiles each layer individually using Integer Descent:

1. **Profile**: Scan every layer, compute its Integer Descent Compressibility Score (IDCS)
2. **Allocate**: High-IDCS layers (lots of free bits) → compress to INT4. Low-IDCS layers (information-dense) → keep INT6-8
3. **Quantize**: Block-wise quantization preserves local weight distribution
4. **Export**: Output a compressed file with deployment instructions

The result: comparable quality to uniform INT8 at INT4 size.

## Running Compressed Models

TinyPort outputs compressed weights. To run them, use these battle-tested runtimes:

- **LLMs on CPU/GPU**: [llama.cpp](https://github.com/ggerganov/llama.cpp)
- **Vision/Audio on mobile**: [TFLite](https://www.tensorflow.org/lite), [MLC-LLM](https://github.com/mlc-ai/mlc-llm)
- **iPhone**: [CoreML](https://developer.apple.com/documentation/coreml)
- **Universal**: [ONNX Runtime](https://onnxruntime.ai/)

## License

MIT
