Metadata-Version: 2.4
Name: hfit
Version: 0.1.0
Summary: Check whether a Hugging Face model fits your local hardware — without downloading the weights.
Author-email: Samir Nuri <samirnuri714@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/snuri00/hfit
Project-URL: Issues, https://github.com/snuri00/hfit/issues
Keywords: huggingface,llm,vram,gpu,memory,tui,hardware,inference
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Hardware
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: huggingface_hub>=0.23
Requires-Dist: rich>=13.0
Requires-Dist: textual>=0.60
Provides-Extra: full
Requires-Dist: psutil>=5.9; extra == "full"
Dynamic: license-file

# hfit

*Hugging Face + fit.*

**Will this Hugging Face model run on my machine?** Find out in seconds without downloading a single weight file.

Give it a model id like `Qwen/Qwen2.5-7B-Instruct`. It reads the exact
parameter count and architecture from the Hub's metadata (a few KB of API
calls), detects your local hardware (GPU VRAM, RAM), and tells you which
precisions fit:

```
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Precision           ┃  Weights ┃ KV cache ┃ Required* ┃ Verdict               ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━┩
│ FP16 / BF16 (native)│ 14.19 GB │  0.22 GB │  15.91 GB │ ◐ CPU/RAM only (slow) │
│ INT8 (quantized)    │  7.09 GB │  0.22 GB │   8.47 GB │ ◐ CPU/RAM only (slow) │
│ INT4 (quantized)    │  3.55 GB │  0.22 GB │   4.74 GB │ ● tight on GPU        │
└─────────────────────┴──────────┴──────────┴───────────┴───────────────────────┘
```

Unlike `accelerate estimate-memory`, this compares the model against **your
actual hardware** and includes a **KV-cache estimate** at your chosen context
length computed exactly from the model's architecture (layers × KV heads ×
head dim), not a flat percentage.

## Install

```bash
pip install hfit
# or from a clone of this repo:
pip install .

```

Dependencies: `huggingface_hub`, `rich`, `textual`. No torch, no CUDA needed.

## Usage

```bash
hfit                                  # interactive TUI
hfit Qwen/Qwen2.5-7B-Instruct         # one-shot report
hfit meta-llama/Llama-3.1-8B-Instruct --ctx 32768
hfit some/gated-model --token hf_xxx
```

In the TUI: type a model id, press Enter; switch the context length from the
dropdown to see the KV cache impact instantly (no refetch). `q` quits.

Exit codes for scripting: `0` = fits somewhere (GPU or RAM), `1` = lookup
error, `3` = does not fit at all.

## What it understands

- **Safetensors repos** — exact parameter count from Hub metadata.
- **GGUF repos** (e.g. `bartowski/...-GGUF`) every quant file assessed
  individually, for llama.cpp / Ollama / LM Studio users.
- **Multi-component repos** (speech/vision pipelines with several weight
  folders) reported as a stored total instead of a fake parameter count.
- Duplicate serializations (safetensors + bin + h5, sharded + consolidated)
  are deduplicated, not double-counted.

## Platform support

| Platform | GPU detection | RAM detection |
|---|---|---|
| Linux | `nvidia-smi`; AMD via sysfs (no ROCm needed), then `rocm-smi` | `/proc/meminfo` |
| Windows | `nvidia-smi` (PATH or System32) | WinAPI via ctypes |
| macOS (Apple Silicon) | unified memory (RAM = VRAM budget) | `sysctl` |

`psutil` is used when available (`pip install .[full]`) but is not required.

## How the estimate works

```
required = weights + KV cache(context) + overhead(0.8 GB + 5% of weights)
```

- Weights: parameters × bytes-per-parameter (4 / 2 / 1 / 0.5 for
  FP32 / FP16 / INT8 / INT4).
- KV cache: `2 × layers × kv_heads × head_dim × 2 bytes × context_tokens`
  from `config.json`; a 20% margin is used when the architecture is unknown.
- Verdicts leave ~7% VRAM and ~15% RAM headroom for the driver and OS.

These are pre-flight estimates. For a definitive runtime answer, vLLM's
`--dry-run` on the actual machine remains the ground truth.
