Metadata-Version: 2.4
Name: LLMmodelHub
Version: 0.2.0
Summary: Turn any machine or Google Colab notebook into a one-line AI model server with an auto-generated public API.
Author: ABS-EMON
License: MIT
Project-URL: Homepage, https://github.com/ABS-EMON/LLMmodelHub
Project-URL: Repository, https://github.com/ABS-EMON/LLMmodelHub
Project-URL: Issues, https://github.com/ABS-EMON/LLMmodelHub/issues
Keywords: llm,vlm,ai,huggingface,gguf,flask,api,colab,model-server,ngrok,machine-learning
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: flask>=2.3
Requires-Dist: huggingface_hub>=0.20
Requires-Dist: requests>=2.28
Requires-Dist: pyngrok>=7.0
Requires-Dist: llama-cpp-python>=0.2.60
Provides-Extra: llm
Requires-Dist: transformers>=4.38; extra == "llm"
Requires-Dist: torch>=2.0; extra == "llm"
Requires-Dist: accelerate>=0.27; extra == "llm"
Provides-Extra: api
Requires-Dist: flask-cors>=4.0; extra == "api"
Requires-Dist: flask-limiter>=3.5; extra == "api"
Provides-Extra: colab
Requires-Dist: ipywidgets>=8.0; extra == "colab"
Provides-Extra: vision
Requires-Dist: pillow>=10.0; extra == "vision"
Provides-Extra: full
Requires-Dist: transformers>=4.38; extra == "full"
Requires-Dist: torch>=2.0; extra == "full"
Requires-Dist: accelerate>=0.27; extra == "full"
Requires-Dist: flask-cors>=4.0; extra == "full"
Requires-Dist: flask-limiter>=3.5; extra == "full"
Requires-Dist: ipywidgets>=8.0; extra == "full"
Requires-Dist: pillow>=10.0; extra == "full"
Dynamic: license-file

# LLMmodelHub

Turn any local machine or Google Colab notebook into a production-ready AI
model server with a single line of code.

```bash
pip install LLMmodelHub
```

```python
from LLMmodelHub import load_model

hub = load_model("TheBloke/Llama-2-7B-Chat-GGUF")

print(hub.local_url)    # http://0.0.0.0:5000
print(hub.public_url)   # https://xxxx.trycloudflare.com -- free, zero setup

hub.chat()               # interactive terminal chat (or a widget UI in Colab)
```

## Features

- **One-line model loading** — pulls models straight from Hugging Face,
  including quantized **GGUF** files for consumer hardware.
- **LLM + VLM support** — text models and vision-language models behind one API.
- **Automatic REST API** — an OpenAI-compatible `/v1/chat/completions`
  endpoint is spun up the moment the model loads.
- **Public tunneling, zero setup** — by default, a free `cloudflared` quick
  tunnel is used (the binary is auto-downloaded on first run; no account
  needed). If you've already configured a free ngrok authtoken, ngrok is
  used instead. (Note: ngrok ended anonymous/no-signup tunnels, so it now
  requires `from pyngrok import ngrok; ngrok.set_auth_token("...")` — set
  that up once at https://dashboard.ngrok.com/signup if you prefer ngrok's
  stable domains.)
- **Terminal chat UI** — color-coded, with `/image`, `/clear`, `/url` commands.
- **Colab widget UI** — chat box, image upload, and URL display inside the notebook.
- **Caching** — downloaded models are cached locally and reused.
- **Optional security** — API key auth, CORS, and rate limiting.

## Installation

```bash
pip install LLMmodelHub                 # core: API + downloader + GGUF support (llama-cpp-python)
pip install "LLMmodelHub[llm]"          # + transformers/torch for full (non-GGUF) HF models
pip install "LLMmodelHub[colab]"        # + ipywidgets for the notebook UI
pip install "LLMmodelHub[vision]"       # + pillow for VLM image input
pip install "LLMmodelHub[full]"         # everything
```

> **Note:** `llama-cpp-python` is a core dependency (needed to run GGUF
> models) and compiles native code on install. On Linux/Colab this
> usually just works via a prebuilt wheel. On some platforms (older
> Windows Python versions, some Macs) pip may need to compile it from
> source, which requires a C++ compiler (e.g. `xcode-select --install`
> on macOS, or Visual Studio Build Tools on Windows) and can take a
> few minutes the first time. For GPU acceleration, install with:
> `CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python --force-reinstall --no-cache-dir`
> (after `pip install LLMmodelHub`) to rebuild it with CUDA support.

## Usage

### Load a full Hugging Face model

```python
from LLMmodelHub import load_model

hub = load_model("meta-llama/Llama-2-7b-chat-hf", model_type="llm")
reply = hub.generate("Explain quantum computing in one sentence.")
print(reply)
```

### Load a quantized GGUF model

```python
hub = load_model("TheBloke/Llama-2-7B-Chat-GGUF", filename="llama-2-7b-chat.Q4_K_M.gguf")
```

### Load a vision-language model

```python
hub = load_model("Salesforce/blip2-opt-2.7b", model_type="vlm")
reply = hub.generate("What is in this image?", image_path="cat.jpg")
```

### Call the REST API from anywhere

```bash
curl -X POST "$PUBLIC_URL/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{"messages": [{"role": "user", "content": "Hello!"}]}'
```

### Stop the server

```python
hub.stop()
```

## License

MIT
