Metadata-Version: 2.4
Name: npuserver
Version: 1.0.2
Summary: 
Author: Durga Sai
Author-email: dsainvg.20.12.24@kgpian.iitkgp.ac.in
Requires-Python: >=3.12
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Requires-Dist: flask (>=3.0.0)
Requires-Dist: huggingface-hub (>=0.20.0)
Requires-Dist: openvino-genai (>=2024.5.0)
Description-Content-Type: text/markdown

# npuserver 🚀

A lightweight, efficient utility library for compiling and preparing Generative AI LLM models for the **Intel NPU (Neural Processing Unit)** using OpenVINO™ GenAI.

---

## Features

- **Intel NPU Optimization:** Fast, local LLM compilation designed for Intel NPU architectures.
- **Robust Model Fallbacks:** Automated properties configuration with retry logic for prompt lengths.
- **Hugging Face Hub Integration:** Seamless resolution and down-caching of models.
- **Clean API Design:** Import and use directly in any Python environment.

---

## Installation

### From PyPI
```bash
pip install npuserver
```

### From Source
1. Clone the repository:
   ```bash
   git clone https://github.com/yourusername/npuserver.git
   cd npuserver
   ```
2. Set up a virtual environment and install:
   ```bash
   python -m venv .venv
   # On Windows:
   .venv\Scripts\activate
   # On macOS/Linux:
   source .venv/bin/activate

   pip install -e .
   ```

---

## Quick Start

Compile your favorite Hugging Face LLM model for the Intel NPU:

```python
from pathlib import Path
from npuserver import compile_model

# Path to store compiled model caches
cache_dir = Path("./npu_cache")
cache_dir.mkdir(exist_ok=True)

# Compile a Hugging Face LLM (e.g., Qwen or Phi)
compile_model(
    repo_id="Qwen/Qwen2.5-0.5B-Instruct",
    cache_dir=cache_dir,
    prompt_len=8192
)
```

---

## Development

### Running with Poetry
This library uses **Poetry** as its package manager:
```bash
poetry install
poetry run python -c "import npuserver; print(npuserver.__all__)"
```

### Directory Structure
```text
npuserver/
├── .github/workflows/   # CI/CD & Automated Publishing
├── src/
│   └── npuserver/
│       ├── __init__.py  # Package entry point
│       └── compile.py   # Core compilation functions
├── tests/               # Test suites
├── pyproject.toml       # Modern packaging configuration
└── requirements.txt     # Standard pip requirements
```

---

## PyPI Automatic Publishing

The project includes an automated GitHub Actions CI/CD pipeline (`.github/workflows/publish.yml`) that builds and publishes releases securely using **OIDC Trusted Publishing**:

1. Tag your release:
   ```bash
   git tag v0.1.0
   git push origin v0.1.0
   ```
2. The GitHub Action will trigger, build source/wheel distributions, and push to PyPI.

