Metadata-Version: 2.4
Name: miniforge
Version: 0.1.2
Summary: High-performance MiniMax M2.7 inference library optimized for GMKtech M7
Project-URL: Homepage, https://github.com/Zapdev-labs/miniforge
Project-URL: Repository, https://github.com/Zapdev-labs/miniforge
Project-URL: Issues, https://github.com/Zapdev-labs/miniforge/issues
Author: Miniforge User
License: MIT
License-File: LICENSE
Keywords: amd,gguf,inference,llm,minimax,quantization
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: aiohttp>=3.9
Requires-Dist: huggingface-hub>=0.20
Requires-Dist: llama-cpp-python>=0.3.20
Requires-Dist: numpy>=1.24
Requires-Dist: pillow>=10.0
Requires-Dist: psutil>=5.9
Requires-Dist: pydantic>=2.0
Requires-Dist: torch<2.3.0,>=2.2.0; sys_platform == 'darwin' and platform_machine == 'x86_64'
Requires-Dist: torch>=2.2.0; sys_platform != 'darwin' or platform_machine != 'x86_64'
Requires-Dist: tqdm>=4.66
Requires-Dist: transformers>=4.40.0
Requires-Dist: typing-extensions>=4.8
Provides-Extra: all
Requires-Dist: black>=24.0; extra == 'all'
Requires-Dist: fastapi>=0.110.0; extra == 'all'
Requires-Dist: mypy>=1.8; extra == 'all'
Requires-Dist: pre-commit>=3.6; extra == 'all'
Requires-Dist: prometheus-client>=0.20.0; extra == 'all'
Requires-Dist: pytest-asyncio>=0.23; extra == 'all'
Requires-Dist: pytest>=8.0; extra == 'all'
Requires-Dist: ruff>=0.2.0; extra == 'all'
Requires-Dist: starlette>=0.36.0; extra == 'all'
Requires-Dist: uvicorn[standard]>=0.27.0; extra == 'all'
Provides-Extra: dev
Requires-Dist: black>=24.0; extra == 'dev'
Requires-Dist: mypy>=1.8; extra == 'dev'
Requires-Dist: pre-commit>=3.6; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: ruff>=0.2.0; extra == 'dev'
Provides-Extra: llama-cpp
Requires-Dist: llama-cpp-python>=0.3.5; extra == 'llama-cpp'
Provides-Extra: server
Requires-Dist: fastapi>=0.110.0; extra == 'server'
Requires-Dist: prometheus-client>=0.20.0; extra == 'server'
Requires-Dist: starlette>=0.36.0; extra == 'server'
Requires-Dist: uvicorn[standard]>=0.27.0; extra == 'server'
Description-Content-Type: text/markdown

# Miniforge

High-performance Python library for MiniMax M2.7 inference, optimized for GMKtech M7 hardware (AMD Ryzen 7 PRO 6850H, 28GB RAM).

## Features

- **GGUF Quantization**: Q4_K_M recommended for best quality/size tradeoff
- **TurboQuant KV Cache**: 3-bit compression (turbo3) for 5x smaller memory footprint
- **Tool Calling**: Native support for function calling
- **Vision/Multimodal**: Image understanding capabilities
- **Streaming**: Real-time token streaming for responsive UIs
- **Memory Management**: Hard 28GB limits with automatic optimization
- **Async Support**: Full asyncio support throughout
- **Runtime Presets**: Speed, balanced, memory, quality, and MoE tuning profiles
- **Config Doctoring**: Inspect detected hardware and resolved runtime config from the CLI
- **Optimization Reports**: Explain why threads, context, KV cache, mmap, and offload were selected
- **Local Model Hosting**: Register GGUF or HF-style directories and run fully offline

## Quick Start

```bash
# Install with uv
uv pip install -e .

# Or install from source
git clone https://github.com/Zapdev-labs/miniforge.git
cd miniforge
uv pip install -e ".[all]"
```

The default install omits `llama-cpp-python` so editable installs work on Windows without the Visual Studio C++ toolchain. For the `llama_cpp` GGUF backend, run `uv pip install -e ".[llama-cpp]"` (on Windows you need [Build Tools for Visual Studio](https://visualstudio.microsoft.com/visual-cpp-build-tools/) with the C++ workload, or a matching prebuilt wheel). The `[all]` extra is server plus dev tools only and does not pull in `llama-cpp-python`.

```python
import asyncio
from miniforge import Miniforge

async def main():
    # Load model (auto-downloads GGUF if available)
    model = await Miniforge.from_pretrained(
        "MiniMaxAI/MiniMax-M2.7",
        quantization="Q4_K_M",
    )
    
    # Simple chat
    response = await model.chat(
        "Explain quantum computing",
        system_prompt="You are a helpful assistant.",
    )
    print(response)
    
    # Streaming
    stream = await model.chat("Tell me a story", stream=True)
    async for token in stream:
        print(token, end="", flush=True)

asyncio.run(main())
```

## Hardware Requirements

**GMKtech M7 Specs:**
- CPU: AMD Ryzen 7 PRO 6850H (8 cores)
- RAM: 28GB available (4GB to iGPU VRAM)
- OS: Windows 11 + WSL2

**Expected Performance:**
- Prompt processing: 50-100 tok/s
- Generation: 15-25 tok/s (exceeds 10 TPS target)
- Memory usage: ~4-5GB with Q4_K_M + turbo3

## Configuration

Miniforge resolves configuration from three layers:

1. Hardware auto-detection
2. Optional performance preset
3. Environment variables or CLI overrides

Useful environment variables:

```bash
MINIFORGE_MODEL=MiniMaxAI/MiniMax-M2.7
MINIFORGE_BACKEND=llama_cpp
MINIFORGE_QUANTIZATION=Q4_K_M
MINIFORGE_PRESET=balanced
MINIFORGE_OFFLINE=1
MINIFORGE_MODEL_DIRS=/models;/mnt/d/AI
MINIFORGE_MODEL_WEIGHTS_PATH=/models/minimax/MiniMax-M2.7-Q4_K_M.gguf
MINIFORGE_MAX_TOKENS=1024
MINIFORGE_TEMPERATURE=0.7
```

Create `~/.config/miniforge/config.yaml`:

```yaml
max_memory_gb: 24.0
n_ctx: 200000
quantization: Q4_K_M
cache_type_k: turbo3
cache_type_v: turbo3
n_threads: 8
flash_attn: true
```

Or use the provided optimized config:

```python
from miniforge.utils.config import M7Config

config = M7Config.from_yaml("configs/m7-optimized.yaml")
model = await Miniforge.from_pretrained(config=config)
```

### CLI Runtime Inspection

```bash
miniforge doctor
miniforge doctor --preset memory --json
miniforge register local/minimax /models/MiniMax-M2.7-Q4_K_M.gguf --quantization Q4_K_M
miniforge serve --model local/minimax --offline
miniforge chat --preset balanced --system-prompt "You are a concise coding assistant."
```

### Local Model Hosting

Miniforge can now resolve models without contacting Hugging Face:

```bash
miniforge register local/m2 /models/MiniMax-M2.7-Q4_K_M.gguf --quantization Q4_K_M
miniforge chat --model local/m2 --offline
```

Resolution order is: explicit `model_weights_path`, direct filesystem paths, registered hosted models, `MINIFORGE_MODEL_DIRS`, GGUF cache, then network download only when offline mode is disabled.

## Examples

See `examples/` directory:

- `basic_chat.py` - Simple chat interface
- `streaming_chat.py` - Real-time streaming
- `tool_agent.py` - Tool calling with custom functions
- `vision_chat.py` - Image understanding

## Backends

### llama.cpp (Recommended)
Fastest CPU inference with GGUF support:

```python
model = await Miniforge.from_pretrained(
    "MiniMaxAI/MiniMax-M2.7",
    backend="llama_cpp",
    quantization="Q4_K_M",
)
```

If no prebuilt GGUF is on the Hub, Miniforge can convert SafeTensors weights automatically using a local [llama.cpp](https://github.com/ggml-org/llama.cpp) checkout: install its Python requirements, build `llama-quantize` (for Q4_K_M etc.), then set `MINIFORGE_LLAMA_CPP` to the repo root (or `llama_cpp_path` in `M7Config` / YAML). The converter runs `convert_hf_to_gguf.py` and caches the result under your miniforge GGUF cache. If conversion is not configured or fails, the library falls back to the Transformers backend as before.

### Transformers (Fallback)
Native HF support with bitsandbytes:

```python
model = await Miniforge.from_pretrained(
    "MiniMaxAI/MiniMax-M2.7",
    backend="transformers",
)
```

## Memory Optimization

The library automatically manages your 28GB constraint:

```python
from miniforge.core.memory import MemoryManager

# Auto-select best quantization
mem = MemoryManager()
quant = mem.select_quantization(model_params=2.7)
# Returns: Q4_K_M (or Q3_K_M if memory constrained)

# Calculate safe context window
max_ctx = mem.calculate_max_context(
    model_quantized_gb=3.1,
    kv_cache_type="turbo3",
)
# Returns: up to 200000 (or less if memory-constrained)
```

## Tool Calling

```python
from miniforge.generation.tools import Tool

weather_tool = Tool(
    name="get_weather",
    description="Get weather for location",
    parameters={
        "type": "object",
        "properties": {
            "location": {"type": "string"}
        },
        "required": ["location"],
    },
    handler=get_weather_func,
)

response = await model.chat(
    "What's the weather in Paris?",
    tools=[weather_tool],
)
```

## Vision

```python
response = await model.chat_vision(
    message="Describe this image",
    image="path/to/image.jpg",
)
```

## License

MIT
