Metadata-Version: 2.4
Name: gh-ai-runner
Version: 0.1.5
Summary: Serverless AI inference via GitHub Actions — no server required
Author: Tanish Chauhan
License: MIT
Project-URL: Homepage, https://github.com/TanishC4444/servai
Project-URL: Repository, https://github.com/TanishC4444/servai
Keywords: ai,llm,inference,github-actions,llama,serverless
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.28
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Dynamic: license-file

# github-ai

[![PyPI version](https://badge.fury.io/py/github-ai.svg)](https://badge.fury.io/py/github-ai)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![Python 3.9+](https://img.shields.io/badge/python-3.9+-blue.svg)](https://www.python.org/downloads/)
[![GitHub Actions](https://img.shields.io/badge/powered%20by-GitHub%20Actions-2088FF?logo=github-actions&logoColor=white)](https://github.com/features/actions)

**Run open-source LLMs for free using GitHub Actions as your compute layer.**

No server. No GPU. No cloud account. No API keys. Just a GitHub token and a prompt.

Built by [Tanish Chauhan](https://github.com/TanishC4444)

---

## How It Works

When you call `ai_call()`, the library does the following automatically:

```
Step 1  Creates a repo on your GitHub account (once, on first call only)
Step 2  Waits for the repo to become fully accessible before proceeding
Step 3  Commits an inference script and workflow file into it
Step 4  Dispatches a GitHub Actions workflow run with your prompt
Step 5  A free Ubuntu runner picks up the job and downloads the model
Step 6  The model runs inference via llama-cpp-python
Step 7  The output is uploaded as a GitHub Actions artifact
Step 8  The library downloads the artifact and returns it to you as a string
```

The runner spins up fresh for each call and shuts down immediately after. Nothing runs between calls — there is no server, no idle cost, and no infrastructure to maintain.

---

## Install

```bash
pip install github-ai
```

Requirements: Python 3.9+, a GitHub account.

---

## Getting Your GitHub Token

You need a **Personal Access Token (PAT)** with repo and workflow permissions.

### Step-by-step

1. Go to [github.com](https://github.com) and sign in
2. Click your **profile picture** (top right) → **Settings**
3. Scroll to the bottom of the left sidebar → click **Developer settings**
4. Click **Personal access tokens** → **Tokens (classic)**
5. Click **Generate new token** → **Generate new token (classic)**
6. Give it a name, e.g. `github-ai`
7. Set an expiration (90 days recommended)
8. Check exactly these two scopes:
   - ✅ `repo` — full control of repositories
   - ✅ `workflow` — update GitHub Actions workflows
9. Scroll to the bottom → click **Generate token**
10. **Copy the token immediately** — GitHub only shows it once

Your token starts with `ghp_`. Treat it like a password.

### Store it as an environment variable (recommended)

```bash
# Mac / Linux — add to ~/.zshrc or ~/.bashrc
export GITHUB_TOKEN="ghp_your_token_here"

# Windows (PowerShell)
$env:GITHUB_TOKEN = "ghp_your_token_here"
```

```python
import os
from github_ai import ai_call

result = ai_call(
    github_token=os.environ["GITHUB_TOKEN"],
    prompt="What is a transformer model?",
)
```

---

## Quickstart

```python
from github_ai import ai_call

result = ai_call(
    github_token="ghp_...",
    prompt="Explain recursion in simple terms.",
)

print(result)
```

On first run, the library will:
- Create a repo called `ai-inference-runner` on your account
- Wait for the repo to become fully accessible
- Set up the inference workflow automatically
- Download and cache the model (~0.6 GB, takes ~5 min)

Every run after that uses the cached repo and model and finishes in **1–2 minutes**.

---

## Examples

### Simple question

```python
from github_ai import ai_call

result = ai_call(
    github_token="ghp_...",
    prompt="What is the difference between TCP and UDP?",
)
print(result)
```

### Custom system prompt

```python
result = ai_call(
    github_token="ghp_...",
    prompt="What are three ways to improve this API design?",
    system="You are a senior backend engineer. Be direct and technical.",
)
print(result)
```

### Code generation

```python
result = ai_call(
    github_token="ghp_...",
    prompt="Write a Python function that flattens a nested list of any depth.",
    system="You are an expert Python developer. Return only code, no explanation.",
    temperature=0.2,
    max_tokens=512,
)
print(result)
```

### Deterministic output

```python
result = ai_call(
    github_token="ghp_...",
    prompt="What is 144 divided by 12?",
    temperature=0.0,   # fully deterministic — same answer every time
    max_tokens=16,
)
print(result)
```

### Using the Llama model

```python
result = ai_call(
    github_token="ghp_...",
    prompt="Summarize the causes of World War I.",
    model="llama",       # better instruction following and reasoning
    max_tokens=1024,
    temperature=0.5,
)
print(result)
```

### Silent mode

```python
result = ai_call(
    github_token="ghp_...",
    prompt="Translate 'good morning' to French, Spanish, and Japanese.",
    verbose=False,   # no logs printed, just the result
)
print(result)
```

### Multiple calls in sequence

```python
from github_ai import ai_call

TOKEN = "ghp_..."

questions = [
    "What is a neural network?",
    "What is gradient descent?",
    "What is backpropagation?",
]

for q in questions:
    answer = ai_call(github_token=TOKEN, prompt=q, verbose=False)
    print(f"Q: {q}\nA: {answer}\n")
```

### Parallel calls

Parallel calls must use separate repos — each repo has its own independent run queue so calls never interfere with each other.

```python
import threading
from github_ai import ai_call

TOKEN = "ghp_..."
results = {}

def run(key, prompt, repo):
    results[key] = ai_call(
        github_token=TOKEN,
        prompt=prompt,
        repo_name=repo,
        verbose=False,
    )

t1 = threading.Thread(target=run, args=("q1", "Explain DNA", "runner-repo-1"))
t2 = threading.Thread(target=run, args=("q2", "Explain RNA", "runner-repo-2"))

t1.start(); t2.start()
t1.join();  t2.join()

print(results["q1"])
print(results["q2"])
```

### Extended context window

```python
result = ai_call(
    github_token="ghp_...",
    prompt="Write a detailed essay on the history of the internet.",
    model="llama",
    max_tokens=2048,
    n_ctx=4096,
    temperature=0.6,
)
print(result)
```

---

## Parameters

| Parameter | Type | Default | Required | Description |
|---|---|---|---|---|
| `github_token` | `str` | — | ✅ | GitHub PAT with `repo` and `workflow` scopes |
| `prompt` | `str` | — | ✅ | The message or question to send to the model |
| `model` | `str` | `"tinyllama"` | — | Which model to use — see Models section |
| `system` | `str` | `"You are a helpful assistant."` | — | System prompt that controls model behavior |
| `max_tokens` | `int` | `512` | — | Max tokens to generate. Hard limit: 4096 |
| `temperature` | `float` | `0.7` | — | `0.0` = deterministic, `2.0` = highly creative |
| `cache` | `bool` | `True` | — | Cache model weights between runs. Strongly recommended |
| `n_ctx` | `int` | `None` | — | Context window size. Defaults to model built-in. Max: 8192 |
| `repo_name` | `str` | `"ai-inference-runner"` | — | GitHub repo to create or reuse for running inference |
| `verbose` | `bool` | `True` | — | Print step-by-step logs. Set `False` for silent mode |

---

## Models

| Key | Model | Size | Default Context | Max Context | Best For |
|---|---|---|---|---|---|
| `tinyllama` | TinyLlama 1.1B Chat Q4_K_M | 0.6 GB | 2048 tokens | 8192 tokens | Fast answers, simple tasks |
| `llama` | Llama 3.2 1B Instruct Q4_K_M | 0.7 GB | 4096 tokens | 8192 tokens | Reasoning, longer outputs |

Both models are quantized GGUF files hosted publicly on HuggingFace. No HuggingFace token required.

**Which model should I use?**
Use `tinyllama` for short factual answers and anything where speed matters. Use `llama` for structured outputs, reasoning tasks, code generation, and longer responses.

---

## Timing

| Scenario | Approximate Time |
|---|---|
| First run, no cache | ~5 min (compiles llama-cpp-python + downloads model) |
| Cached run | ~1–2 min |
| Inference only (warm runner) | ~10–40 sec depending on output length |

The first-run delay comes from compiling `llama-cpp-python` from source on the runner. This is a one-time cost per repo. All subsequent calls reuse the cached venv and model weights.

---

## GitHub Actions Free Tier

| Resource | Free Tier |
|---|---|
| Minutes (public repos) | **Unlimited** |
| Minutes (private repos) | 2,000 / month |
| Runner RAM | 7 GB |
| Runner CPUs | 2 vCPUs |
| Cache storage | 10 GB per repo |
| Concurrent jobs | Up to 20 |

Keep your runner repo public and usage is entirely free with no monthly cap.

---

## Validation and Limits

The library validates all parameters before dispatching to prevent wasted runner time:

- `temperature` must be between `0.0` and `2.0`
- `max_tokens` must be between `1` and `4096`
- `n_ctx` must not exceed `8192` (OOM risk above this on the runner)
- `max_tokens` must be less than `n_ctx`
- Estimated RAM usage is checked against the 7 GB runner limit

If any check fails, a `ValueError` is raised locally before anything is sent to GitHub.

---

## Troubleshooting

**`403 Forbidden` when dispatching workflow**
Your token is missing the `workflow` scope. Edit the token on GitHub, check the `workflow` box, and regenerate.

**`Timed out waiting for workflow run to appear`**
The runner queue was busy. Retry — GitHub Actions can delay by a minute or two during peak times.

**Output is cut off mid-sentence**
Increase `max_tokens`. Default is 512. For longer responses use 1024 or 2048.

**`Workflow failed (failure)`**
Check the Actions tab on your runner repo for the full log. Most common cause is a HuggingFace rate limit on model download — retry with `cache=True` (default) so the model only downloads once.

**`model must be one of: ['tinyllama', 'llama']`**
The `model` parameter only accepts those two exact strings. Check for typos.

**VS Code doesn't recognize the import after pip install**
Interpreter mismatch. Press `Ctrl+Shift+P` → `Python: Select Interpreter` and select the environment where you ran `pip install github-ai`. Run `pip show github-ai` in your terminal to confirm where it installed.

---

## Changelog

### 0.1.5
- Added repo accessibility wait on first-time creation to prevent race condition on new accounts
- Added automatic concise response enforcement via system prompt addendum
- Improved validation error messages
- Full README rewrite with complete examples and token setup guide

### 0.1.1
- Initial public release

---

## License

MIT — see [LICENSE](LICENSE) for details.

---

<p align="center">
  Built by <a href="https://github.com/TanishC4444">Tanish Chauhan</a>
</p>
