Metadata-Version: 2.4
Name: linode-llm-toolkit
Version: 0.1.0a3
Summary: Unofficial one-command catalog LLM deployment CLI for Linode
Project-URL: Homepage, https://github.com/w7089/linode-llm-toolkit
Project-URL: Documentation, https://github.com/w7089/linode-llm-toolkit#readme
Project-URL: Issues, https://github.com/w7089/linode-llm-toolkit/issues
Project-URL: Changelog, https://github.com/w7089/linode-llm-toolkit/blob/main/CHANGELOG.md
Author: w7089
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai,akamai,deployment,kubernetes,linode,llm,ollama
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.12
Requires-Dist: click>=8.1
Requires-Dist: httpx>=0.27
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Provides-Extra: dev
Requires-Dist: mypy==2.3.0; extra == 'dev'
Requires-Dist: pre-commit>=3.7; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-httpx>=0.30; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff==0.16.1; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Provides-Extra: security
Requires-Dist: pip-audit==2.10.1; extra == 'security'
Description-Content-Type: text/markdown

# linode-llm-toolkit

> **Unofficial project.** Not affiliated with, endorsed by, or supported by Akamai Technologies, Inc. or Linode LLC. Akamai and Linode are trademarks of their respective owners.

`linode-llm-toolkit` is an independently maintained, experimental open-source CLI for deploying a
small, reviewed catalog of Ollama-hosted LLMs on Linode instances. The compatibility command is
`linode-llm`; `w7089` is the maintainer and release authority.

One-command LLM deployment is limited to the reviewed public-alpha instance path described below.

## Quickstart: deploy a personal model

You need Python 3.12+, a Linode account, and a Linode API token. Install the exact alpha with `uv`:

```bash
uv tool install 'linode-llm-toolkit==0.1.0a3'
```

Then deploy with only the token:

```bash
export LINODE_TOKEN='…'
linode-llm deploy
```

In a terminal, `deploy` shows only reviewed catalog models for which this account currently has a
viable live placement and complete price. Select one, or deploy a known catalog model directly:

```bash
linode-llm deploy qwen2.5:1.5b
```

The CLI calculates sizing, cheapest eligible plan, region capacity, and live price behind the scenes,
then shows the resolved model, plan, region, and price before creating anything. Non-interactive
callers that omit the model use the reviewed Qwen 1.5B default.

When the model is ready, the success output includes values ready for an OpenAI-compatible app:

```bash
export OPENAI_BASE_URL='http://<endpoint>/v1'
export OPENAI_MODEL='<selected-model>'
export OPENAI_API_KEY='not-needed'
```

WebUI and SSH are off. The CLI detects your public IPv4 address and creates a default-deny firewall
that allows plain HTTP on port 80 only from that address's `/32`.

This is a personal prototype boundary, not production security: the endpoint is plain HTTP with no
application credential, protected only by source-IP filtering. Do not use it for sensitive prompts
or data, shared access, or a public service. Resources remain billable until you destroy them. Save
the deployment ID from the success output, then run exactly:

```bash
linode-llm destroy <deployment-id>
```

See [Simple deployment](docs/SIMPLE_DEPLOY.md) for the tested lifecycle, exact boundary, app setup,
and cleanup.

## Status and limitations

This is not a production service and carries no production or Marketplace support commitment. The instance path is the public-alpha path; the bundled LKE/Helm chart and StackScript are experimental developer-preview paths and are not Marketplace release candidates.

Before using it, understand these limits:

- A deployment needs a live, authenticated Linode API overlay. Cached plan data is for offline planning only and cannot establish account or regional availability.
- Billing can continue until you remove billable resources; a guest shutdown does not make its Linode free.
- The tool records local state, but state can be lost or damaged. Use `reconcile` and `destroy --plan` to inspect owned remote resources before cleanup.
- Runtime artifacts are pinned for this experimental profile, but the runtime/deployment release profile remains fail-closed and is not release-eligible. The published production-PyPI alpha `0.1.0a3` is a package-scoped release only; it does not change that runtime boundary. See [RELEASING.md](RELEASING.md).

## Installation alternatives

The package-scoped `0.1.0a3` alpha is available from [PyPI](https://pypi.org/project/linode-llm-toolkit/0.1.0a3/).
If you do not use `uv`, install it with `pip` inside a Python 3.12+ virtual environment:

```bash
python3.12 -m venv .venv
. .venv/bin/activate
python -m pip install 'linode-llm-toolkit==0.1.0a3'
linode-llm --help
```

Do not install into the operating system's Python with a bare `pip install`. Debian and Ubuntu protect
that environment under PEP 668; activating a virtual environment and using `python -m pip` also makes
it clear which interpreter receives the CLI.

This package publication does not make the deployment runtime, LKE chart, or StackScript production-ready or Marketplace-supported. The earlier TestPyPI `0.1.0a1` rehearsal remains transport evidence only.

For a source checkout, use `uv sync --extra dev` and `uv run linode-llm --help`.

## Reviewed catalog

The alpha catalog snapshot is `phase0-alpha-2026-08-04`. Inspect the installed snapshot rather than copying a model name from older material:

```bash
linode-llm catalog
linode-llm catalog --json
```

Entries with additional acceptance requirements need their exact catalog ID. The permissively
licensed simple default does not require a separate acknowledgement:

| Catalog ID | Alias | Quantization | Context | Preferred reviewed plan |
| --- | --- | --- | --- | --- |
| `qwen2.5-1.5b-instruct-q4-k-m` | `qwen2.5:1.5b` | `Q4_K_M` | 32,768 tokens | `g6-standard-2` (4 GB shared CPU; live lifecycle passed 2026-08-04) |
| `gpt-oss-20b-mxfp4` | `gpt-oss:20b` | `MXFP4` | 131,072 tokens | `g2-gpu-rtx4000a1-s` |
| `gpt-oss-120b-mxfp4` | `gpt-oss:120b` | `MXFP4` | 131,072 tokens | `g3-gpu-rtxpro6000-blackwell-1` |

The `models` command also shows pre-alpha compatibility records. Those are `legacy_unverified`, remain discoverable for compatibility, and cannot be planned or deployed. Artifact identities, license evidence, and acceptance requirements are in [MODEL_NOTICES.md](MODEL_NOTICES.md).

## Advanced options

### Authenticated TLS, credentials, licenses, and explicit placement

Set a Linode API token only in your environment; never commit or print it. A plan and deployment also require separate inference and administration endpoint credentials. The values are resolved at use time from references and only references are stored in deployment state.

```bash
export LINODE_TOKEN='…'
export INFERENCE_KEY='…'
export ADMINISTRATION_KEY='…'

linode-llm plan \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --fqdn llm.example.com

linode-llm deploy \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --fqdn llm.example.com \
  --inference-credential environment:INFERENCE_KEY \
  --administration-credential environment:ADMINISTRATION_KEY
```

Public mode requires a caller-managed FQDN and Caddy automatic HTTPS. Without a domain, use authenticated, source-restricted developer mode instead:

```bash
linode-llm deploy \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --allow-cidr 203.0.113.0/24 \
  --inference-credential environment:INFERENCE_KEY \
  --administration-credential environment:ADMINISTRATION_KEY
```

When explicitly selecting an advanced security profile, use one of `--fqdn`, `--allow-cidr`, or the
public-HTTP acknowledgement. Omitting all three selects the source-restricted personal-prototype
default. Public anonymous HTTP is experimental and requires the exact acknowledgement
`I_ACKNOWLEDGE_PUBLIC_HTTP_IS_UNSAFE`.

Use the inference credential for `/v1/*` and the separate administration credential for `/api/*`. Do not place either value in source code, a shell history you share, or deployment state.

### Security

For an advanced internet-facing deployment, use authenticated public TLS or authenticated
source-restricted developer mode. Provide an SSH public key with `--ssh-key` when you need instance
login; without one, you cannot log in to the instance. See [SECURITY.md](SECURITY.md) for private
reporting and the full security boundary.

### Cost and planning

For advanced cost policy, run `linode-llm plan` before `deploy`. It is mutation-free and emits a
stable, itemized JSON plan. It distinguishes live, cached, estimated, and unknown amounts; unknown
amounts are never treated as zero. When `--budget` is supplied, a rejected budget plan cannot proceed
to deployment. The simple path displays live compute pricing without requiring this guard.

`--budget` is a pre-deploy estimate guard, not spend enforcement. `--maximum-lifetime-hours` is a planning horizon; it does not schedule deletion.

`linode-llm cost` displays the approved cached snapshot `linode-plans-2026-08-04`, observed `2026-08-04`, for offline planning only. It is not live pricing. The approved cached hourly/monthly values in USD are:

| Plan | Hardware | Hourly | Monthly | Availability note |
| --- | --- | ---: | ---: | --- |
| `g6-standard-2` | Shared CPU (4 GiB) | $0.036 | $24.00 | Live lifecycle passed 2026-08-04; live overlay still required |
| `g6-standard-4` | Shared CPU (8 GiB) | $0.072 | $48.00 | Lifecycle validation candidate; live overlay still required |
| `g6-dedicated-16` | CPU | $0.432 | $288.00 | Live overlay still required |
| `g2-gpu-rtx4000a1-s` | NVIDIA RTX 4000 Ada (20 GiB) | $0.52 | $350.00 | Live overlay still required |
| `g1-gpu-rtx6000-1` | NVIDIA Quadro RTX 6000 (24 GiB) | $1.50 | $1,000.00 | Limited availability |
| `g3-gpu-rtxpro6000-blackwell-1` | NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GiB) | $2.50 | $1,665.00 | Limited availability; must appear in the authenticated live API for the account and region |

Actual regional pricing, capacity, account restrictions, storage, network, and other charges can differ or be unknown. The plan reports exclusions and unknowns explicitly.

## Cleanup and recovery

Inspect before deleting, then remove the deployment and verify it is gone:

```bash
linode-llm reconcile
linode-llm destroy --plan <deploy-id>
linode-llm destroy <deploy-id>
linode-llm reconcile
```

`reconcile` classifies resources as owned, foreign, or ambiguous. Foreign and ambiguous resources are not automatically adopted or deleted. Check the Linode control plane as well: cleanup is your responsibility, and there is no approved automatic account-side expiry deletion mechanism in this alpha.

## Commands

Run `linode-llm --help` for the authoritative options. Older compatibility-oriented commands may expose records that are not deployable; use `catalog` and `plan` for alpha decisions.

| Command | Purpose |
| --- | --- |
| `linode-llm bench` | Benchmark a deployed LLM. |
| `linode-llm catalog` | Inspect the reviewed catalog snapshot. |
| `linode-llm chat` | Open an interactive chat session with a deployment. |
| `linode-llm cost` | Display dated cached cost estimates. |
| `linode-llm deploy` | Validate and create an instance deployment. |
| `linode-llm destroy` | Preview or remove a deployment. |
| `linode-llm init` | Create an application starter. |
| `linode-llm logs` | Show deployment diagnostics. |
| `linode-llm models` | List deployable and legacy compatibility records. |
| `linode-llm plan` | Create a mutation-free deployment plan. |
| `linode-llm ps` | List models on a deployment. |
| `linode-llm pull` | Pull an additional model on a deployment. |
| `linode-llm reconcile` | Inspect owned, foreign, and ambiguous resources. |
| `linode-llm rm` | Remove a model from a deployment. |
| `linode-llm status` | Check deployment status. |
| `linode-llm templates` | List application starters. |

## Support, compatibility, and security

- [SUPPORT.md](SUPPORT.md) defines the best-effort community support boundary.
- [COMPATIBILITY.md](COMPATIBILITY.md) describes the pre-1.0 compatibility and deprecation policy.
- [SECURITY.md](SECURITY.md) explains private vulnerability reporting.
- [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) and [MODEL_NOTICES.md](MODEL_NOTICES.md) identify reviewed third-party/runtime and model notices.
- [RELEASING.md](RELEASING.md) documents the alpha release gate; [docs/REPOSITORY_SETTINGS_CHECKLIST.md](docs/REPOSITORY_SETTINGS_CHECKLIST.md) lists human GitHub/PyPI settings.

For a guided introduction see [docs/BEGINNER_GUIDE.md](docs/BEGINNER_GUIDE.md); contributors should start with [docs/DEVELOPER_GUIDE.md](docs/DEVELOPER_GUIDE.md).
