Metadata-Version: 2.4
Name: linode-llm-toolkit
Version: 0.1.0a2
Summary: Unofficial one-command LLM deployment CLI for Akamai/Linode cloud
Project-URL: Homepage, https://github.com/w7089/linode-llm-toolkit
Project-URL: Documentation, https://github.com/w7089/linode-llm-toolkit#readme
Project-URL: Issues, https://github.com/w7089/linode-llm-toolkit/issues
Project-URL: Changelog, https://github.com/w7089/linode-llm-toolkit/blob/main/CHANGELOG.md
Author: w7089
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: ai,akamai,deployment,kubernetes,linode,llm,ollama
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.12
Requires-Dist: click>=8.1
Requires-Dist: httpx>=0.27
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Provides-Extra: dev
Requires-Dist: mypy==2.3.0; extra == 'dev'
Requires-Dist: pre-commit>=3.7; extra == 'dev'
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
Requires-Dist: pytest-httpx>=0.30; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: respx>=0.21; extra == 'dev'
Requires-Dist: ruff==0.16.1; extra == 'dev'
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
Provides-Extra: security
Requires-Dist: pip-audit==2.10.1; extra == 'security'
Description-Content-Type: text/markdown

# linode-llm-toolkit

> **Unofficial project.** Not affiliated with, endorsed by, or supported by Akamai Technologies, Inc. or Linode LLC. Akamai and Linode are trademarks of their respective owners.

`linode-llm-toolkit` is an independently maintained, experimental open-source CLI for planning and deploying a small, reviewed set of Ollama-hosted LLMs on Linode instances. The compatibility command is `linode-llm`; `w7089` is the maintainer and release authority.

One-command LLM deployment is limited to the reviewed public-alpha instance path described below.

## Status and limitations

This is not a production service and carries no production or Marketplace support commitment. The instance path is the public-alpha path; the bundled LKE/Helm chart and StackScript are experimental developer-preview paths and are not Marketplace release candidates.

Before using it, understand these limits:

- A deployment needs a live, authenticated Linode API overlay. Cached plan data is for offline planning only and cannot establish account or regional availability.
- `--budget` is a pre-deploy estimate guard, not spend enforcement. `--maximum-lifetime-hours` is a planning horizon; it does not schedule deletion. Billing can continue until you remove billable resources.
- The tool records local state, but state can be lost or damaged. Use `reconcile` and `destroy --plan` to inspect owned remote resources before cleanup.
- Runtime artifacts are pinned for this experimental profile, but the runtime/deployment release profile remains fail-closed and is not release-eligible. A production-PyPI alpha, when published, is a package-scoped release only; it does not change that runtime boundary. See [RELEASING.md](RELEASING.md).

## Install

When a production-PyPI alpha has been published, install that released version with:

```bash
uv tool install linode-llm-toolkit
# or
pip install linode-llm-toolkit

linode-llm --help
```

Publication has not happened merely because a TestPyPI rehearsal exists. The recorded TestPyPI `0.1.0a1` rehearsal is evidence for the same-artifact release procedure, not a claim that a production-PyPI alpha is already available.

For a source checkout, use `uv sync --extra dev` and `uv run linode-llm --help`.

## Reviewed catalog

The alpha catalog snapshot is `phase0-alpha-2026-08-01`. Inspect the installed snapshot rather than copying a model name from older material:

```bash
linode-llm catalog
linode-llm catalog --json
```

Deployable entries require acceptance of their exact catalog ID:

| Catalog ID | Alias | Quantization | Context | Preferred reviewed plan |
| --- | --- | --- | --- | --- |
| `qwen2.5-1.5b-instruct-q4-k-m` | `qwen2.5:1.5b` | `Q4_K_M` | 32,768 tokens | `g6-dedicated-16` |
| `gpt-oss-20b-mxfp4` | `gpt-oss:20b` | `MXFP4` | 131,072 tokens | `g2-gpu-rtx4000a1-s` |
| `gpt-oss-120b-mxfp4` | `gpt-oss:120b` | `MXFP4` | 131,072 tokens | `g3-gpu-rtxpro6000-blackwell-1` |

The `models` command also shows pre-alpha compatibility records. Those are `legacy_unverified`, remain discoverable for compatibility, and cannot be planned or deployed. Artifact identities, license evidence, and acceptance requirements are in [MODEL_NOTICES.md](MODEL_NOTICES.md).

## Secure quickstart

Set a Linode API token only in your environment; never commit or print it. A plan and deployment also require separate inference and administration endpoint credentials. The values are resolved at use time from references and only references are stored in deployment state.

```bash
export LINODE_TOKEN='…'
export INFERENCE_KEY='…'
export ADMINISTRATION_KEY='…'

linode-llm plan \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --fqdn llm.example.com

linode-llm deploy \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --fqdn llm.example.com \
  --inference-credential environment:INFERENCE_KEY \
  --administration-credential environment:ADMINISTRATION_KEY
```

Public mode requires a caller-managed FQDN and Caddy automatic HTTPS. Without a domain, use authenticated, source-restricted developer mode instead:

```bash
linode-llm deploy \
  --model gpt-oss:20b \
  --accept-license gpt-oss-20b-mxfp4 \
  --budget 400 \
  --maximum-lifetime-hours 24 \
  --allow-cidr 203.0.113.0/24 \
  --inference-credential environment:INFERENCE_KEY \
  --administration-credential environment:ADMINISTRATION_KEY
```

Exactly one of `--fqdn`, `--allow-cidr`, or the explicit public-HTTP acknowledgement is required. Public HTTP is deliberately excluded from this quickstart; it is experimental and requires the exact acknowledgement `I_ACKNOWLEDGE_PUBLIC_HTTP_IS_UNSAFE`.

Use the inference credential for `/v1/*` and the separate administration credential for `/api/*`. Do not place either value in source code, a shell history you share, or deployment state.

## Security

The secure default is authenticated public TLS or authenticated source-restricted developer mode. Provide an SSH public key with `--ssh-key` when you need instance login; without one, you cannot log in to the instance. Do not disable the default boundary to obtain a simpler example. See [SECURITY.md](SECURITY.md) for private reporting and the full security boundary.

## Cost and planning

Run `linode-llm plan` before `deploy`. It is mutation-free and emits a stable, itemized JSON plan. It distinguishes live, cached, estimated, and unknown amounts; unknown amounts are never treated as zero. A rejected budget plan cannot proceed to deployment.

`linode-llm cost` displays the approved cached snapshot `linode-plans-2026-08-01`, observed `2026-08-01T15:20:40Z`, for offline planning only. It is not live pricing. The approved cached hourly/monthly values in USD are:

| Plan | Hardware | Hourly | Monthly | Availability note |
| --- | --- | ---: | ---: | --- |
| `g6-dedicated-16` | CPU | $0.432 | $288.00 | Live overlay still required |
| `g2-gpu-rtx4000a1-s` | NVIDIA RTX 4000 Ada (20 GiB) | $0.52 | $350.00 | Live overlay still required |
| `g1-gpu-rtx6000-1` | NVIDIA Quadro RTX 6000 (24 GiB) | $1.50 | $1,000.00 | Limited availability |
| `g3-gpu-rtxpro6000-blackwell-1` | NVIDIA RTX PRO 6000 Blackwell Server Edition (96 GiB) | $2.50 | $1,665.00 | Limited availability; must appear in the authenticated live API for the account and region |

Actual regional pricing, capacity, account restrictions, storage, network, and other charges can differ or be unknown. The plan reports exclusions and unknowns explicitly.

## Cleanup and recovery

Inspect before deleting, then remove the deployment and verify it is gone:

```bash
linode-llm reconcile
linode-llm destroy --plan <deploy-id>
linode-llm destroy <deploy-id>
linode-llm reconcile
```

`reconcile` classifies resources as owned, foreign, or ambiguous. Foreign and ambiguous resources are not automatically adopted or deleted. Check the Linode control plane as well: cleanup is your responsibility, and there is no approved automatic account-side expiry deletion mechanism in this alpha.

## Commands

Run `linode-llm --help` for the authoritative options. Older compatibility-oriented commands may expose records that are not deployable; use `catalog` and `plan` for alpha decisions.

| Command | Purpose |
| --- | --- |
| `linode-llm bench` | Benchmark a deployed LLM. |
| `linode-llm catalog` | Inspect the reviewed catalog snapshot. |
| `linode-llm chat` | Open an interactive chat session with a deployment. |
| `linode-llm cost` | Display dated cached cost estimates. |
| `linode-llm deploy` | Validate and create an instance deployment. |
| `linode-llm destroy` | Preview or remove a deployment. |
| `linode-llm init` | Create an application starter. |
| `linode-llm logs` | Show deployment diagnostics. |
| `linode-llm models` | List deployable and legacy compatibility records. |
| `linode-llm plan` | Create a mutation-free deployment plan. |
| `linode-llm ps` | List models on a deployment. |
| `linode-llm pull` | Pull an additional model on a deployment. |
| `linode-llm reconcile` | Inspect owned, foreign, and ambiguous resources. |
| `linode-llm rm` | Remove a model from a deployment. |
| `linode-llm status` | Check deployment status. |
| `linode-llm templates` | List application starters. |

## Support, compatibility, and security

- [SUPPORT.md](SUPPORT.md) defines the best-effort community support boundary.
- [COMPATIBILITY.md](COMPATIBILITY.md) describes the pre-1.0 compatibility and deprecation policy.
- [SECURITY.md](SECURITY.md) explains private vulnerability reporting.
- [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) and [MODEL_NOTICES.md](MODEL_NOTICES.md) identify reviewed third-party/runtime and model notices.
- [RELEASING.md](RELEASING.md) documents the alpha release gate; [docs/REPOSITORY_SETTINGS_CHECKLIST.md](docs/REPOSITORY_SETTINGS_CHECKLIST.md) lists human GitHub/PyPI settings.

For a guided introduction see [docs/BEGINNER_GUIDE.md](docs/BEGINNER_GUIDE.md); contributors should start with [docs/DEVELOPER_GUIDE.md](docs/DEVELOPER_GUIDE.md).
