Metadata-Version: 2.4
Name: forewarm
Version: 0.0.1
Summary: Token-aware predictive autoscaling for LLM inference (early development)
Author-email: Alexey Kazantsev <kazantsevalant@gmail.com>
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/Kazantsev-Alexey/forewarm
Project-URL: Repository, https://github.com/Kazantsev-Alexey/forewarm
Keywords: llm,vllm,autoscaling,forecasting,kubernetes,keda
Classifier: Development Status :: 1 - Planning
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: System Administrators
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: System :: Distributed Computing
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# Forewarm

**Token-aware predictive autoscaling for LLM inference.**

Forewarm reads vLLM serving metrics from Prometheus, learns the workload's own
daily and weekly structure, and publishes a quantile forecast of near-future
demand — so replicas can be warmed up *before* the load arrives instead of after
a queue has already formed.

LLM pods cold-start in minutes (weights have to be loaded), which forces reactive
autoscalers to keep a guessed, permanently inflated floor of always-on replicas.
Forewarm exists to lower that floor safely.

## Status

Early development — this release is a placeholder that reserves the name while
the first working version is built in the open. There is no usable functionality
yet; please do not depend on it.

Planned for the first real release (`0.1.0`):

- Prometheus ingestion of vLLM metrics (requests, prompt/generation tokens, queue depth)
- Zero-config model fitting on the cluster's own history, with an automatically
  selected champion benchmarked against a seasonal-naive baseline
- Quantile forecasts exposed as Prometheus metrics and via a gRPC KEDA external scaler
- A backtest CLI reporting forecast accuracy, SLO violation risk, and GPU dollars

Follow the work at https://github.com/Kazantsev-Alexey/forewarm

## License

Apache-2.0
