Metadata-Version: 2.4
Name: vllm-hust-clm-lifecycle
Version: 0.1.1
Summary: CLM: context-lifecycle management perception plugin for vLLM-HUST (request-lifecycle event sink)
License: Apache-2.0
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: test
Requires-Dist: pytest>=8; extra == "test"
Requires-Dist: ruff>=0.12; extra == "test"
Dynamic: license-file

# vllm-hust-clm-lifecycle

CLM: Context Lifecycle Management perception plugin for vLLM-HUST.

Consumes the scheduler-level request lifecycle events exposed by the host
(`RequestFinished` / `RequestPreempted`, contract `vllm.request-lifecycle.v1`)
and reports session-level KV growth to an out-of-tree CLM controller, which
decides when to compact a long conversation.

Bundle ID: `org.vllm-hust.clm-lifecycle` (Extension Bundle v1, schema 1.0)

## What it does

- **Lifecycle-sink component** (contract `vllm.request-lifecycle.v1`,
  execution plane `scheduler`): registers a sink on the engine's default-off
  typed `EventBus`. When a request finishes, is preempted, or has its KV
  actually reclaimed, the sink attributes the footprint to a session
  (engine-side `session_id`, or a `__default__` bucket) and reports it to a
  controller endpoint.
- **Reports distinguish decisions from reclaims.** `kv_growth` carries
  `kv_reclaim_deferred`, which is `true` when the host deferred the block
  release (a connector, or an in-flight step); `kv_reclaimed` is the event that
  means the blocks really returned to the pool. A controller should not treat a
  deferred `kv_growth` as free capacity.
- **Default-off, zero-overhead when absent**: the events only exist on hosts
  that carry the request-lifecycle hook; without them `register()` skips
  gracefully. Reports never block serving (failures are swallowed).

## Install

```bash
pip install vllm-hust-clm-lifecycle
```

## Enable

Requires a host that exposes `vllm.v1.events` (vllm-hust with the
request-lifecycle hook). Enable via plugin loading:

```bash
# runtime registration (vllm.general_plugins)
export VLLM_PLUGINS=ascend,clm_lifecycle
# ECPA static discovery and activation intent
vllm-hust-ext extension inspect org.vllm-hust.clm-lifecycle
vllm-hust-ext extension check org.vllm-hust.clm-lifecycle
```

### Environment

| Variable | Default | Meaning |
|---|---|---|
| `VLLM_CLM_CONTROLLER` | *(empty)* | HTTP endpoint receiving JSON reports. Empty = observation-only (no reports sent, no sender thread started). |
| `VLLM_CLM_REPORT_BLOCKS` | `128` | KV-block accumulation threshold per session before a report fires. |
| `VLLM_CLM_REPORT_INTERVAL` | `5.0` | Minimum seconds between two reports of the same session. |
| `VLLM_CLM_QUEUE_MAXSIZE` | `1024` | Bound on the in-process report queue. |
| `VLLM_CLM_QUEUE_POLICY` | `newest` | What to drop when the queue is full: `newest` discards the incoming report, `oldest` discards the queued head to make room. |
| `VLLM_CLM_SEND_TIMEOUT` | `2.0` | Per-attempt HTTP timeout, applied on the sender thread only. |
| `VLLM_CLM_MAX_ATTEMPTS` | `3` | Attempts per report before it is counted as failed and dropped. |
| `VLLM_CLM_CLOSE_TIMEOUT` | `5.0` | Upper bound on how long `close()` will spend draining. |

Parse failures on these variables fall back to defaults (empty values are
tolerated).

### Reports never block the scheduler

`emit()` runs on the engine's scheduler thread, so it performs **no I/O**: it
aggregates in memory and hands the report to a bounded queue drained by a
private daemon thread. A slow or unreachable controller therefore cannot stall
scheduling, and the bounded queue means it cannot grow memory without limit.

Every counter is available via `ClmLifecycleSink.stats_snapshot()` so
"non-blocking" is observable rather than asserted:

| Counter | Meaning |
|---|---|
| `events_seen` | lifecycle events received from the host |
| `reports_queued` | reports accepted into the queue |
| `reports_sent` | reports the controller accepted |
| `failed_reports` | reports that exhausted their attempts |
| `dropped_on_full` | reports discarded because the queue was full |
| `dropped_on_shutdown` | reports discarded during/after `close()` |
| `emit_duration_max_s` | worst observed `emit()` wall-clock time |
| `last_error` | most recent send failure, for diagnosis |

The test suite drives a real localhost HTTP server that stalls for seconds and
asserts that `emit()` stays far below the sender timeout, that drops are
counted, and that `close()` neither hangs nor loses reports silently.

## Disable

```bash
unset VLLM_CLM_CONTROLLER VLLM_CLM_REPORT_BLOCKS VLLM_CLM_REPORT_INTERVAL
# or simply do not list clm_lifecycle in VLLM_PLUGINS
pip uninstall vllm-hust-clm-lifecycle
```

## Development

```bash
pip install -e ".[test]"
python -m pytest tests/
ruff check .
```

`tests/conftest.py` stubs `vllm.v1.events` before the plugin is imported, so
the test suite runs without a vLLM install.

## License

Apache-2.0
