Metadata-Version: 2.4
Name: llm-ot-cost-monitor
Version: 1.0.1
Summary: Local collector for LLM token usage emitted through OpenTelemetry
Author-email: Roberto Rossi <robros@gmail.com>
License: MIT
Keywords: llm,cost,monitoring,opentelemetry,token-usage,github-copilot
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE.txt
Requires-Dist: genai-pricing==0.2.3
Requires-Dist: opentelemetry-proto==1.36.0
Dynamic: license-file

# LLM Open Telemetry Cost Monitor

A small local collector for token usage emitted by GitHub Copilot through OpenTelemetry. It accepts OTLP/HTTP trace protobuf and JSON, stores token usage metadata in SQLite, and estimates cost with `genai-pricing`.

The monitor stores trace ID, span ID, timestamp, model, input tokens, cache-read input tokens, cache-creation input tokens, and output tokens. It does not capture prompts or responses.

[![Release](https://img.shields.io/github/v/release/gwr3n/llm-ot-cost-monitor)](https://github.com/gwr3n/llm-ot-cost-monitor/releases)
[![License](https://img.shields.io/github/license/gwr3n/llm-ot-cost-monitor)](LICENSE)
 [![PyPI](https://img.shields.io/pypi/v/llm-ot-cost-monitor)](https://pypi.org/project/llm-ot-cost-monitor/)
 [![Downloads](https://pepy.tech/badge/llm-ot-cost-monitor)](https://pepy.tech/project/llm-ot-cost-monitor) 

## Requirements

- Python 3.10 or newer
- VS Code and a Copilot Chat version that supports OpenTelemetry export
- Network access for the first pricing lookup, unless `genai-pricing` already has a pricing cache

This project uses the repository-local virtual environment in `venv/`.

## Install

Install the package into an activated virtual environment:

```bash
python -m pip install llm-ot-cost-monitor
```

To work from a repository checkout instead, install its dependencies:

```bash
./venv/bin/python -m pip install -r requirements.txt
```

The dependencies are pinned in `requirements.txt`.

## Configure VS Code

Copilot's OpenTelemetry settings are application-scoped. Amend only `~/Library/Application Support/Code/User/settings.json`; do not add them to a workspace `.vscode/settings.json` file:

```json
{
	"github.copilot.chat.otel.enabled": true,
	"github.copilot.chat.otel.exporterType": "otlp-http",
	"github.copilot.chat.otel.protocol": "http/protobuf",
	"github.copilot.chat.otel.otlpEndpoint": "http://127.0.0.1:4318",
	"github.copilot.chat.otel.captureContent": false
}
```

Reload VS Code after changing the settings, then run Copilot requests while the collector is running.

Do not enable content capture unless you explicitly want prompts, responses, or source code included in telemetry. This collector does not require content capture.

## Start the collector

Start the local OTLP/HTTP receiver:

```bash
llm-monitor serve --db usage.db
```

The `llm-monitor` command is installed with the package. From a repository checkout,
you can instead run `./venv/bin/python monitor.py serve --db usage.db`.

The receiver listens on `127.0.0.1:4318` and accepts trace exports at:

```text
http://127.0.0.1:4318/v1/traces
```

To use another database, pass a different path:

```bash
llm-monitor serve --db data/usage.db
```

To change the listening address or port:

```bash
llm-monitor serve --host 127.0.0.1 --port 4318 --db usage.db
```

Keep the collector process running while using Copilot.

## View usage and estimated cost

After running Copilot requests, stop or leave the collector running and execute:

```bash
llm-monitor summary --db usage.db
```

The output contains:

- `model`: reported model identifier
- `requests`: number of stored usage spans
- `input`: input tokens reported by `gen_ai.usage.input_tokens`
- `cache_read`: input tokens reported by `gen_ai.usage.cache_read.input_tokens`
- `cache_creation`: input tokens reported by `gen_ai.usage.cache_creation.input_tokens`
- `output`: total output tokens
- `estimated_cost_usd`: estimated cost in US dollars

Example:

```text
model       | requests | input | cache_read | cache_creation | output | estimated_cost_usd
------------+----------+-------+------------+----------------+--------+--------------------
gpt-4o-mini |        2 |  1200 |        800 |            100 |    340 |           0.000248
```

Costs are calculated by `genai-pricing` using its pricing table, including provider-specific cache-read and cache-creation rates when available. Unknown models or unavailable prices are shown as `n/a`; the monitor does not guess rates.

`genai-pricing` treats `input` as the total prompt token count and subtracts the two cached-token counts before applying the regular input rate. This matches telemetry payloads where `gen_ai.usage.input_tokens` includes cached tokens. If an extension emits only non-cached tokens in that attribute, its estimate may undercount regular input because the semantic convention permits either payload interpretation.

## Data and duplicate handling

The default database is `usage.db` in the repository root. SQLite creates it automatically when the collector starts.

Each record is keyed by trace ID and span ID. Repeated exports of the same span are ignored, preventing duplicate cost totals.

The database may contain sensitive operational metadata such as model names and usage volume. Keep it local and do not commit `usage.db` to source control.

## Test

Run the unit tests with the local virtual environment:

```bash
env -u PYTHONHOME -u PYTHONPATH ./venv/bin/python -m unittest -v
```

The tests cover OTLP usage extraction, duplicate span handling, and the `genai-pricing` API call.

## Limitations

- Exact usage depends on Copilot emitting the relevant `gen_ai.usage.*` token attributes.
- Missing token attributes cannot be reconstructed from trace metadata alone.
- Costs are estimates based on the pricing data available to `genai-pricing` and may not match account-specific billing, discounts, or included usage.
- The receiver is intentionally local and minimal; it does not provide authentication, a web dashboard, or remote storage.
