Metadata-Version: 2.4
Name: contextops
Version: 0.3.3
Summary: Deterministic context linter for LLM applications — analyze, score, and optimize your LLM context payloads.
Author: Abhijeet Baug
License: Sustainable Use License
        
        Copyright (c) 2026 Abhijeet Baug
        
        Acceptance
        
        By using the software, you agree to all of the terms and conditions below.
        
        Copyright License
        
        The copyright holder (Abhijeet Baug, "Licensor") grants you a non-exclusive,
        royalty-free, worldwide, non-sublicensable, non-transferable license to use,
        copy, distribute, and create derivative works of the software, in each case
        subject to the limitations set out below.
        
        Limitations
        
        1. Internal / Non-Commercial Use Only. You may use or modify the software
           only for your own internal business operations, personal use, or
           non-commercial purposes.
        
        2. No Resale or Commercial Hosting. You may not provide the software, or
           any derivative work of the software, to third parties as a hosted or
           managed service, or as part of a commercial product or service offering,
           where the core value delivered to those third parties is substantially
           derived from the software's functionality.
        
        3. No Charge for Distribution. You may distribute or make the software
           (or a derivative work) available to others only if you do so free of
           charge, and only for non-commercial purposes.
        
        4. No Removal of Notices. You must not remove or obscure any licensing,
           copyright, or other notices contained in the software.
        
        Contributions
        
        Any contribution intentionally submitted by you for inclusion in the
        software shall be under the terms and conditions of this license, without
        any additional terms or conditions, unless explicitly agreed otherwise in
        writing with the Licensor.
        
        Patents
        
        This license does not grant you any right in any patent held by the
        Licensor. No patent license is granted by implication, estoppel, or
        otherwise.
        
        Notices
        
        You must ensure that anyone who gets a copy of any part of the software
        from you also gets a copy of these terms.
        
        No Other Rights
        
        These terms do not imply any licenses other than those expressly granted
        in this license.
        
        Termination
        
        If you use the software in violation of these terms, such use is not
        licensed, and your license will automatically terminate. If the Licensor
        provides you with a notice of your violation, and you cease all violation
        of this license no later than 30 days after you receive that notice,
        your license will be reinstated retroactively. However, if you violate
        these terms after such reinstatement, any additional violation of these
        terms will cause your license to terminate automatically and permanently.
        
        No Liability
        
        As far as the law allows, the software comes "as is," without any
        warranty or condition, and the Licensor will not be liable to you for any
        damages arising out of these terms or the use or nature of the software,
        under any kind of legal claim.
        
        Definitions
        
        "Software" means the present software and any associated documentation
        made available by the Licensor under these terms, as may be updated by
        the Licensor from time to time.
        
        "You" means the individual or entity agreeing to these terms.
        
        "Licensor" means Abhijeet Baug.
        
Project-URL: Homepage, https://github.com/Abhijeet777/contextops
Project-URL: Repository, https://github.com/Abhijeet777/contextops
Project-URL: Issues, https://github.com/Abhijeet777/contextops/issues
Project-URL: Documentation, https://github.com/Abhijeet777/contextops#readme
Project-URL: Changelog, https://github.com/Abhijeet777/contextops/blob/main/CHANGELOG.md
Keywords: llm,context,observability,rag,token,optimization,linter,ci,deterministic,prompt-engineering
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: tiktoken>=0.5.0
Requires-Dist: click>=8.0.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Dynamic: license-file

# ContextOps

> **Static analysis for LLM context.**

[![PyPI](https://img.shields.io/pypi/v/contextops.svg)](https://pypi.org/project/contextops/)
[![Python](https://img.shields.io/pypi/pyversions/contextops.svg)](https://pypi.org/project/contextops/)
[![License](https://img.shields.io/badge/license-Sustainable%20Use-blue.svg)](./LICENSE)
[![Stability](https://img.shields.io/badge/stability-contract-brightgreen)](./STABILITY.md)

ContextOps is a **deterministic, model-independent context linter** for LLM applications.

It analyzes the context sent to an LLM **before inference** and detects structural problems such as redundancy, token waste, context imbalance, and source concentration. It produces a reproducible **Context Health Score (CHS)** together with actionable diagnostics — without embeddings, model calls, or external services.

Think of ContextOps as **ESLint for LLM context**.

---

## Why ContextOps?

Modern software engineering has deterministic quality gates.

- Compilers catch syntax errors.
- Linters catch code smells.
- Formatters enforce consistency.
- Static analyzers detect architectural problems.

LLM applications rarely have an equivalent layer for the context they send into models.

Instead, prompts quietly grow over time:

- duplicated retrieval chunks
- bloated system prompts
- runaway conversation history
- excessive tool output
- hidden token waste

These issues increase latency and cost, make model behavior less predictable, and often go unnoticed until production.

ContextOps makes context quality **observable, measurable, and testable before inference.**

---

## What ContextOps Measures

ContextOps evaluates four structural dimensions.

| Dimension | Max Penalty | What It Measures |
|-----------|-------------|------------------|
| **Redundancy** | 30 pts | Lexical duplication across context items |
| **Density** | 30 pts | Token waste from formatting and structural bloat |
| **Structure** | 20 pts | Distribution imbalance between context components |
| **Concentration** | 20 pts | Over-reliance on a single document or source |

The result is a deterministic **0–100 Context Health Score** together with detailed findings and suggested fixes.

---

## What ContextOps Does **Not** Do

ContextOps intentionally does **not** evaluate:

- prompt engineering quality
- reasoning ability or hallucinations
- factual correctness
- retrieval relevance
- LLM outputs

It focuses exclusively on the **structural quality of the context before inference.**

---

## Example

```bash
contextops inspect context.json
```

```
Context Health Score: 81 / 100

✓ Low structural complexity
✓ Good source diversity

Warnings

• 214 duplicated tokens detected
• Retrieval occupies 78% of context
• Two retrieval chunks are near duplicates

Estimated token savings: 12%
```

Use `--roast` mode for more candid diagnostics:

```bash
contextops inspect context.json --roast
```

---

## Installation

```bash
pip install contextops
```

Run the interactive demo:

```bash
contextops demo
```

**Dependencies:** Only `tiktoken` and `click`. No GPU, no network, no external API keys required.

---

## Quick Start

### 1. Capture your context

Save your LLM payload before sending it to the model.

```python
import json

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": user_query},
]

# Add two lines before your API call
with open("context.json", "w") as f:
    json.dump(messages, f, indent=2)

response = openai.chat.completions.create(model="gpt-4o", messages=messages)
```

Or use the structured dict format for richer analysis:

```json
{
  "system": "You are a helpful customer support bot.",
  "messages": [
    {"role": "user", "content": "How long will my refund take?"}
  ],
  "chunks": [
    {"content": "Refunds take 3-5 business days.", "source": "docs/refunds.md"}
  ],
  "memory": ["The user asked about a refund yesterday."],
  "tools": [{"name": "search_api", "output": "Tool response text here"}]
}
```

### 2. Inspect it

```bash
contextops inspect context.json
```

### 3. Enforce quality in CI

```bash
contextops check context.json --min-score 75
```

---

## CLI Reference

### `contextops inspect <file>`

Analyse a context file and display a rich report.

```bash
contextops inspect context.json
contextops inspect context.json --roast
contextops inspect context.json --profile rag
contextops inspect context.json --explain
contextops inspect context.json --json-output
```

| Flag | Default | Description |
|------|---------|-------------|
| `--json-output` | off | Output raw JSON instead of terminal format |
| `--model <name>` | `gpt-4o` | Model for token encoding (e.g. `gpt-4`, `claude-3`) |
| `--profile <name>` | `general` | Archetype: `rag`, `agent`, `chatbot`, `toolchain` |
| `--config <path>` | none | Path to a JSON config file with custom thresholds |
| `--retrieval-max-ratio <f>` | 0.70 | Override max allowed ratio for retrieval chunks |
| `--system-max-ratio <f>` | 0.50 | Override max allowed ratio for system prompt |
| `--memory-max-ratio <f>` | 0.50 | Override max allowed ratio for memory entries |
| `--tool-max-ratio <f>` | 0.60 | Override max allowed ratio for tool outputs |
| `--explain` | off | Show detailed "Top Score Drivers" for each penalty |
| `--roast` | off | Enable brutally honest score-band commentary |

---

### `contextops check <file> --min-score N`

CI gate. Exits with code `0` (pass) or `1` (fail).

```bash
contextops check context.json --min-score 75
```

| Exit Code | Meaning |
|-----------|---------|
| `0` | PASS — score meets or exceeds threshold |
| `1` | FAIL — score is below threshold or analysis error |
| `2` | ERROR — invalid JSON or unreadable input |

**GitHub Actions example:**

```yaml
- name: Check context quality
  run: contextops check context.json --min-score 75 --profile rag
```

---

### `contextops diff <file_a> <file_b>`

Compare two context snapshots to detect regressions.

```bash
contextops diff before.json after.json
```

Shows score delta, which penalties changed, and whether the change is an improvement or regression. Useful for A/B testing retrieval strategies or prompt refactors.

---

### `contextops stability [file]`

Run a deterministic stability report to verify the scoring engine is working correctly.

```bash
contextops stability
contextops stability context.json
```

---

### `contextops telemetry`

Local-only, opt-in telemetry for tracking context quality trends over time.

```bash
contextops telemetry status          # Check if telemetry is active
contextops telemetry log --limit 25  # Show recent events
contextops telemetry trends --days 7 # Show 7-day quality trends
```

---

### `contextops badge [--score N]`

Generate a GitHub shields.io markdown badge.

```bash
contextops badge             # Uses your last telemetry score
contextops badge --score 87  # Use a specific score
```

Output:
```markdown
[![ContextOps](https://img.shields.io/badge/ContextOps-87-green)](https://github.com/Abhijeet777/contextops)
```

---

## Python API

### `inspect_context()`

```python
from contextops.api.inspect import inspect_context

result = inspect_context(
    payload,          # dict, list, or plain string
    model="gpt-4o",   # model for token counting
    archetype="rag",  # archetype profile (optional)
)

print(result.score)               # int 0–100
print(result.token_breakdown.wasted_tokens)
for rec in result.recommendations:
    print(f"  -> {rec.fix}")
```

**Full result fields:**

```python
result.score                  # int 0–100
result.score_breakdown        # redundancy, density, structure, concentration penalties
result.token_breakdown        # total_tokens, by_type, wasted_tokens, estimated_cost_usd
result.redundancy_findings    # list[RedundancyFinding]
result.structure_findings     # list[StructureFinding]
result.recommendations        # list[Recommendation]
result.density_signal         # DensitySignal
result.archetype_resolved     # e.g. "rag"
result.roast                  # str | None (if roast_enabled)
result.metadata               # item_count, model, version
```

### `diff_contexts()`

```python
from contextops.api.diff import diff_contexts

result = diff_contexts(payload_a, payload_b)
```

### `ContextOpsConfig`

```python
from contextops.core.config import ContextOpsConfig

config = ContextOpsConfig.default(profile="rag")

config = ContextOpsConfig.from_dict({
    "retrieval_max_ratio": 0.90,
    "system_max_ratio": 0.30,
    "roast_enabled": True
})
```

---

## LangChain Integration

```bash
pip install contextops langchain-core
```

```python
from contextops import ContextOps

# Log the score (default — non-blocking)
chain = chain.with_config({
    "callbacks": [ContextOps.auto()]
})

# Block execution if context quality is too low
chain = chain.with_config({
    "callbacks": [
        ContextOps.auto(
            mode="block",
            min_score=75,
            profile="rag"
        )
    ]
})

result = chain.invoke({"question": "What is the refund policy?"})
```

| Mode | Behaviour |
|------|-----------|
| `"log"` | Prints score report to stdout (default) |
| `"warn"` | Emits Python warning if score < min_score |
| `"block"` | Raises `ContextOpsScoreError` if score < min_score — blocks LLM call |

---

## Archetype Profiles

Archetypes adjust structural thresholds for your specific use case. The global 0–100 score is never affected — only which warnings fire.

| Profile | When to Use | Retrieval | System | Memory | Tool |
|---------|-------------|-----------|--------|--------|------|
| `general` | Default — mixed use cases | 70% | 50% | 50% | 60% |
| `rag` | Pure document retrieval | **95%** | 40% | 20% | 30% |
| `agent` | Autonomous agents with tool loops | 50% | 40% | 40% | **90%** |
| `chatbot` | Conversational apps with large history | 40% | 50% | **85%** | 30% |
| `toolchain` | Multi-tool pipelines | 50% | 40% | 30% | **95%** |

**Resolution order** (highest priority wins):

1. `--profile` CLI flag
2. `archetype=` Python API argument
3. `"archetype"` key inside the JSON payload
4. `config.context_profile`
5. Default: `"general"`

---

## Input Formats

ContextOps accepts three input formats.

**Structured dict** (recommended — gives the richest analysis):
```json
{
  "system": "...",
  "messages": [...],
  "chunks": [...],
  "memory": [...],
  "tools": [...]
}
```

**OpenAI message list** (paste your existing payload directly):
```json
[
  {"role": "system", "content": "You are a helpful bot."},
  {"role": "user", "content": "Hello"}
]
```

**Plain string** (treated as a system prompt).

---

## Scoring Engine

```
Score = max(0, min(100, round(100 − total_penalty)))
```

Where `total_penalty = redundancy + density + structure + concentration`.

| Penalty | Max | What It Detects |
|---------|-----|-----------------|
| **Redundancy** | 30 | Lexical duplication via Jaccard + N-gram overlap |
| **Density** | 30 | Format overhead, whitespace waste, entropy compression |
| **Structure** | 20 | Retrieval dominance, system bloat, memory explosion |
| **Concentration** | 20 | Source dominance + entropy imbalance across chunks |

> **Lexical only.** ContextOps uses N-gram overlap, Jaccard similarity, and exact string matching — not semantic similarity. This is intentional: it is a structural analyser, not a semantic one.

---

## ContextBench — The LeetCode for Context Engineering

If LeetCode is **DSA for algorithms**, ContextBench is **DSA for context**.

Just as competitive programming teaches you that the algorithm matters — not just the answer — ContextBench teaches you that **context architecture matters**, not just whether the LLM eventually gets it right.

A brute-force O(N²) solution that produces the correct answer still gets penalized for time complexity. A bloated, redundant context that still produces a good LLM response still gets penalized for structural waste. The leaderboard notices both.

---

### The Parallel

| LeetCode / DSA | ContextBench Leaderboard |
|---|---|
| Problem set | **1,500 pre-built context windows** across 5 failure categories |
| Judge | **ContextOps engine** — deterministic scorer, same output every time |
| Your solution | `optimize_context(ctx)` — takes a broken context, returns a better one |
| Score metric | Quality (50%) + Compression (35%) + Latency (15%) |
| Leaderboard | Ranked by `final_score` across all benchmark samples |
| Adversarial track | **ContextSecBench** — 9,500 adversarial attack payloads |

---

### The Problem Set

ContextBench contains **1,500 samples** across 5 categories — think of these as your difficulty tiers:

| Category | Samples | What It Tests |
|----------|---------|---------------|
| Optimal Architectures | 300 | Healthy pipelines — prove you don't produce false positives |
| Structural Failures | 300 | System prompt bloat, retrieval flooding, memory explosion |
| Redundancy Failures | 300 | Near-duplicate clusters, boilerplate explosion, paraphrase loops |
| Agent Architecture Failures | 300 | Multi-agent context explosion, recursive planning loops, tool chain bloat |
| Temporal Context Drift | 300 | Stale memory injection, retrieval drift, invalidated historical state |

Every sample has a ground truth — just like a LeetCode problem has expected output:

```json
{
  "ground_truth": {
    "failure_modes": ["system_prompt_bloat"],
    "expected_properties": {
      "contains_redundancy": false,
      "contains_density_bloat": true,
      "contains_structure_imbalance": true
    }
  }
}
```

Flag redundancy when `contains_redundancy: false`? **False positive. Score drops.** Miss the density bloat? **False negative. Score drops.**

---

### Your Solution

Instead of writing a sorting algorithm, you write a context optimizer:

```python
def optimize_context(ctx: dict) -> dict:
    # Your logic: prune, compress, deduplicate, rebalance
    return better_ctx
```

The harness runs your function across all 1,500 samples and scores:

```python
final_score = (
    0.50 * quality_score     # ContextOps CHS must stay ≥ 78
  + 0.35 * compression       # Token reduction reward
  + 0.15 * latency_mult      # Speed penalty if you're slow
) * 100
```

The **Quality Floor Gate** (score ≥ 78) is non-negotiable — optimizers that destroy context quality to hit compression targets are disqualified.

---

### The Security Track: ContextSecBench

ContextSecBench is the adversarial extension — the CTF track on top of LeetCode.

**9,500 attack payloads** covering:

- **Prompt injection hiding** — malicious instructions buried deep inside retrieved context
- **Truncation smuggling** — payloads designed to survive chunking with harmful content intact
- **Semantic Denial of Service (SDoS)** — padding the context window to exhaust the attention budget
- **Context poisoning** — subtle corruption of retrieval content to bias model behavior
- **Format corruption** — structural attacks that break parser assumptions

Your optimizer must handle malicious context the same way a secure sorting algorithm must handle adversarial inputs.

---

### Why This Matters in Production

Poor context architecture causes problems long before the model itself becomes the bottleneck:

- Higher token costs and inference latency
- Lost-in-the-middle failures on long contexts
- Degraded reasoning quality from attention bandwidth collapse
- Context window exhaustion in long-running agent loops
- Hidden operational waste that compounds silently at scale

Better models don't solve poor context architecture. ContextBench measures the layer that does.

**[View the Leaderboard →](https://Abhijeet777ui.github.io/contextops/)**

---

## Core Principles

### Deterministic
The same input always produces the same score, breakdown, findings, and recommendations — on any machine, at any time. No randomness.

### Model-Independent
No embeddings, GPUs, inference APIs, or external AI services. The entire engine is pure Python math.

### CI-Friendly
Guaranteed `<2s` for payloads ≤ 5,000 tokens. `<10s` for 50,000-token payloads. Works offline. Exit-code driven.

### Structurally Scoped
Measures context architecture — not semantic quality, not model correctness, not factual truth.

---

## Performance Guarantees

| Payload Size | Max Execution Time |
|---|---|
| ≤ 5,000 tokens | < 2 seconds |
| ≤ 20,000 tokens | < 5 seconds |
| ≤ 50,000 tokens | < 10 seconds |

---

## Vision

ContextOps aims to make LLM context as **observable, measurable, and testable as source code.**

Just as modern software uses compilers, linters, and static analysis before deployment, LLM systems should validate the structural quality of their context before inference.

```
Retrieve
      │
      ▼
 Build Context
      │
      ▼
  ContextOps   ←── structural gate
      │
      ▼
     LLM
```

---

## Documentation

| Document | Contents |
|----------|----------|
| [HIGH_LEVEL_DOC.md](./HIGH_LEVEL_DOC.md) | Full technical reference — scoring engine, all CLI flags, API, ContextBench, ContextSecBench |
| [STABILITY.md](./STABILITY.md) | Formal stability contract, versioning policy, schema guarantees |
| [USER_GUIDE.md](./USER_GUIDE.md) | Getting started guide with practical examples |
| [HOW_TO_GET_JSON.md](./HOW_TO_GET_JSON.md) | How to capture context from your running application |
| [CONTRIBUTING.md](./CONTRIBUTING.md) | Contribution guide — setup, project structure, PR checklist |
| [CHANGELOG.md](./CHANGELOG.md) | Release history |

---

## Contributing

Contributions are welcome — bug fixes, new ContextBench samples, documentation improvements, and new analyzer signals.

Read **[CONTRIBUTING.md](./CONTRIBUTING.md)** for:

- Development setup and project structure
- How to run the test suite (including the chaos and signal contract tests)
- Rules for adding new signals or changing the scoring engine
- The PR checklist and commit style guide
- What the stability contract forbids

For anything non-trivial, open an issue first so we can align before you write code.

**Repository:** [github.com/Abhijeet777/contextops](https://github.com/Abhijeet777/contextops)

---

## License

Released under the [Sustainable Use License](./LICENSE).
