Metadata-Version: 2.4
Name: radlab-llm-router-plugins
Version: 0.1.4
Summary: Plugins for the LLM Router (guardrails, maskers, etc.)
Home-page: https://github.com/radlab-dev-group/llm-router-plugins
Author: RadLab.dev Team
Author-email: hello@radlab.dev
License: Apache-2.0
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: radlab-pii-classification>=0.1.0
Provides-Extra: memory
Requires-Dist: redis>=5; extra == "memory"
Provides-Extra: ml
Requires-Dist: faiss-cpu; extra == "ml"
Requires-Dist: sentence-transformers; extra == "ml"
Requires-Dist: numpy; extra == "ml"
Requires-Dist: scipy; extra == "ml"
Provides-Extra: ml-gpu
Requires-Dist: faiss-gpu; extra == "ml-gpu"
Requires-Dist: sentence-transformers; extra == "ml-gpu"
Requires-Dist: numpy; extra == "ml-gpu"
Requires-Dist: scipy; extra == "ml-gpu"
Provides-Extra: rag
Requires-Dist: langchain; extra == "rag"
Requires-Dist: Pillow; extra == "rag"
Provides-Extra: agentic-knowledge
Requires-Dist: sse-lib>=0.1.0; extra == "agentic-knowledge"
Requires-Dist: redis>=5; extra == "agentic-knowledge"
Dynamic: author
Dynamic: author-email
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

## Overview

The **LLM‑Router** project ships with a modular plugin system that lets you plug‑in **anonymizers** (also called
*maskers*) and **guardrails** into request‑processing pipelines.  
Each plugin implements a tiny, well‑defined interface (`apply`) and can be composed in an ordered list to form a *
*pipeline**. Pipelines are instantiated by the `MaskerPipeline` and `GuardrailPipeline` classes and are driven
automatically by the endpoint logic in `endpoint_i.py`.

---

## 1. Anonymizers (Maskers)

### 1.1 What they do

* **Goal** – Remove or replace personally‑identifiable information (PII) from a payload before it reaches the LLM or an
  external service.
* **Typical strategy** – Run a pipeline of maskers that locate spans corresponding to IDs, emails, IPs, etc., and
  replace each span with a placeholder such as `{{MASKED_ITEM}}`.

### 1.2 Built‑in anonymizer plugins

| Plugin                                         | Description                                                                                                                                       | Technical notes                                                                                                                                                            |
|------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **FastMaskerPlugin** (`fast_masker_plugin.py`) | Thin wrapper around the `FastMasker` utility class. Receives a JSON‑compatible payload and returns the same payload with all detected PII masked. | Implements `PluginInterface`. The heavy lifting is delegated to `FastMasker.mask_payload(payload)`. No extra I/O; the `FastMasker` instance is created once in `__init__`. |

### 1.3 How a masker is used

1. The endpoint (e.g. `EndpointI._do_masking_if_needed`) checks the global flag `FORCE_MASKING`.
2. If enabled, it creates a `MaskerPipeline` with the list of masker plugin identifiers (e.g. `["fast_masker"]`).
3. The pipeline calls each plugin’s `apply` method sequentially, feeding the output of one as the input of the next.
4. The final payload – now stripped of PII – proceeds to the rest of the request flow (guardrails, model dispatch,
   etc.).

---

## 2. Guardrails

### 2.1 What they do

* **Goal** – Verify that a request (or its response) complies with policy rules (e.g. no hateful, illegal, or unsafe
  content).
* **Typical strategy** – Split the payload into manageable text chunks, run a pipeline of guardrails, aggregate
  per‑chunk scores, and decide whether the overall request is safe.

### 2.2 Built‑in guardrail plugins

| Plugin                                             | Description                                                                                                                                                                                                                  | Technical notes                                                                                                                                                                                                                       |
|----------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **NASKGuardPlugin** (`nask_guard_plugin.py`)       | HTTP‑based guardrail that forwards the payload to the external NASK guardrail service (`/nask_guard` endpoint) and returns a boolean *safe* flag together with the raw response.                                             | Inherits from `HttpPluginInterface`. The `apply` method calls `_request(payload)` (provided by the base class) and extracts `results["safe"]`. Errors are caught and logged; on failure the plugin returns `(False, {})`.             |
| **SojkaGuardPlugin** (`sojka_guard_plugin.py`)     | HTTP‑based guardrail that forwards the payload to the **Sójka** guardrail service (`/sojka_guard` endpoint) and returns a safety flag.                                                                                       | Mirrors the design of `NASKGuardPlugin`. The `endpoint_url` is built from the `LLM_ROUTER_GUARDRAIL_SOJKA_GUARD_HOST` environment variable. On success it returns `(True, response)`, otherwise `(False, {})`.                        |
| **(Implicit) GuardrailProcessor** (`processor.py`) | Core logic used by the internal NASK guardrail Flask route (`nask_guardrail`). Tokenises the payload, creates overlapping chunks, runs a Hugging‑Face `text‑classification` pipeline, and produces a detailed safety report. | Handles model loading (`AutoTokenizer`, `pipeline("text‑classification")`), chunking (`_chunk_text`), and scoring thresholds (`MIN_SCORE_FOR_SAFE`, `MIN_SCORE_FOR_NOT_SAFE`). Returns a dict: `{"safe": <bool>, "detailed": [...]}`. |

### 2.3 How a guardrail is used

1. The endpoint calls `_is_request_guardrail_safe(payload)` (or the analogous response guardrail).
2. If `FORCE_GUARDRAIL_REQUEST` is true, a `GuardrailPipeline` is built from the configured plugin IDs (e.g.
   `["nask_guard", "sojka_guard"]`).
3. The pipeline iterates over each guardrail plugin; each `apply` returns `(is_safe, message)`.
4. The first plugin that reports `is_safe=False` short‑circuits the pipeline and the request is rejected with a 400/500
   error payload.

---

## 2.5 ML-Based PII Classification

For cases where regex patterns alone are insufficient (e.g. context-dependent PII detection), the project integrates
with the **anonymizer-model** repository:

**Repository**: [radlab-dev-group/anonymizer-model](https://github.com/radlab-dev-group/anonymizer-model)

| Feature               | Description                                                                                     |
|-----------------------|-------------------------------------------------------------------------------------------------|
| **Approach**          | NER (Named Entity Recognition) model based on Hugging Face `AutoModelForTokenClassification`    |
| **What it does**      | Identifies PII entities in Polish text with context-aware detection (not just pattern matching) |
| **Training pipeline** | Configurable via JSON, logs to W&B, exports best model (F1 macro) to `final_model/`             |
| **REST API**          | Flask service supporting multiple model versions with optional dynamic quantization             |
| **CLI**               | `pii-classifier convert` / `generalise` / `report` for data prep and analysis                   |
| **Inference**         | Sub-token merging, punctuation cleaning, gap preservation for human-readable entity spans       |
| **Web tester**        | HTML/JS interface for real-time PII detection testing                                           |

### How it complements regex maskers

| Approach                              | Strength                                                           | Best for                                                    |
|---------------------------------------|--------------------------------------------------------------------|-------------------------------------------------------------|
| **Regex maskers** (FastMasker)        | Deterministic, zero false negatives for known formats, fast        | Structured IDs (KRS, NIP, PESEL, NRB, REGON, VIN, etc.)     |
| **PII classifier** (anonymizer-model) | Context-aware, handles unknown formats, generalizes across domains | Free-text entities, names, addresses, context-dependent PII |

### Quick start

```bash
# Clone and install
git clone https://github.com/radlab-dev-group/anonymizer-model.git
cd anonymizer-model
pip install .

# Run the API (serves multiple model versions)
python3 -m pii_classification.api.app

# API endpoints
#   GET  /models   — list available models + default
#   POST /predict   — { "text": "...", "model": "optional_name" }
```

---

## 2.6 Polish Identification Regex Patterns

The **FastMaskerPlugin** ships with a comprehensive set of rules for detecting Polish business and personal identifiers.
These rules use regex matching + checksum/form validation to minimize false positives.

### Available Polish rules

| Rule                | Placeholder        | Format                                            | Validation                                           |
|---------------------|--------------------|---------------------------------------------------|------------------------------------------------------|
| **KrsRule**         | `{{KRS}}`          | `1234567890` or `123-456-78-90` / `123 456 78 90` | 10 digits (format-only)                              |
| **NrbRule**         | `{{NRB}}`          | `PL58105012981000009062923173` (26 digits)        | 26 digits                                            |
| **NipRule**         | `{{NIP}}`          | `1234567890` or `123-456-78-90`                   | Checksum with weights `[6,5,7,2,3,4,5,6,7]` mod 11   |
| **PeselRule**       | `{{PESEL}}`        | 11 digits                                         | Checksum with weights `[1,3,7,9,1,3,7,9,1,3]` mod 10 |
| **RegonRule**       | `{{REGON}}`        | 9 or 14 digits                                    | Checksum (different weights per form)                |
| **BankAccountRule** | `{{BANK_ACCOUNT}}` | `PL58 1050 1298 1000 0090 6292 3173`              | Polish IBAN (28 chars), supports masked `XX` groups  |

### How they work

Each rule follows the same pattern:

1. **Regex match** — detects candidate strings (e.g. 10-digit sequences for KRS)
2. **Checksum validation** — validates the candidate against the official algorithm
3. **Placeholder replacement** — valid matches are replaced with `{{PLACEHOLDER}}`; invalid ones are left untouched
4. **Optional anonymizer** — if an `anonymizer_fn` is provided, it's called with `(original, tag_type)` and its result (
   wrapped in `{}`) is used instead

### Adding rules to the pipeline

To enable Polish rules in your masking pipeline, add their plugin IDs to `MASKING_STRATEGY_PIPELINE`:

```python
MASKING_STRATEGY_PIPELINE = ["fast_masker"]
```

All Polish rules are included in the FastMasker plugin by default. For a full list of available rules, see
the [fast_masker README](llm_router_plugins/maskers/fast_masker/README.md).

---

## 2.7 Semantic Routing (Model Selection)

Four routing plugins are available for model selection. The two semantic plugins
(`simple_semantic_routing`, `semantic_biencoder_routing`) activate when `payload["model"] == "auto"`; the Codex plugin
(`agentic_routing_codex`) activates on `"auto_codex"`, routing the OpenAI-Responses-style requests emitted by the Codex
CLI agent by request class and work mode. The Claude Code plugin (`agentic_routing_claude_code`) decides by model name
alone, mapping the model IDs Claude Code asks for onto the models this router serves.

Sitting behind them, the **agentic knowledge layer** (2.7.5) is not a classifier: it injects repository
knowledge into an already-routed request and can escalate its model when the agent is looping with no known fix.

### 2.7.1 Simple Semantic Routing (Heuristic)

The **Simple Semantic Routing plugin** (`simple_semantic_routing`, `utils/routing/simple_semantic/`) activates on
`payload["model"] == "auto"` and rewrites it in three dependency-free stages: **intent** scoring over the last user
message — weighted keywords (case-insensitive substrings), phrases with a `:weight` suffix and regex patterns (+3.0
each) — giving `code` / `math` / `creative` / `general`, or `none` when nothing scores; **complexity** from a word-count
estimate (`int(words * 1.25)`) against the `simple` / `medium` thresholds; and **model selection** from an ordered pool
where index 0 is the cheapest and index `n-1` the strongest. `intent_adjustment` only ever escalates the intents that
declare one; an intent with an empty adjustment is demoted a single pool step. Only `payload["model"]` is written, so
the decision lives in the log:
`SimpleSemanticRouting: intent=code, complexity=simple (10 tokens) -> qwen3.6:35b`.

Configuration ships in
[simple_semantic.json](llm_router_plugins/resources/routing/simple_semantic.json). Overrides:
`LLM_ROUTER_ROUTING_COMPLEXITY_THRESHOLDS` (`simple|medium`), `LLM_ROUTER_ROUTING_MODELS` (the pool),
`LLM_ROUTER_ROUTING_DEFAULT_MODEL` and `LLM_ROUTER_ROUTING_INTENT_<name>` — that last one **replaces** an intent and
clears its patterns and weights, and there is no `..._CONFIG` variable for this plugin.

**Full documentation** — stage-by-stage mechanics, measured intent × complexity → model matrices for two- and
three-model pools, configuration reference, tuning, gotchas and a runnable verification snippet:
[Simple Semantic Routing README](llm_router_plugins/utils/routing/simple_semantic/README.md).

### 2.7.2 Bi-Encoder Semantic Routing (Model Selection)

The **Bi-Encoder routing plugin** (`semantic_biencoder_routing`, `utils/routing/semantic_biencoder/`) also answers
`payload["model"] == "auto"`, but decides by **embedding similarity**. Each routing target is indexed at startup as
`"Target: {name}. {description}"` plus its `examples`, split into overlapping token chunks, embedded by a
sentence-transformers BiEncoder (`google/embeddinggemma-300m` by default), L2-normalized and stored in a
`faiss.IndexFlatIP`; a docstore maps doc IDs back to target names. At request time the last user message is embedded
(text longer than the model's `max_seq_length` is windowed and averaged, capped at `MAX_QUERY_WINDOWS = 4`), the
`top_k` hits are **averaged per target**, and the winning target's `model_name` is selected when its cosine reaches
`similarity_threshold`; the payload gains `routing.target_name` and `routing.similarity`. The index is persisted as
`index.faiss` + `docstore.pkl` and reloaded on the next start, rebuilt automatically when the embedding model outputs a
different dimension. Needs the `[ml]` / `[ml-gpu]` extras — and it fails hard rather than passing traffic through.

Configuration ships in [semantic_biencoder.json](llm_router_plugins/resources/routing/semantic_biencoder.json).
Overrides: `LLM_ROUTER_ROUTING_SEMANTIC_BIENCODER_CONFIG` (inline JSON *or* a path), `_MODEL`, `_TARGETS`,
`_CHUNK_SIZE`, `_CHUNK_OVERLAP`, `_PERSIST_DIR`; `similarity_threshold` and `top_k` are config-only. The shipped
`similarity_threshold: 0.0` reroutes **every** request — measured on the shipped targets, real matches score 0.70–0.82
while unrelated or nonsense text still scores ≈0.35–0.37, so `0.5`–`0.6` is the usual production band.

**Full documentation** — startup and query mechanics, measured similarities per target, configuration reference, failure
modes, tuning, gotchas (persisted-index staleness included) and a verification recipe:
[Bi-Encoder Routing README](llm_router_plugins/utils/routing/semantic_biencoder/README.md).

### 2.7.3 Codex CLI routing (`auto_codex`)

The **Codex Routing plugin** (`agentic_routing_codex`, `utils/routing/agentic_routing/codex/`) routes the
OpenAI-Responses-style requests emitted by the **Codex CLI** coding agent. It activates only when `payload["model"]`
equals its configured trigger — `"auto_codex"` by default — so it never intercepts the `"auto"` traffic owned by the
two semantic plugins. It rewrites `payload["model"]` and adds `payload["agent_mode"]` plus a `payload["routing"]`
block, and nothing else; routing **fails open**, so a payload it cannot classify is passed through untouched.

A work mode is resolved by a deterministic cascade — an explicit `agent_mode` override, the request class
(`compaction`, `aux_title`), the `<collaboration_mode>` Plan block injected by the CLI, PL+EN keyword scoring, then an
optional embedding cosine-similarity lookup — and falls back to `implement`:

| Mode         | Model                     | Routed by                                      |
| ------------ | ------------------------- | ---------------------------------------------- |
| `plan`       | `qwen/Qwen3.8-Flash-Next` | `<collaboration_mode>` Plan Mode block         |
| `implement`  | `qwen/Qwen3.8-Flash-Next` | Fallback for a plain main turn                 |
| `test`       | `qwen/Qwen3.8-27B`        | Keywords in the classified user text           |
| `git_review` | `qwen/Qwen3.8-27B`        | Keywords in the classified user text           |
| `review`     | `qwen/Qwen3.8-Flash-Next` | Keywords in the classified user text           |
| `debug`      | `qwen/Qwen3.8-Flash-Next` | Keywords in the classified user text           |
| `aux_title`  | `qwen/Qwen3.8-27B`        | Request class — system thread or title shape    |
| `compaction` | `qwen/Qwen3.8-27B`        | Request class — context compaction             |

Modes ship in
[agentic_routing_codex.json](llm_router_plugins/resources/routing/agentic_routing_codex.json) and are overridable with
`LLM_ROUTER_ROUTING_SEMANTIC_AGENTIC_CODEX_*` environment variables (`…_CONFIG`, `…_TRIGGER`, `…_MODEL_<MODE>`,
`…_MODELS`, `…_MODES`, `…_FALLBACK_MODE`, `…_HEURISTIC_ENABLED`, `…_HEURISTIC_MIN_SCORE`, `…_CLASSIFY_MAX_CHARS`,
`…_MODEL`, `…_SEMANTIC_ENABLED`, `…_SIMILARITY_THRESHOLD`, `…_TOP_K`, `…_CHUNK_SIZE`, `…_CHUNK_OVERLAP`,
`…_PERSIST_DIR`, `…_MODE_<name>_KEYWORDS`). `MODES` filters the mode list but does not move the fallback: if the
whitelist excludes the configured `fallback_mode` (`implement`), set `FALLBACK_MODE` as well — otherwise configuration
validation fails at startup.

**Full documentation** — the cascade and scoring model, installation, loading the plugin into the router, model
selection and its pitfalls, tuning, a verification recipe and a troubleshooting table — lives in the
[Codex CLI Routing README](llm_router_plugins/utils/routing/agentic_routing/codex/README.md).

**Calibration and evaluation guides** live in the same `llm_router_plugins/utils/routing/agentic_routing/codex/`
directory: [Polski — CODEX_EVAL_HOWTO_PL.md](llm_router_plugins/utils/routing/agentic_routing/codex/CODEX_EVAL_HOWTO_PL.md)
and [English — CODEX_EVAL_HOWTO_EN.md](llm_router_plugins/utils/routing/agentic_routing/codex/CODEX_EVAL_HOWTO_EN.md).
They explain configuration roles, heuristic scores and embedding similarity, threshold/margin tuning, calibration/holdout
datasets, the evaluation CLI and report metrics, and deployment/index rebuilding. They also document the automated
[`scripts/codex-tune-eval.sh`](scripts/codex-tune-eval.sh) workflow; quality is evaluated by work mode, independently of model assignments.

### 2.7.4 Claude Code Model Swap (`claude-*` → configured models)

The **Claude Code Model Swap plugin** (`agentic_routing_claude_code`, `utils/routing/agentic_routing/claude_code/`)
rewrites the model Claude Code asks for into the model the operator serves. Claude Code resolves its `fable` / `opus` /
`sonnet` / `haiku` aliases client-side and sends a versioned ID, so pointing it at your own backend means overriding one
environment variable per tier (`ANTHROPIC_DEFAULT_FABLE_MODEL`, `ANTHROPIC_DEFAULT_OPUS_MODEL`,
`ANTHROPIC_DEFAULT_SONNET_MODEL`, `ANTHROPIC_DEFAULT_HAIKU_MODEL`, `CLAUDE_CODE_SUBAGENT_MODEL`) on every host. This
plugin moves that mapping into one config file: a **tier** names the model IDs it answers and the model to serve them
with, and a matching request gets the target written into `payload["model"]` (and `payload["model_name"]`, when the
payload carries it) plus a `payload["routing"]` block.

| Tier     | Model IDs Claude Code sends                        | Setting it replaces             |
| -------- | -------------------------------------------------- | ------------------------------- |
| `fable`  | `claude-fable-5-1`, `claude-fable-5`, `best`       | `ANTHROPIC_DEFAULT_FABLE_MODEL` |
| `opus`   | `claude-opus-5-5` … `claude-opus-4-6`              | `ANTHROPIC_DEFAULT_OPUS_MODEL`  |
| `plan`   | `opusplan`                                         | the `model: "opusplan"` phase   |
| `sonnet` | `claude-sonnet-5`, `claude-sonnet-4-6`, `-4-5`     | `ANTHROPIC_DEFAULT_SONNET_MODEL`|
| `haiku`  | `claude-haiku-4-5`, `claude-3-5-haiku-*`           | `ANTHROPIC_DEFAULT_HAIKU_MODEL` |

Matching is content-free and deterministic: the requested name is normalized (window suffix `[1m]`, provider version
`:0`, `@YYYYMMDD` revision, `us.anthropic.` / `anthropic/` prefix, `models/` path and `-YYYYMMDD` release date stripped)
and resolved through four layers, most specific first — literal, normalized exact, longest wildcard prefix
(`claude-opus-*`), then family token (`claude-3-5-haiku-20241022` → Haiku). Configuration ships in
[agentic_routing_claude_code.json](llm_router_plugins/resources/routing/agentic_routing_claude_code.json) and is
overridable with `LLM_ROUTER_ROUTING_SEMANTIC_AGENTIC_CLAUDE_CODE_*` (`…_CONFIG`, `…_ENABLED`, `…_MATCH_FAMILIES`,
`…_FIELDS`, `…_MODEL_<TIER>`, `…_MODELS`, `…_MODES`, `…_MODE_<tier>_MODELS`). Routing **fails open and silent**: a model
no tier claims comes back as the same object it arrived as, with no log line, while an inconsistent config (duplicate tier
names, an embedded or bare wildcard, one exact name claimed twice) raises at startup. No ML dependencies.

**Full documentation** — the normalization and match layers, installation, router and CLI wiring, tier configuration,
linting, gotchas, a verification recipe and a troubleshooting table — lives in the
[Claude Code Model Swap README](llm_router_plugins/utils/routing/agentic_routing/claude_code/README.md).

### 2.7.5 Agentic knowledge layer (`agentic_knowledge`)

The **Agentic knowledge layer** (`agentic_knowledge`, `utils/knowledge/agentic_knowledge/`) stands
*behind* the Codex routing plugin and enriches a request with what the repository has already
learned, **without the agent cooperating**. It reads the routing decision (`routing`,
`agent_mode`, `model`), searches the **Semantic Search Engine** for verified fixes, dead ends and
the pre-compaction archive of the current scope, and injects the result as a separate, removable
context block. It also learns from the traffic: the miner turns a `test_failure → patch → test
pass` episode into a `recipe`, a `patch → test pass` episode that no failure opened into a
`playbook` — the correct steps of a task are knowledge too, and without this kind a turn that went
right from the first try wrote nothing at all — and a repeated failure into a `deadend`, publishes
them to a Redis Stream, and an indexing worker masks them with `FastMasker` before indexing; the
window a `compaction` request is about to summarize is archived whole. Every mined episode also carries the
work **phase** its evidence names (`test`, `implement`, …), and a retrieval source can be
restricted to phases in its policy, so a mode can read only what was learned while doing its kind
of work. The retrieval result is a routing signal: an agent looping on one error signature with no
known fix is moved to a configured, stronger model (*knowledge-or-escalate*, **off by default**).

The repository's own documentation joins the same knowledge base: a scan reads the `*.md` files below
the directory the request declared, splits them at their headings, and publishes them as `doc`
documents in a collection of their own (`<prefix>_<scope>_docs`), searched on the turn's subject
rather than on its failure signature. Each work mode reads the kinds it wants — `implement` the
readme, guides and contributing, `plan` those plus the changelog and notes, `review` the guides,
contributing and security — and a guide that documents a file the turn is editing outranks one that
does not. On top of retrieval the session carries a **standing map** of that documentation — which
files exist and what they cover, structural only (paths, kinds, headings), stable per repository and
budgeted before anything is retrieved — so an agent stops spending commands rediscovering what a
project has written down, and a mined fix or dead end points at the page that documents the module
it touched (`src/export.py` → `docs/export.md`), so a recipe arrives with the rule it has to keep.
A file is republished when its content changes and the document it
superseded is retired; a scan that ran out of its ceilings publishes no retirement at all.
`llm-router-agentic-kb-docs` does the same scan on demand (`--dry-run` to see what would be
published), which is also how a repository whose router runs on another machine gets documentation.

The engine may be reached by a pre-issued token or by a username/password login
(`SSE_API_USERNAME`/`SSE_API_PASSWORD`; the client logs in on its first call).

The worker is a separate process the operator starts —
`scripts/plugins/agentic_knowledge/run-agentic-kb-worker.sh` in the `llm-router` checkout takes its
environment from the router's own run script. It creates a repository's collection on that repository's
first event; a project that has produced nothing yet simply has an empty knowledge base.

A mode routing did not resolve falls back to the `"*"` policy rather than to no search at all, and
the read budget (`read.timeout_seconds`) is a *sum* of engine calls — one per collection a mode
reads — not a per-call timeout, which is set apart by `SSE_API_TIMEOUT`.

Connection settings (SSE and Redis) come only from `LLM_ROUTER_KNOWLEDGE_AGENTIC_*`; the policy
(adapters, per-mode sources, weights and budgets, escalation models, retention) is
[agentic_knowledge.json](llm_router_plugins/resources/knowledge/agentic_knowledge.json). No SSE
host means no reads, no Redis means no writes and a per-process cache instead of the shared one, and every failure is
**fail-open** — the request passes through untouched. Codex is served fully (reads and writes);
Claude Code is served in a basic, read-only form: scope, latest user text, injection into
`system`.

**Full documentation** — the read and write paths, per-mode policies, the generation cache,
knowledge-or-escalate, configuration and environment tables, the worker, privacy and retention,
stateless mode, offline evaluation and limitations — lives in the [Agentic knowledge layer
README](llm_router_plugins/utils/knowledge/agentic_knowledge/README.md).

---

## 3. Pipelines

Both masker and guardrail pipelines share the same design pattern:

| Class                                                     | Purpose                                                                            |
|-----------------------------------------------------------|------------------------------------------------------------------------------------|
| **MaskerPipeline** (`pipeline.py` – masker version)       | Executes a list of masker plugins in order, transforming the payload step‑by‑step. |
| **GuardrailPipeline** (`pipeline.py` – guardrail version) | Executes guardrail plugins sequentially, stopping on the first failure.            |

### 3.1 Registration

* Plugins are registered lazily via `MaskerRegistry.register(name, logger)` or
  `GuardrailRegistry.register(name, logger)`.
* The registry maps a string identifier (e.g. `"fast_masker"`) to a concrete plugin class, allowing pipelines to resolve
  the classes at runtime.

### 3.2 Configuration

All plugin identifiers are stored in environment variables or constants such as:

```python
MASKING_STRATEGY_PIPELINE = ["fast_masker"]
GUARDRAIL_STRATEGY_PIPELINE_REQUEST = ["nask_guard", "sojka_guard"]
```

These lists are consumed by the endpoint initialization (`EndpointI._prepare_masker_pipeline`,
`EndpointI._prepare_guardrails_pipeline`).

---

## 4. Adding a New Plugin

1. **Create a subclass** of either `PluginInterface` (for maskers) or `HttpPluginInterface` / a custom guardrail base.
2. **Define a `name` class attribute** – this is the identifier used in pipeline configuration.
3. **Implement `apply(self, payload: Dict) -> Dict`** (masker) **or `apply(self, payload: Dict) -> Tuple[bool, Dict]`
   ** (guardrail).
4. **Register the plugin** – either automatically via the registry’s `register` call in the pipeline constructor, or
   manually by calling `MaskerRegistry.register(name=MyPlugin.name, logger=logger)`.

*Example stub for a new masker:*

```python
# my_custom_masker.py
from llm_router_plugins.maskers.plugin_interface import PluginInterface
import logging
from typing import Dict, Optional


class MyCustomMasker(PluginInterface):
    name = "my_custom_masker"

    def __init__(self, logger: Optional[logging.Logger] = None):
        super().__init__(logger=logger)
        # Load any heavy resources here (e.g., a spaCy model)

    def apply(self, payload: Dict) -> Dict:
        # Perform your masking logic and return the modified payload
        return payload
```

After placing the file in `llm_router_plugins/maskers/plugins/`, enable it by adding `"my_custom_masker"` to
`MASKING_STRATEGY_PIPELINE`.

---

## 5. Retrieval‑Augmented Generation (RAG) Support

The project now includes a **LangChain‑based RAG plugin** that enables semantic search over user‑provided documents. The
implementation lives in `llm_router_plugins/utils/rag/langchain_plugin.py` and is driven by the helper CLI scripts
located in `scripts/`.

### 5.1 What the plugin does

| Feature           | Description                                                                                                                                                                                                                             |
|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| **Indexing**      | Reads a directory of text‑like files (`.txt`, `.md`, `.html`, `.js`, …), splits them into token‑based windows, embeds each chunk with a configurable transformer model, and stores the vectors in a FAISS (or compatible) vector store. |
| **Searching**     | Given a user query, retrieves the most similar chunks and injects them into the payload (e.g., appends to the last user message) so that downstream LLM calls can use the retrieved context.                                            |
| **Configuration** | All parameters (collection name, embedder model, device, chunk size, overlap, persistence directory) are driven by environment variables prefixed with `LLM_ROUTER_`. See the table below for the full list.                            |
| **CLI helpers**   | Two ready‑to‑use scripts: `scripts/llm-router-rag-langchain-index.sh` (indexes a repository) and `scripts/llm-router-rag-langchain-search.sh` (runs a search or starts an interactive REPL).                                            |

### 5.2 Environment variables

| Variable                                 | Default                                                                        | Meaning                                                     |
|------------------------------------------|--------------------------------------------------------------------------------|-------------------------------------------------------------|
| `LLM_ROUTER_LANGCHAIN_RAG_COLLECTION`    | *must be set*                                                                  | Name of the FAISS collection (e.g. `sample_collection`).    |
| `LLM_ROUTER_LANGCHAIN_RAG_EMBEDDER`      | `/mnt/data2/llms/models/community/google/embeddinggemma-300m`                  | Path or Hugging‑Face identifier of the embedding model.     |
| `LLM_ROUTER_LANGCHAIN_RAG_DEVICE`        | `cuda:2`                                                                       | Torch device (`cpu`, `cuda:0`, …).                          |
| `LLM_ROUTER_LANGCHAIN_RAG_CHUNK_SIZE`    | `1024`                                                                         | Number of tokens per chunk.                                 |
| `LLM_ROUTER_LANGCHAIN_RAG_CHUNK_OVERLAP` | `100`                                                                          | Number of overlapping tokens between consecutive chunks.    |
| `LLM_ROUTER_LANGCHAIN_RAG_PERSIST_DIR`   | `./workdir/plugins/utils/rag/langchain/${LLM_ROUTER_LANGCHAIN_RAG_COLLECTION}` | Directory where the FAISS index and docstore are persisted. |

#### Example export block (add to your shell profile or a `.env` file)

```shell script
export LLM_ROUTER_LANGCHAIN_RAG_COLLECTION="${LLM_ROUTER_LANGCHAIN_RAG_COLLECTION:-sample_collection}"
export LLM_ROUTER_LANGCHAIN_RAG_EMBEDDER="${LLM_ROUTER_LANGCHAIN_RAG_EMBEDDER:-/mnt/data2/llms/models/community/google/embeddinggemma-300m}"
export LLM_ROUTER_LANGCHAIN_RAG_DEVICE="${LLM_ROUTER_LANGCHAIN_RAG_DEVICE:-cuda:2}"
export LLM_ROUTER_LANGCHAIN_RAG_CHUNK_SIZE="${LLM_ROUTER_LANGCHAIN_RAG_CHUNK_SIZE:-1024}"
export LLM_ROUTER_LANGCHAIN_RAG_CHUNK_OVERLAP="${LLM_ROUTER_LANGCHAIN_RAG_CHUNK_OVERLAP:-100}"
export LLM_ROUTER_LANGCHAIN_RAG_PERSIST_DIR="${LLM_ROUTER_LANGCHAIN_RAG_PERSIST_DIR:-./workdir/plugins/utils/rag/langchain/${LLM_ROUTER_LANGCHAIN_RAG_COLLECTION}}"
```

### 5.3 Using the CLI scripts

**Index a repository** (example for the documentation site):

```shell script
scripts/llm-router-rag-langchain-index.sh
# Internally runs:
# llm-router-rag-langchain index --path "../.github/pages/llmrouter.cloud/" --ext .html .js .md
```

**Search** (interactive REPL):

```shell script
scripts/llm-router-rag-langchain-search.sh
# Internally runs:
# llm-router-rag-langchain search
# (you will be prompted for a query, type “exit” to quit)
```

**One‑shot search**:

```shell script
llm-router-rag-langchain search --query "What is Retrieval‑Augmented Generation?" --top_n 5
```

The CLI returns the raw matching chunks together with similarity scores. The `LangchainRAGPlugin` automatically formats
the retrieved text and appends it to the user’s last message, prefixed with:

```
If the context below will help answer the above question, use it.
Context separated with double enter
```
