Metadata-Version: 2.5
Name: norcaster-litellm
Version: 0.2.1
Summary: Norcaster check for LiteLLM: your LiteLLM proxy asks Norcaster for a decision (allow, hide or block) before it calls the AI provider, on chat completions, the Responses API and Anthropic messages, and Norcaster keeps a record of every answer.
Project-URL: Homepage, https://www.norcaster.com
Author-email: "Norcaster (Navego AB)" <security@norcaster.com>
License: Apache-2.0
License-File: LICENSE
Keywords: ai-gateway,ai-governance,audit-evidence,guardrail,litellm
Requires-Python: >=3.12
Requires-Dist: litellm[proxy]<2,>=1.108.0
Requires-Dist: norcaster<0.2,>=0.1.5
Provides-Extra: test
Requires-Dist: pytest>=7; extra == 'test'
Description-Content-Type: text/markdown

# norcaster-litellm

A plug-in for a [LiteLLM](https://github.com/BerriAI/litellm) proxy you already run. Before a
request leaves the proxy, the plug-in asks Norcaster for a decision: a chat completion, a request to
OpenAI's Responses API, or an Anthropic messages request. Norcaster answers from the app's rules and
keeps a record of every answer. The proxy then calls the AI provider itself: your apps keep pointing
at LiteLLM, and your provider keys stay where they are.

| Norcaster's decision | What the proxy does |
|---|---|
| **allow** | Sends the request to the provider as it is. |
| **hide** | Sends the version Norcaster returned, with the sensitive data hidden. |
| **block** | Does not call the provider. The caller gets an error that names the decision and the record. |

**Status:** new. Tested with LiteLLM 1.108.0 and 1.109.1.

## What the record shows

The app asked Norcaster before it called the provider. Here the proxy asks on the app's behalf.

What it does not show:

- **What the proxy sent afterwards.** Norcaster does not see the provider call. The record holds a
  SHA-256 of what was checked, so you can compare it with what your proxy logged.
- **That every request was checked.** Only requests that reach this plug-in are. A request that
  goes to the provider another way (another proxy, a direct call, a route where the plug-in is not
  turned on) leaves no record.
- **Anything about the provider's answer,** unless you turn on
  [output checks](#checking-the-models-answer-too). Without them only what the app sends is checked.

If you need a record that nothing reached the provider unchecked, send the app's traffic through
the Norcaster gateway instead.

The request is sent to Norcaster to be checked. If your data may not leave your network, this
plug-in does not change that; a Norcaster you host yourself does.

## Set up

You need a Norcaster key with the `runs:write` scope and the id of the app in Norcaster.

1. Install the plug-in into the Python environment that runs LiteLLM. It brings the Norcaster SDK
   (`norcaster`) with it:

   ```bash
   pip install norcaster-litellm
   ```

2. Add the plug-in to the proxy's config file ([`examples/config.yaml`](examples/config.yaml)):

   ```yaml
   guardrails:
     - guardrail_name: "norcaster-support-assistant"
       litellm_params:
         guardrail: norcaster_litellm.NorcasterCheck
         mode: "pre_call"
         default_on: true
         api_key: os.environ/NORCASTER_API_KEY
         api_base: https://api.norcaster.com
         deployment_id: "<the app's id>"
   ```

3. Start the proxy with the key in its environment, and send one request:

   ```bash
   NORCASTER_API_KEY=... litellm --config config.yaml
   ```

   The response carries `x-norcaster-decision: allow` and `x-norcaster-receipt-id`. The same
   request is in the app's records in Norcaster, marked **Checked**. If the headers are missing, the
   plug-in is not running: see [If the plug-in is not running](#if-the-plug-in-is-not-running).

## Settings

| In the config file | Environment variable | Default | What it is |
|---|---|---|---|
| `api_key` | `NORCASTER_API_KEY` | none, required | The Norcaster key. Use `os.environ/NORCASTER_API_KEY`; never write the key in the file. |
| `api_base` | `NORCASTER_API_URL` | `https://api.norcaster.com` | Your Norcaster address. |
| `deployment_id` | `NORCASTER_DEPLOYMENT_ID` | the key's app | The app's id. Leave it out only when the key belongs to one app. |
| `when_unreachable` | `NORCASTER_WHEN_UNREACHABLE` | `block` | `block` or `let_through`. See below. |
| `timeout` | | `10` | Seconds to wait for Norcaster's answer. |
| `check_output` | `NORCASTER_CHECK_OUTPUT` | `false` | `true` also asks about the model's answer. See [Checking the model's answer too](#checking-the-models-answer-too). |
| `mode` | | | Must be `pre_call`. With `check_output` on it must be `["pre_call", "post_call"]`. |

The config file wins over the environment. LiteLLM's own `unreachable_fallback` setting is not
used; `when_unreachable` is.

**One entry per app.** Each entry has its own `deployment_id` and its own `when_unreachable`, so
each app gets its own rules and its own records. Give each entry its own `guardrail_name`.

## Which requests are checked

Three kinds, streamed or not. The check runs before the stream starts.

| Request | Route | What is checked |
|---|---|---|
| Chat completion | `/v1/chat/completions` | The messages. |
| OpenAI's Responses API | `POST /v1/responses` (also `/responses`, `/openai/v1/responses`) | `instructions`, as a leading system message, then each input item. |
| Anthropic messages | `POST /v1/messages` | `system`, as a leading system message, then the messages. |

A Responses or Anthropic request is read the way the Norcaster SDK and the Norcaster gateway read
one, so a receipt carries the same hash whichever of them asked. A hide goes back into the same
places: `instructions` and the input items; `system` and the messages.

- **Reasoning items** in a Responses request are checked without their encrypted state, and sent
  back exactly as they came. A hide that would change one is not sent (HTTP 503).
- **Thinking the provider signed** in an Anthropic request is left out of the check and sent back
  untouched: the signature covers it as it was.
- **Earlier turns the provider keeps** (`previous_response_id`, `conversation`, an
  `item_reference`) are not in the request, so they are not checked again. Each request checks
  what the app sends in it.
- **Earlier turns LiteLLM keeps are refused.** On a model LiteLLM serves through another
  provider's API, `previous_response_id` makes LiteLLM rebuild the earlier turns itself, from the
  requests it kept, and it keeps each one as the app sent it, before any hide. So while LiteLLM
  keeps requests (`store_prompts_in_spend_logs`, or the `STORE_PROMPTS_IN_SPEND_LOGS` variable, or
  cold storage through `cold_storage_custom_logger`), a Responses request with
  `previous_response_id` is refused with HTTP 400, on every model. Send the earlier turns in the
  input, or turn that setting off. With none of them on, `previous_response_id` goes on as it is.
- **LiteLLM may then send the request on in another form:** a Responses request to an Anthropic
  model, or an Anthropic request to an OpenAI model. The check is of the request as the app wrote
  it.

**Any other request the plug-in is applied to is refused** with HTTP 400 and is not sent: text
completions, embeddings, images, reading back, listing, deleting, cancelling or compacting a stored
response, a turn on LiteLLM's Responses websocket, LiteLLM's Cursor route, pass-through routes
(`/anthropic/...`) and the rest. Norcaster is not asked about them, so they leave no record. They
are refused so that an app cannot skip the check by calling another route on the same proxy. A
stored response is refused on purpose: the provider keeps the answer it gave, not the version the
caller got after a hide.

**Two routes no guardrail sees.** LiteLLM runs no guardrail on `/v1/responses/input_tokens` and
`/v1/messages/count_tokens`, so the plug-in can't refuse them, and they send the request's content
to the provider to be counted. Keep the app's keys off them (LiteLLM's allowed routes per key).

**A request that carries content a check doesn't read is refused too**, with HTTP 400, before
Norcaster is asked. LiteLLM would send it to the provider beside or in place of what was checked.

- **Another kind of request's content.** On a chat completion: a top-level `system` (put it in the
  messages, as a system message), `instructions`, `prompt` or `input`. On a Responses request:
  `system` or `messages`. On an Anthropic request: `instructions`, `prompt` or `input`. LiteLLM
  sends some of them on when it turns one kind of request into another. A chat completion's
  `instructions`, for example, reaches OpenAI as the system prompt on a model LiteLLM sends through
  the Responses API.
- **`extra_body`** that holds `messages`, `system`, `input`, `instructions` or `prompt`, or that is
  not an object. Other keys in `extra_body` are sent as they are.
- **A predicted output** (`prediction`, OpenAI's Predicted Outputs, also inside `extra_body`). The
  provider gets its text, usually the same file a message holds, and a check doesn't read it, so a
  hide in that file would go out unhidden there.
- **An MCP tool LiteLLM runs itself**: `{"type": "mcp"}` with a `server_url` of `litellm_proxy`, or
  one ending in `/mcp/<name>`, which LiteLLM can take for one of its own servers. LiteLLM would run
  the tool and call the provider again with its results, and that second call is never checked. An
  MCP server the provider itself calls goes on as it is.
- **A prompt stored at OpenAI** (`prompt` on a Responses request). Its template can add
  instructions and messages the check never sees.

This matters for `default_on: true`, which applies the plug-in to every request to the proxy. Keep
it when the proxy serves only the three kinds above. Otherwise leave `default_on` out and attach the
entry where the app's requests go, the way your LiteLLM version offers (by key, by team, or per
request). If the app names the guardrail in each request, the app can also leave it out.

### Images, files and audio

A request can hold images, documents and audio: chat `image_url`, `file` and `input_audio`;
Responses `input_image`, `input_file`, `input_audio`, computer screenshots and the files its tools
are given; Anthropic `image` and `document` (inline, by URL, from the Files API, or inside a tool
result). They meet the app's rules for them, the same rules the Norcaster gateway applies: allow,
warn, block, or "scan required" (which blocks, since nothing reads them), and limits on how many,
how large, how many pixels and which media types. With no rules set, they are allowed.

Norcaster never gets the file. The plug-in sends a description of each inline one in its place (the
SHA-256 of its bytes, their size, the media type its first bytes show, an image's width and height)
and the request's shape, as the Norcaster SDK's wrappers do. URLs and file ids are sent as they are.
On hide, the provider gets the hidden text and the same files: a hide never changes a file. The
record lists each file as not read, and its hash covers each file's SHA-256. This needs a Norcaster
(or check engine) from 19 November 2026 or later; an older one refuses the check (HTTP 503,
`refused`) and the request is not sent.

Two things to know:
- **What was checked is what the app sent.** LiteLLM can turn an image URL into bytes for some
  providers after the check (it fetches the image to send to Anthropic). The check saw the URL, as
  Norcaster's gateway would; the app's rule for files by reference (`refs: block`) still blocks it.
- **Shapes no rule covers.** LiteLLM's `video_url` (video, for Gemini) is not one of the rules' kinds
  (image, document, audio): it is passed on as an unknown part, as on the gateway. A PDF sent as an
  `image_url` data URL counts as an image, not a document, as on the gateway.

## Checking the model's answer too

Off unless you turn it on. Turned on, the plug-in also asks Norcaster about the model's answer after
the provider call, before the caller gets it. Norcaster records that decision under the request's
id, so one call reads as one request with two decisions.

```yaml
      mode: ["pre_call", "post_call"]
      check_output: true
```

Both lines are needed: the mode is what makes LiteLLM call the plug-in after the provider call. If
the two disagree, the proxy does not start. The app's rules in Norcaster must cover answers.

| Norcaster's decision | What the caller gets |
|---|---|
| **allow** | The answer as the provider gave it. |
| **hide** | The answer with Norcaster's text in place of the provider's, and each tool call's arguments with the sensitive values hidden. |
| **block** | No answer, its tool calls included. An error (HTTP 400) that names the decision and the record. |

The plug-in asks about the answer's text and the tool calls it asks for (each function's name and
its arguments), so the caller gets the tool calls only after Norcaster has decided, before it can
run any of them. A hidden tool call keeps its name, and its arguments stay JSON of the same shape
(the same keys and value types). A placeholder can change what a tool does: a mail tool would get
`[REDACTED: EMAIL]` as the address. If a caller's tool can't run with a placeholder, treat hide as
block there.

What this record shows: the proxy asked Norcaster about a text and tool calls before it handed the
answer over, and which tools (never their arguments). It does not show that they were the
provider's answer, that the caller got the hidden version, or which tools it ran.

**Streamed answers are held until they are checked.** With output checks on, a request with
`stream: true` works, but the caller gets nothing until the whole answer has been checked. Nothing
is checked while it streams. The plug-in reads the provider's whole stream, asks Norcaster once
about the whole answer's text and its tool calls (each put together from its pieces), and only then
passes the stream's parts on:

- **allow**: the parts as the provider sent them.
- **hide**: the same parts, with Norcaster's text where the answer's text starts and the later
  pieces of text empty, and each tool call's hidden arguments in its first piece, its later pieces
  empty. A Responses stream carries the text and the arguments more than once (in pieces, then
  whole, then in the finished response): every copy gets the hidden version, and a citation that
  pointed into the original text is dropped. An Anthropic stream is written out again from its
  events; with allow it is passed on as the provider's own bytes.
- **block**, or an answer withheld for another reason in the table below, before the first part,
  in each route's own way:
  - **Chat completions:** an error with the table's HTTP status. LiteLLM sends it as one
    server-sent event (`data: {"error": ...}`) rather than JSON, so an OpenAI client raises on the
    status and the reason is in the response body.
  - **Responses API:** HTTP 200 and one `response.failed` event, whose `error.message` holds the
    reason (LiteLLM writes it there), then the end of the stream. None of the answer.
  - **Anthropic messages:** HTTP 200 and one Anthropic `error` event (`event: error`), whose
    `error.message` names the decision and the record, with the details beside it in `norcaster`.
    Anthropic's clients raise on it. It is an event, not an error status, because LiteLLM sends
    keep-alive pings on `/v1/messages` while the answer is held (every 15 seconds by default), and
    with the first ping the status. An error status after it would end the connection without a
    word, and the client would try again.

What it costs: the wait for the first part becomes the time the provider takes to finish the whole
answer, plus one round trip to Norcaster. A caller that streams so that people see the first words
early loses that while output checks are on.

LiteLLM sends a streamed response's headers before the answer exists, so a streamed answer's
decision is not in the `x-norcaster-answer-*` headers. It is on LiteLLM's log entry for the request,
in Norcaster's records, and in the error when the answer is withheld. The request's own headers
are there as for any call. If the provider's stream fails part-way, the caller gets the provider's
error and none of the text before it, and Norcaster is not asked.

**What is read.** The answer's text and the tool calls the caller's app would run; nothing else.

| Request | The text | The tool calls |
|---|---|---|
| Chat completion | the message's `content` | `tool_calls` (functions) and the older `function_call` |
| Responses API | its `output_text`: every `output_text` part, run together | `function_call` items (with their namespace, as `crm.lookup`) |
| Anthropic messages | its `text` blocks, run together | `tool_use` blocks, their `input` as JSON |

**What is not checked:**

- **Tools the provider ran itself, reasoning and refusals.** A Responses answer's web search, file
  search, code interpreter and remote MCP items, its reasoning and its refusals; an Anthropic
  answer's thinking, `server_tool_use` and their results. They already ran before the answer came
  back, and are passed on as they came.
- **On the Responses API, an answer a check can't read is withheld** (HTTP 400, `not_checked`): an
  image the model generated, or a call to a tool that is not a function (a custom, computer, shell
  or patch tool). With output checks on, a request that offers such a tool, or the
  `image_generation` tool, is refused before anything is sent, and so is a background response
  (`background: true`), whose answer is fetched later, outside the plug-in.

- **More than one answer per call.** A request with `n` above 1 (also inside `extra_body`) is
  refused with HTTP 400 before anything is sent, streamed or not.
- **`stream` inside `extra_body`.** LiteLLM would not know the answer is streamed, so it could not
  be held for a check. Refused with HTTP 400 before anything is sent; set `stream` on the request
  itself.
- **An answer in audio.** A request with `modalities` holding anything but `"text"`, or with an
  `audio` parameter (also inside `extra_body`), is refused with HTTP 400 before anything is sent,
  streamed or not: its transcript would not be read. An answer that comes back with audio anyway
  (the proxy's own model settings can ask for it) is withheld with HTTP 400; the provider was
  called.
- **Reasoning text and images** in a chat completion. They are not read and are passed on as they
  came.
- **A tool call of a type other than a function** (a custom tool), or a streamed tool call whose
  pieces can't be put together. The answer is withheld with HTTP 400 (`not_checked`).
- **A stream a check can't read**, such as an event it doesn't know or copies of the text that
  disagree. The answer is withheld (`not_checked`), never passed on unread.
- **An answer with neither text nor a tool call.** Norcaster is not asked. The answer is passed on
  and the response says `x-norcaster-answer-decision: no_text`.
- **The check engine in your own network** reads tool calls too. An older engine refuses them (409
  `tool_calls_not_on_engine`), and the answer is withheld with HTTP 503 (`refused`).
- **A `norcaster` SDK older than 0.1.5** can't read a Responses or Anthropic request, or describe
  images, files and audio. The plug-in requires `norcaster>=0.1.5`; installed around that rule, the
  proxy does not start.
- **A call whose request was let through without a check.** Its answer is not checked either; the
  call is one let-through record.

**When an answer is not cleared, the caller does not get it.**

| Why | The caller gets | `x-norcaster-answer-decision` |
|---|---|---|
| Norcaster's decision was block | HTTP 400 | `block` |
| Norcaster can't be reached | HTTP 503, unless `when_unreachable` is `let_through` | `unreachable` |
| Norcaster refused: the rate limit | HTTP 429 with `Retry-After` | `refused` |
| Norcaster refused: the app is stopped | HTTP 403 | `refused` |
| Norcaster refused for another reason, such as `output_rules_off`: the app's rules don't cover answers | HTTP 503 | `refused` |
| The answer can't be read: more than one came back, it holds audio or an image, it is not text, or it holds a tool call a check can't read | HTTP 400 | `not_checked` |

On a streamed Responses or Anthropic answer the status is 200 and the reason comes as the stream's
one event (above). On `/v1/messages` LiteLLM answers in Anthropic's error shape and leaves the
`x-norcaster-*` headers off its errors: the decision and the record are in
`error.provider_specific_fields`.

`when_unreachable` covers both sides. With `let_through`, an answer is handed over when Norcaster
can't be reached after the provider answered, and it is recorded the way a request is (see
[When Norcaster can't be reached](#when-norcaster-cant-be-reached)): in the proxy's log, on
LiteLLM's log entry, in `x-norcaster-answer-decision: unchecked` and
`x-norcaster-answer-unchecked-id`, and in Norcaster as an answer let through, with the request it
belongs to.

Know these limits before you turn it on:

- **A withheld answer was already produced, and the provider bills for it.** OpenAI clients retry
  429 and 5xx answers by default, and each retry calls the provider again.
- **Each call waits for Norcaster twice.** The answer check has a rate limit of its own, at the same
  rate as the key's, so it doesn't use up the request check's. Neither counts toward the plan's
  monthly request limit yet.
- **If the app's rules don't cover answers, every call fails** (HTTP 503, `output_rules_off`) until
  you turn answers on in the app's rules or turn `check_output` off.
- **Other LiteLLM guardrails that run after the provider call and before this one** see the answer
  as it came.
- **LiteLLM's `run_in_parallel`, `scan_raw_request` and `stream_scope` settings** must be left off
  this entry. With them LiteLLM may drop the hidden text or skip the answer check, so the proxy
  does not start.

## What the caller gets

On every checked request, as response headers (on `/v1/messages` only when it succeeds; see below),
also when the answer is an error:

- `x-norcaster-decision`: `allow`, `hide` or `block`; `unchecked` for a request let through without
  a check; `unreachable`, `refused` or `not_checked` for a request that was not sent.
- `x-norcaster-receipt-id`: the record's id in Norcaster, when Norcaster answered.
- `x-norcaster-unchecked-id`: the id of a request let through without a check.

With output checks on, the answer's check is beside the request's, also on an error:

- `x-norcaster-answer-decision`: `allow`, `hide` or `block`; `unchecked` for an answer handed over
  without a check; `unreachable`, `refused` or `not_checked` for an answer that was withheld;
  `no_text` for an answer with neither text nor a tool call to check.
- `x-norcaster-answer-receipt-id`: the id of the answer's record in Norcaster.
- `x-norcaster-answer-unchecked-id`: the id of an answer let through without a check.

A request refused on another route gets the reason in the error. LiteLLM does not pass extra
headers on every route's errors: on `/v1/messages` it leaves them off every error, and the decision
and the record are in the error body. There the error is in Anthropic's shape:

```json
{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "message": "Norcaster's decision was block (contextual_block). A secret or access credential was detected in the message. The provider was not called. Receipt: 7b9df112-2055-4921-938e-882bcf2800a8.",
    "provider_specific_fields": {
      "decision": "block",
      "receipt_id": "7b9df112-2055-4921-938e-882bcf2800a8",
      "...": "the same fields as below"
    }
  }
}
```

On the other routes it is OpenAI's.

A block is HTTP 400. The provider is not called:

```json
{
  "error": {
    "message": "Norcaster's decision was block (contextual_block). A secret or access credential was detected in the message. The provider was not called. Receipt: 7b9df112-2055-4921-938e-882bcf2800a8.",
    "type": "invalid_request_error",
    "param": null,
    "code": "400",
    "provider_specific_fields": {
      "decision": "block",
      "code": "contextual_block",
      "reason": "A secret or access credential was detected in the message.",
      "receipt_id": "7b9df112-2055-4921-938e-882bcf2800a8",
      "content_sha256": "45cc63610bc3049637d570e5ba5e5b6d779a9f8ee9697b7deeac15934113f798",
      "rules_version": 3,
      "decided_at": "2026-11-12T12:01:28.403512+00:00",
      "guardrail_name": "norcaster-support-assistant",
      "guardrail_mode": "pre_call"
    }
  }
}
```

LiteLLM builds the envelope; `provider_specific_fields` also repeats the message and carries
Norcaster's suggestion and its request id.

A blocked answer is HTTP 400 in the same envelope. Its message starts "Norcaster's decision on the
model's answer was block", ends "The answer was withheld. Receipt: ..." and its fields carry
`"about": "answer"`.

The error never repeats what was blocked.

## What your own logs get

The plug-in puts the outcome on LiteLLM's log entry for the request (its guardrail information), so
it reaches whatever LiteLLM logs to: decision, receipt id, the SHA-256 of what was checked, the
version of the app's rules, and when. It never puts the request's content or the key there.

With output checks on there is a second outcome on the same entry, for the answer. On the LiteLLM
versions this is tested with, the answer LiteLLM itself logs is the one the caller got: the hidden
text after a hide, and none after a block.

## When Norcaster can't be reached

"Can't be reached" means no connection, no answer within `timeout`, or an error on Norcaster's side.

**By default the request is blocked:** HTTP 503, the provider is not called.

`when_unreachable: "let_through"` sends the request on instead. Every request that is let through
is recorded, each time:

1. A warning in the proxy's log, with the request's id and its SHA-256.
2. On LiteLLM's log entry for the request, and in the `x-norcaster-decision: unchecked` and
   `x-norcaster-unchecked-id` response headers.
3. In Norcaster, once it answers again: the next check that succeeds reports what was let through,
   and it shows in the app's records as **Let through without a check**. When the proxy stops, it
   tries once more to report what is still waiting.

The record holds a SHA-256 of the request, when it happened and why. It never holds the request
itself. No rule ran on a request that was let through, and it is not counted as a check.

Know these limits before you turn it on:

- A request that is let through goes to the provider as the app wrote it. Nothing is hidden.
- Requests waiting to be reported are kept in the proxy's memory, one list per proxy process. If a
  process ends before Norcaster answers again, the proxy's own log line is the only record.
- At most 10,000 wait at a time per process. Past that, requests are blocked.
- A refusal from Norcaster always blocks, whatever the setting: a wrong or revoked key, a missing
  scope, a stopped app, the rate limit, or more than 1,000,000 characters.

## In LiteLLM's SDK, without a proxy

The plug-in also runs in an app that calls LiteLLM's SDK itself (`litellm.acompletion`,
`litellm.aresponses`, `litellm.anthropic.messages.acreate`). Put it in `litellm.callbacks`, and name
it on each call:

```python
litellm.callbacks = [NorcasterCheck(guardrail_name="norcaster-support-assistant", api_key=..., deployment_id=...)]
await litellm.aresponses(model="openai/gpt-5", input="...", guardrails=["norcaster-support-assistant"])
```

- **Only calls that name it are checked.** LiteLLM's SDK runs a guardrail on the calls that list
  it in `guardrails`, whatever `default_on` says.
- **The same three kinds are checked**, with the same refusals. Any other call that names it is
  refused, before anything is sent.
- **With output checks on, a streamed call is refused** before anything is sent. LiteLLM's SDK
  hands a streamed answer to no plug-in, so it would reach the app unchecked. Answers that are not
  streamed are checked as in a proxy.
- **The caller gets** a FastAPI `HTTPException` when a call is refused, blocked or withheld, with
  the decision and the record in its `detail`.

## If the plug-in is not running

LiteLLM starts without a guardrail it considers wrongly configured, and then sends requests on
unchecked. The plug-in is written so that this does not happen quietly: with no key, a mode other
than `pre_call`, or a wrong `when_unreachable` or `timeout`, **the proxy does not start**, and the
log says `Norcaster check:` and what is wrong. The same goes for a plug-in or SDK that is not
installed, and for `check_output` and a `mode` that disagree.
LiteLLM's `scan_raw_request` must be left off the entry, or the proxy does not start: with it
LiteLLM drops the text Norcaster hid and sends the original to the provider.

After any change to the config, send one request and look for `x-norcaster-decision`, and for
`x-norcaster-answer-decision` when output checks are on.

## Limits to know

- **Each check draws on the key's rate limit**, the same one as requests through Norcaster's
  gateway: 600 a minute by default, shared by all of one person's keys, or by one service account's
  or agent's keys. Over the limit, Norcaster refuses and the request is blocked with HTTP 429. Like
  gatewayed requests, checks don't count toward the plan's monthly request limit yet.
- **Each request waits for Norcaster's answer** before the provider is called. With output checks
  on, the caller also waits for Norcaster after the provider answered.
- **The check is of the conversation.** The model name, tools and other settings are not sent to
  Norcaster.
- **Pin your LiteLLM version**, and install it from a source you trust. This plug-in runs inside
  the proxy and uses LiteLLM's guardrail interface, which changes between releases. A Responses
  stream reaches the plug-in as LiteLLM's own event classes, and an event it doesn't know is
  withheld.

## Develop

The tests need LiteLLM's proxy and the SDK from this repository (`sdk/python`). From this folder:

```bash
PYTHONPATH=src:../../sdk/python/src uv run --isolated --no-project --python 3.12 \
  --with "litellm[proxy]==1.109.1" --with "httpx2!=2.16.1" --with "anthropic>=1,<2" \
  --with "pytest>=7" python -m pytest -q
```

- `test/test_guardrail.py` (chat) and `test/test_responses_and_messages.py` call the plug-in the way
  the proxy does.
- `test/test_proxy.py` starts a real LiteLLM proxy against a stand-in provider and a stand-in
  Norcaster, both on `127.0.0.1`.
- `test/test_sdk_mode.py` makes real calls through LiteLLM's SDK.
- `test/test_vectors.py` checks the shared hash vectors (`sdk/testdata/`): the plug-in asks about
  each request and answer exactly as the SDKs and the API read them.
