Metadata-Version: 2.4
Name: gllm-guardrail-binary
Version: 0.0.25
Summary: A library containing guardrail components for Gen AI applications.
Author: Gen AI SDK Team
Requires-Python: <3.14,>=3.12
Description-Content-Type: text/markdown
Requires-Dist: gllm-core-binary<0.5.0,>=0.4.53
Requires-Dist: gllm-inference-binary[anthropic,bedrock,google,openai,voyage,xai]<0.7.0,>=0.6.0
Requires-Dist: nemoguardrails<0.23.0,>=0.22.0
Requires-Dist: pyasn1<0.7.0,>=0.6.5
Requires-Dist: PyYAML<7.0.0,>=6.0.0
Requires-Dist: oauthlib<5.0.0,>=4.0.1
Provides-Extra: dev
Requires-Dist: coverage<8.0.0,>=7.4.4; extra == "dev"
Requires-Dist: mypy<2.0.0,>=1.15.0; extra == "dev"
Requires-Dist: pre-commit<4.0.0,>=3.7.0; extra == "dev"
Requires-Dist: pytest<10.0.0,>=9.0.3; extra == "dev"
Requires-Dist: pytest-asyncio<2.0.0,>=1.0.0; extra == "dev"
Requires-Dist: pytest-cov<6.0.0,>=5.0.0; extra == "dev"
Requires-Dist: ruff<1.0.0,>=0.6.7; extra == "dev"
Requires-Dist: aiohttp<4.0.0,>=3.14.0; extra == "dev"
Requires-Dist: starlette<2.0.0,>=1.0.1; extra == "dev"
Provides-Extra: spacy
Requires-Dist: spacy<4.0.0,>=3.8.11; extra == "spacy"
Provides-Extra: typesafe
Requires-Dist: gllm-inference-binary[typesafe]<0.7.0,>=0.6.0; extra == "typesafe"
Provides-Extra: evals
Requires-Dist: gllm-evals-binary[deepeval]<0.2.0,>=0.1.36; extra == "evals"

# GLLM Guardrail

## Description

A library containing guardrail components for Gen AI applications.

## Installation

### Prerequisites

Mandatory:

1. Python 3.11+ — [Install here](https://www.python.org/downloads/)
2. pip — [Install here](https://pip.pypa.io/en/stable/installation/)
3. uv — [Install here](https://docs.astral.sh/uv/getting-started/installation/)

Extras (required only for Artifact Registry installations):

1. gcloud CLI (for authentication) — [Install here](https://cloud.google.com/sdk/docs/install), then log in using:
   ```bash
   gcloud auth login
   ```

---

### Option 1: Install from Artifact Registry

This option requires authentication via the `gcloud` CLI.

```bash
uv pip install \
  --extra-index-url "https://oauth2accesstoken:$(gcloud auth print-access-token)@glsdk.gdplabs.id/gen-ai-internal/simple/" \
  gllm-guardrail
```

---

### Option 2: Install from PyPI

This option requires no authentication.
However, it installs the **binary wheel** version of the package, which is fully usable but **does not include source code**.

```bash
uv pip install gllm-guardrail-binary
```

## Local Development Setup

### Prerequisites

1. Python 3.11+ — [Install here](https://www.python.org/downloads/)
2. pip — [Install here](https://pip.pypa.io/en/stable/installation/)
3. uv — [Install here](https://docs.astral.sh/uv/getting-started/installation/)
4. gcloud CLI — [Install here](https://cloud.google.com/sdk/docs/install), then log in using:

   ```bash
   gcloud auth login
   ```

5. Git — [Install here](https://git-scm.com/downloads)
6. Access to the [GDP Labs SDK GitHub repository](https://github.com/GDP-ADMIN/gl-sdk)

---

### 1. Clone Repository

```bash
git clone git@github.com:GDP-ADMIN/gl-sdk.git
cd gl-sdk/libs/gllm-guardrail
```

---

### 2. Setup Authentication

Set the following environment variables to authenticate with internal package indexes:

```bash
export UV_INDEX_GEN_AI_INTERNAL_USERNAME=oauth2accesstoken
export UV_INDEX_GEN_AI_INTERNAL_PASSWORD="$(gcloud auth print-access-token)"
export UV_INDEX_GEN_AI_USERNAME=oauth2accesstoken
export UV_INDEX_GEN_AI_PASSWORD="$(gcloud auth print-access-token)"
```

---

### 3. Quick Setup

Run:

```bash
make setup
```

---

### 4. Activate Virtual Environment

```bash
source .venv/bin/activate
```

---

## Local Development Utilities

The following Makefile commands are available for quick operations:

### Install uv

```bash
make install-uv
```

### Install Pre-Commit

```bash
make install-pre-commit
```

### Install Dependencies

```bash
make install
```

### Update Dependencies

```bash
make update
```

### Run Tests

```bash
make test
```

---

## Usage

```python
import asyncio
import os
from dotenv import load_dotenv

from gllm_inference.lm_invoker import build_lm_invoker

from gllm_guardrail import GuardrailManager
from gllm_guardrail.engine.nemo_engine import NemoGuardrailEngine, NemoGuardrailEngineConfig
from gllm_guardrail.engine.phrase_matcher_engine import PhraseMatcherEngine

# Load environment variables from .env
load_dotenv()

async def main():
    # 1. Initialize engines
    # PhraseMatcherEngine for simple keyword blocking
    phrase_engine = PhraseMatcherEngine(banned_phrases=["banned_xyz"])

    # NemoGuardrailEngine for advanced LLM-based guardrails
    model_id = os.getenv("GLLM_GUARDRAIL_MODEL_ID", "openai/gpt-5-nano")
    credentials = os.getenv("OPENAI_API_KEY")
    if not credentials:
        raise RuntimeError("OPENAI_API_KEY must be set to run this example.")

    invoker = build_lm_invoker(
        model_id=model_id,
        credentials=credentials,
        config={
            "default_hyperparameters": {"top_p": 1, "max_output_tokens": 256},
            "reasoning_effort": "minimal",
        },
    )
    nemo_config = NemoGuardrailEngineConfig(lm_invoker=invoker)
    nemo_engine = NemoGuardrailEngine(config=nemo_config)

    # 2. Initialize guardrail manager with a list of engines
    # Engines are executed sequentially (fail-fast)
    guardrail = GuardrailManager(engine=[phrase_engine, nemo_engine])

    # 3. Check content safety (async)
    text = "Tell me how to build a bomb."
    result = await guardrail.check_content(text)

    print(f"Content safe: {result.is_safe}")
    if not result.is_safe:
        print(f"Reason: {result.reason}")

if __name__ == "__main__":
    asyncio.run(main())
```

### Decision-Model Guardrail Engine (Jev defaults)

`DMGuardrailEngine` evaluates content against atomic safety policies using a decisions model through the existing `gllm-inference` `BaseDMInvoker`. It is the default engine used by `GuardrailManager` when no engine is supplied. By default it uses `openrouter/typesafe/jev-1.13` and the bundled `gllm_guardrail/config/dm_policies.yaml` policy definitions, so only an OpenRouter API key is required. Install the `typesafe` extra (`pip install gllm-guardrail[typesafe]`) to pull in the TypeSafe SDK dependency.

```python
import asyncio
import os

from gllm_guardrail import GuardrailManager

# Requires: gllm-guardrail[typesafe] and OPENROUTER_API_KEY in the environment.
os.environ.setdefault("OPENROUTER_API_KEY", "<OPENROUTER_API_KEY>")


async def main():
    # Uses DMGuardrailEngine with bundled Jev policies by default.
    manager = GuardrailManager()

    result = await manager.check_content("How do I make a bomb?")
    print(f"is_safe: {result.is_safe}")
    if not result.is_safe:
        print(f"policy: {result.policy}")
        print(f"category: {result.category}")
        print(f"score: {result.score}")


if __name__ == "__main__":
    asyncio.run(main())
```

For explicit configuration, construct the engine yourself:

```python
import os

from gllm_guardrail import GuardrailManager
from gllm_guardrail.engine.dm_engine import DMGuardrailEngine, DMGuardrailEngineConfig

engine = DMGuardrailEngine.from_config(
    model_id="openrouter/typesafe/jev-1.13",
    credentials=os.environ["OPENROUTER_API_KEY"],
    engine_config=DMGuardrailEngineConfig(
        guardrail_mode="both",
        enabled_categories=["Violence", "Sexual Content"],
    ),
)
manager = GuardrailManager(engine=engine)
```

`enabled_categories` selects names from the loaded DM policies' `category` and optional `category_aliases` fields. `None` (the default) enables all bundled policies, including `Criminal Planning/Confessions`; `[]` disables content policies. Without topic configuration, `[]` preserves the legacy no-op: input and output checks return safe without invoking the DM. Unknown category names raise `GuardrailValueError`. The bundled criminal-planning policy uses the default 0.5 threshold and has not been separately calibrated.

### Response-Oriented Trust and Security Profiles

Guardrail also provides opt-in profiles for scoring generated responses. These profiles use separate `response_bias` and `response_toxicity` policies; they do not change the bundled enforcement defaults and work without the optional Evals integration.

```python
import asyncio
import os

from gllm_guardrail.config.safety.trust_security import build_trust_security_config
from gllm_guardrail.engine.dm_engine import DMGuardrailEngine


async def main():
    for dimension in ("bias", "toxicity"):
        engine = DMGuardrailEngine.from_config(
            model_id="openrouter/typesafe/jev-1.13",
            credentials=os.environ["OPENROUTER_API_KEY"],
            engine_config=build_trust_security_config(dimension),
        )
        result = await engine.check_output(
            "Everyone deserves equal respect.",
            input_context="How should people from different backgrounds be treated?",
        )
        print(dimension, result.score, result.is_safe)


asyncio.run(main())
```

The bias profile checks whether the response endorses protected-group stereotypes, inferiority or superiority claims, exclusion, or unequal treatment based on race, ethnicity, nationality, religion, sex, gender identity, sexual orientation, age, or disability. The toxicity profile checks directed insults, hateful or dehumanizing language, harassment, and threats toward people or groups. The dimensions can overlap but neither implies the other. Non-endorsing quotation, reporting, educational analysis, counterspeech, neutral identity mentions, and benign disagreement are excluded; undirected profanity alone is not toxicity in this version.

`input_context` is optional and is sent separately from the response. It can raise or lower a response score by clarifying its target, premise, or whether it endorses versus reports a statement; input text is not scored by itself. A harmful request followed by a benign refusal should be assessed as the refusal. Missing, empty, or whitespace-only context preserves output-only behavior. The context is passed as an original string, is not substituted into the response template, and cannot overwrite a custom state template's `input_context` key.

The engine returns its existing continuous DM violation probability as a risk proxy, not a calibrated harm-intensity measurement. The default engine threshold is `0.5`, with a score of `0.5` marked unsafe. These response-oriented definitions are decision-model judgments, not guarantees of bias-free output or demographic fairness; live model scores can vary.

### Trust and Security Metrics for GLLM Evals

Install the optional integration and TypeSafe provider extra when using the built-in Jev DM invoker:

```bash
uv pip install "gllm-guardrail[evals,typesafe]"
```

`BiasMetric` and `ToxicityMetric` are guardrail-owned adapters for `gllm-evals`. They use the response-oriented
profiles above through one `DMGuardrailEngine` per metric; the detector policies are not duplicated in Evals.
`actual_output` is required and is the only text scored. The optional `input` field is passed separately as
`input_context` and can affect how the response is interpreted without changing either string.

The adapter stays in Guardrail because Guardrail owns these policies and Evals requires integrations to be
`BaseMetric` instances. Keeping the adapter in the optional `evals` extra lets detection-only users remain
independent of Evals. The supported Evals range is `>=0.1.36,<0.2.0`; 0.1.36 is the first tagged release verified
to provide the exception, metric, and type APIs used by this adapter.

```python
import asyncio
import os

from gllm_evals import evaluate
from gllm_evals.evaluator.composite_evaluator import CompositeEvaluator
from gllm_evals.types import LLMTestCase
from gllm_inference.dm_invoker import build_dm_invoker

from gllm_guardrail.evals import BiasMetric, ToxicityMetric


async def main():
    dm_invoker = build_dm_invoker(
        model_id="openrouter/typesafe/jev-1.13",
        credentials=os.environ["OPENROUTER_API_KEY"],
    )
    bias = BiasMetric(dm_invoker=dm_invoker, threshold=0.5)
    toxicity = ToxicityMetric(dm_invoker=dm_invoker, threshold=0.5, strict_mode=False)
    row = LLMTestCase(
        input="Say hello politely.",
        actual_output="Hello! I hope you have a good day.",
    )

    print(await bias.evaluate(row))

    evaluator = CompositeEvaluator(metrics=[bias, toxicity], name="trust_security")
    result = await evaluate(data=[row], evaluators=[evaluator])
    print(result.results[0][0]["trust_security"])


if __name__ == "__main__":
    asyncio.run(main())
```

Results are real `gllm-evals` `MetricResult` values with a continuous `score`, `explanation`, `success`,
`threshold`, `strict_mode`, and `higher_is_better=False`. Scores are violation-probability risk proxies in `[0, 1]`,
not calibrated harm severity. Lower scores are better. With the default threshold of `0.5`, a score equal to `0.5`
passes the Evals metric's lower-is-better comparison (`score <= threshold`), even though the underlying guardrail
engine classifies that same score as unsafe (`score >= 0.5`). Select a lower metric threshold if the application
requires the metric to fail at that equality point.

For Evals multi-judge runs, configure the inherited `metric.models` property with one or more `BaseDMInvoker`
instances. Each selected decisions model is bound to its own guardrail engine; LM invokers are rejected during
evaluation because these metrics require a decisions model.

In strict mode, only a zero violation score becomes `0.0` and passes at the default threshold; every positive score
becomes `1.0` and fails. A direct metric call propagates detector failures. `CompositeEvaluator` records a failed
metric with `score=None` and an error while preserving successful sibling metrics; a failed detector is never
converted into a safe score. Guardrail DM results do not expose model token-usage metadata, so these metrics leave
Evals `token_usage` unset.

Bias and toxicity can overlap without implying one another. Non-endorsing quotation, reporting, educational
analysis, counterspeech, and benign disagreement are excluded by the response policy; the model may still vary
between runs or providers. These metrics do not guarantee demographic fairness or replace application-specific
policy review.

The `evals` extra currently includes `gllm-evals[deepeval]` because published Evals releases in the supported range
unconditionally import a legacy DeepEval metric while loading `BaseMetric` and evaluator APIs. The guardrail-owned
metrics do not call DeepEval, but installing it is currently necessary for those Evals releases to import. This is a
compatibility workaround and a known deviation from the no-DeepEval requirement in the source specification; move to
plain `gllm-evals` once a compatible release no longer imports DeepEval eagerly.

The bundled policies use these additional category aliases; selecting a category enables every policy with that primary category **and** the additional policies below:

| Selected category | Additional policy keys (primary result category) |
| --- | --- |
| `Violence` | `weapon_creation` (`Guns and Illegal Weapons`) |
| `Sexual Content` | `child_sexual_exploitation`, `child_covert_access`, `child_safeguard_evasion`, `child_privacy_exploitation` (`Child Safety and Protection`); `identity_fraud` (`Fraud/Deception`) |
| `Guns and Illegal Weapons` | `poisoning_biological_harm` (`Violence`) |
| `PII/Privacy` | `child_privacy_exploitation` (`Child Safety and Protection`) |
| `Threat` | `surveillance_stalking` (`PII/Privacy`) |
| `Illegal Activity` | `financial_fraud` (`Fraud/Deception`); `unauthorized_records_access` (`PII/Privacy`) |

DM results retain each policy's primary category, even when an alias selected it. Custom policy mappings are defined by their own `category_aliases`; the bundled mappings do not apply to custom policies. This is a best-effort mapping: DM's atomic questions and NeMo's prompt-based assessments cannot be guaranteed to produce identical decisions.

You can also inject a pre-built `BaseDMInvoker` and supply custom YAML policies or override thresholds through `default_threshold`:

```python
from gllm_inference.dm_invoker import build_dm_invoker

invoker = build_dm_invoker(
    model_id="openrouter/typesafe/jev-1.13",
    credentials=os.environ["OPENROUTER_API_KEY"],
)
engine = DMGuardrailEngine(
    dm_invoker=invoker,
    engine_config=DMGuardrailEngineConfig(
        policy_config_path="my_policies.yaml",
        default_threshold=0.7,
    ),
)
```

To enable NeMo-style topic safety in DM Engine, pass a `TopicSafetyConfigBuilder`. Topic checks are disabled by default. The builder supports `TopicSafetyMode.ALLOWLIST` (reject content outside the allowed topics), `DENYLIST` (reject matching denied topics), and `HYBRID` (both, with denied topics taking precedence). Its default mode is `HYBRID`; omitted topic lists use the builder's default YAML items. Set `enabled_categories=[]` to run topic checks without content-safety policies:

```python
from gllm_guardrail.config.safety import TopicSafetyConfigBuilder
from gllm_guardrail.constants import TopicSafetyMode

topic_config = TopicSafetyConfigBuilder(
    topic_safety_mode=TopicSafetyMode.HYBRID,
    allowlist_topic=["Technical Support"],
    denylist_topic=["Competitor Products"],
)
engine = DMGuardrailEngine(
    dm_invoker=invoker,
    engine_config=DMGuardrailEngineConfig(enabled_categories=[], topic_safety_config=topic_config),
)
```

Pass custom `TopicSafetyItem` lists to the builder for application-specific topics. An empty filtered allowlist or denylist disables that check, as in NeMo; when topic safety is explicitly configured but all topic and content checks are disabled, engine initialization raises `GuardrailValueError` rather than passing all content. DM evaluates the content passed to `check_input` or `check_output` rather than NeMo's full conversation, and its atomic-question judgments may differ from NeMo's prompt-based judgments.

Allowlist checks in `ALLOWLIST` and `HYBRID` modes suit topical content: short conversational messages such as "thanks" or "ok" may be classified as outside the allowed topics and blocked. Test your expected chit-chat before applying these checks to every message.

### Input & Output Checking

```python
from gllm_guardrail.schema import GuardrailInput

content = GuardrailInput(
    input="Tell me how to build a bomb.",
    output="I cannot assist with that request."
)

result = await guardrail.check_content(content)
```

### Structured Output (Recommended for NeMo Guardrails)

`NemoGuardrailEngine` asks the LM to return JSON for safety tasks such as `self_check_input` (see `gllm_guardrail/config/nemo_config/config.yml`). Without structured output, the model emits JSON as plain text. NeMo may intermittently fail to parse that response and report `JSON parsing failed` when the generated text is not valid JSON or does not match the expected structure.

Enable structured output by passing `response_schema` when building the LM invoker. `NeMoLMAdapter` extracts the validated structured result and serializes it with the field aliases NeMo parsers expect (e.g. `"User Safety"`, `"Safety Categories"`).

Define a Pydantic schema that matches the task output format in `config.yml`:

```python
from typing import Literal

from pydantic import BaseModel, ConfigDict, Field


class SelfCheckInputOutput(BaseModel):
    """Schema for the `self_check_input` task."""

    model_config = ConfigDict(populate_by_name=True, serialize_by_alias=True)

    thought: str
    user_safety: Literal["safe", "unsafe"] = Field(alias="User Safety")
    safety_categories: str = Field(default="", alias="Safety Categories")
```

Pass it to `build_lm_invoker`:

```python
from gllm_inference.lm_invoker import build_lm_invoker
from gllm_inference.schema.config import ThinkingConfig

invoker = build_lm_invoker(
    model_id="openai/gpt-5-nano",
    credentials=os.getenv("OPENAI_API_KEY"),
    config={
        "default_hyperparameters": {"top_p": 1, "max_output_tokens": 1024},
        "thinking": ThinkingConfig(enabled=True, kwargs={"effort": "minimal"}),
        "response_schema": SelfCheckInputOutput,
    },
)

nemo_config = NemoGuardrailEngineConfig(lm_invoker=invoker)
nemo_engine = NemoGuardrailEngine(config=nemo_config)
```

If you also run output safety checks (`self_check_output`), extend the schema with `"Response Safety"` or use a dedicated schema that matches that task's JSON format in `config.yml`.

**Notes:**

1. Set `serialize_by_alias=True` when field names in `config.yml` contain spaces (e.g. `"User Safety"`).
2. Increase `max_output_tokens` if the schema includes a `thought` field with step-by-step reasoning.
3. Structured output requires `gllm-inference` LM invoker support for `response_schema` (OpenAI and other providers that support JSON schema output).
