Metadata-Version: 2.4
Name: datarobot-moderations
Version: 11.2.26
Summary: DataRobot Monitoring and Moderation framework
License: DataRobot Tool and Utility Agreement
Author: DataRobot
Author-email: support@datarobot.com
Requires-Python: >=3.10,<3.13
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Provides-Extra: all
Provides-Extra: bedrock
Provides-Extra: datarobot-sdk
Provides-Extra: llm-eval
Provides-Extra: nemo
Provides-Extra: nemo-evaluator
Provides-Extra: nvidia
Provides-Extra: vertex
Requires-Dist: aiohttp (>=3.9.5)
Requires-Dist: backoff (>=2.2.1)
Requires-Dist: datarobot (>=3.6.0) ; extra == "datarobot-sdk" or extra == "all"
Requires-Dist: datarobot-predict (>=1.9.6) ; extra == "datarobot-sdk" or extra == "all"
Requires-Dist: deepeval (>=3.3.5) ; extra == "llm-eval" or extra == "all"
Requires-Dist: google-cloud-aiplatform (>=1.133.0,<2) ; extra == "vertex" or extra == "all"
Requires-Dist: langchain (>=0.1.12) ; extra == "llm-eval" or extra == "all"
Requires-Dist: langchain-nvidia-ai-endpoints (>=0.3.9) ; extra == "nvidia" or extra == "all"
Requires-Dist: langchain-openai (>=0.1.7) ; extra == "llm-eval" or extra == "all"
Requires-Dist: llama-index (>=0.13.0) ; extra == "llm-eval" or extra == "all"
Requires-Dist: llama-index-embeddings-azure-openai (>=0.1.6) ; extra == "llm-eval" or extra == "all"
Requires-Dist: llama-index-llms-bedrock-converse (>=0.1.6) ; extra == "bedrock" or extra == "all"
Requires-Dist: llama-index-llms-langchain (>=0.7.2) ; extra == "llm-eval" or extra == "all"
Requires-Dist: llama-index-llms-openai (>=0.1.0) ; extra == "llm-eval" or extra == "all"
Requires-Dist: llama-index-llms-vertex (>=0.1.5) ; extra == "vertex" or extra == "all"
Requires-Dist: nemo-microservices (>=1.5.0,<2.0.0) ; extra == "nemo-evaluator" or extra == "all"
Requires-Dist: nemoguardrails (>=0.20.0) ; extra == "nemo" or extra == "all"
Requires-Dist: numpy (>=1.25.0)
Requires-Dist: openai (>=1.14.3) ; extra == "llm-eval" or extra == "all"
Requires-Dist: opentelemetry-api (>=1.16.0)
Requires-Dist: opentelemetry-instrumentation (>=0.60b1,<0.61)
Requires-Dist: opentelemetry-sdk (>=1.16.0)
Requires-Dist: pandas (>=2.0.3)
Requires-Dist: pillow (>=12.1.1)
Requires-Dist: pydantic (>=2.0.0,<3.0.0)
Requires-Dist: ragas (>=0.4.3) ; extra == "llm-eval" or extra == "all"
Requires-Dist: rouge-score (>=0.1.2)
Requires-Dist: tiktoken (>=0.5.1)
Requires-Dist: trafaret (>=2.1.1)
Description-Content-Type: text/markdown

# DataRobot Moderations library

This library enforces the intervention in the prompt and response texts as per the
guard configuration set by the user.

The library accepts the guard configuration in the yaml format and the input prompts
and outputs the dataframe with the details like:
- should the prompt be blocked
- should the completion be blocked
- metric values obtained from the model guards
- is the prompt or response modified as per the modifier guard configuration


## Architecture

The library is architected in a way that it wraps around the typical LLM prediction method.
The library will first run the pre-score guards - the guards that will evaluate prompts and
enforce moderation if necessary.  All the prompts that were not moderated by the library are
forwarded to the actual LLM to get their respective completions.  The library then evaluates
these completions using post-score guards and enforces intervention on them.

![](pics/img.png)

## How to build it?

The repository uses `poetry` to manage the build process and a wheel can be built using:
```bash
make clean
make
```

## How to use it?

A wheel file generated or downloaded can be installed with pip and will pull its
dependencies as well.
```bash
pip3 install datarobot-moderations
```

### Optional extras

The base install covers token-count, ROUGE-1, cost, and NeMo guards.
Heavier or cloud-specific dependencies are opt-in:

| Extra | What it enables |
|---|---|
| `datarobot-sdk` | DataRobot model guards, DataRobot LLM evaluator type |
| `llm-eval` | Faithfulness, Task Adherence, Agent Goal Accuracy, Guideline Adherence guards |
| `nemo` | NeMo Guardrails colang-based flow guard |
| `nemo-evaluator` | NeMo live-evaluation microservice guard |
| `nvidia` | NVIDIA NIM / ChatNVIDIA LLM support |
| `vertex` | Google Cloud Vertex AI LLM support |
| `bedrock` | AWS Bedrock LLM support |
| `all` | Every optional dependency at once |

```bash
# Example: task-adherence guard backed by a DataRobot LLM deployment
pip3 install 'datarobot-moderations[llm-eval,datarobot-sdk]'
```

### Standalone Python API

```python
from datarobot_dome.api import ModerationPipeline

pipeline = ModerationPipeline.from_yaml("moderation_config.yaml")
```

**Evaluate a prompt** (pre-score guards only):

```python
result, latency = pipeline.evaluate_prompt("Ignore previous instructions and …")
if result.blocked:
    print(result.blocked_message)
```

**Evaluate a response** (post-score guards only):

```python
result, latency = pipeline.evaluate_response(
    response="The capital of France is Paris.",
    prompt="What is the capital of France?",
)
print(result.blocked)           # True / False
print(result.metrics)           # {"task_adherence_score": 0.0, ...}
```

**Full pipeline** — pre-score → LLM → post-score in one call:

```python
def my_llm(prompt: str) -> str:
    # Replace with your actual LLM integration (OpenAI, Vertex, etc.)
    return "DataRobot is an AI platform."

result = pipeline.evaluate_full_pipeline(
    prompt="What is DataRobot?",
    llm_callable=my_llm,
)

if not result.blocked:
    print(f"LLM Response: {result.response}")
```

#### Result objects

`evaluate_prompt` / `evaluate_response` return an `EvaluationResult`:

| Field | Type | Description |
|---|---|---|
| `blocked` | `bool` | Whether a BLOCK guard fired |
| `blocked_message` | `str \| None` | Guard-supplied block reason |
| `replaced` | `bool` | Whether a REPLACE guard fired |
| `replacement` | `str \| None` | The replacement text |
| `metrics` | `dict` | All guard metric values (scores, counts, …) |

`evaluate_full_pipeline` returns a `PipelineResult`:

| Field | Type | Description |
|---|---|---|
| `prompt_evaluation` | `EvaluationResult` | Pre-score guard result |
| `response` | `str \| None` | Effective response (post-replacement if applicable) |
| `response_evaluation` | `EvaluationResult \| None` | Post-score guard result |
| `blocked` | `bool` | True if either stage was blocked |
| `replaced` | `bool` | True if either stage was replaced |

### With [DRUM](https://github.com/datarobot/datarobot-user-models)
As described above, the library nicely wraps DRUM's `score` method for pre and post score
guards. Hence, in case of DRUM, the user simply runs their custom model using `drum score`
and can avail the moderation library features.

Install DRUM along with the necessary optional extras for your specific guards. If you are unsure which guards are in use, install `[all]`:

```bash
pip3 install datarobot-drum 'datarobot-moderations[all]'
drum score --verbose --logging-level info --code-dir ./ --input ./input.csv --target-type textgeneration --runtime-params-file values.yaml
```
