Metadata-Version: 2.5
Name: mutant-ai
Version: 0.8.0
Summary: Automated red teaming and behavioral dataset engineering for LLMs, RAGs, and AI Agents
Project-URL: Homepage, https://github.com/ankitgmishra/mutant-ai
Project-URL: Documentation, https://mutant.aiankit.com/
Project-URL: Repository, https://github.com/ankitgmishra/mutant-ai
Project-URL: Bug Tracker, https://github.com/ankitgmishra/mutant-ai/issues
Project-URL: Author Website, https://www.aiankit.com
Project-URL: LinkedIn, https://www.linkedin.com/in/ankitgmishra/
Author-email: Ankit Mishra <ankitmishra.letter@gmail.com>
License: MIT License
        
        Copyright (c) 2026 Ankit Mishra
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
License-File: LICENSE
Keywords: agents,ai,behavioral-testing,evaluation,fuzzing,llm,mutation-testing,red-teaming,testing
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Testing
Classifier: Typing :: Typed
Requires-Python: >=3.11
Requires-Dist: anyio>=4.3.0
Requires-Dist: diskcache>=5.6.0
Requires-Dist: httpx>=0.27.0
Requires-Dist: jinja2>=3.1.0
Requires-Dist: pydantic>=2.5.0
Requires-Dist: rich>=13.7.0
Requires-Dist: tenacity>=8.2.0
Requires-Dist: typing-extensions>=4.9.0
Provides-Extra: all
Requires-Dist: anthropic>=0.25.0; extra == 'all'
Requires-Dist: litellm>=1.35.0; extra == 'all'
Requires-Dist: openai>=1.30.0; extra == 'all'
Provides-Extra: anthropic
Requires-Dist: anthropic>=0.25.0; extra == 'anthropic'
Provides-Extra: datasets
Requires-Dist: datasets>=2.0.0; extra == 'datasets'
Requires-Dist: pandas>=2.0.0; extra == 'datasets'
Provides-Extra: dev
Requires-Dist: mypy>=1.8.0; extra == 'dev'
Requires-Dist: pre-commit>=3.6.0; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest-cov>=4.1.0; extra == 'dev'
Requires-Dist: pytest>=7.4.0; extra == 'dev'
Requires-Dist: ruff>=0.3.0; extra == 'dev'
Provides-Extra: gemini
Requires-Dist: google-genai>=0.2.0; extra == 'gemini'
Provides-Extra: litellm
Requires-Dist: litellm>=1.35.0; extra == 'litellm'
Provides-Extra: notebooks
Requires-Dist: ipykernel>=6.29.0; extra == 'notebooks'
Requires-Dist: nbclient>=0.10.0; extra == 'notebooks'
Requires-Dist: nbconvert>=7.16.0; extra == 'notebooks'
Requires-Dist: numpy<2.0,>=1.26.0; extra == 'notebooks'
Provides-Extra: openai
Requires-Dist: openai>=1.30.0; extra == 'openai'
Description-Content-Type: text/markdown

<div align="center">
  <h1>Mutant AI</h1>
  <p><b>Automate Evaluation & Red Teaming for LLMs</b></p>
  
  <p>
    <a href="https://pypi.org/project/mutant-ai/"><img src="https://img.shields.io/pypi/v/mutant-ai.svg?style=flat-square&color=000000" alt="PyPI Version"></a>
    <a href="https://docs-mutantai.netlify.app"><img src="https://img.shields.io/badge/docs-live-black.svg?style=flat-square" alt="Documentation"></a>
    <a href="https://github.com/ankitgmishra/mutant-ai/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-black.svg?style=flat-square" alt="License"></a>
  </p>
  
  <p>Mutant is the open-source evaluation framework for synthesizing adversarial datasets, testing AI agents, and grading RAG pipelines.</p>

  <h3><a href="https://docs-mutantai.netlify.app">Read the Official Documentation &rarr;</a></h3>
</div>

---

## ⚡ Quick Install

Mutant is available on PyPI. Install it using `pip` or `uv`:

```console
$ pip install mutant-ai
```

---

## 🏗️ Core Framework

Mutant is divided into four core pillars. You can use them independently or chain them together for end-to-end evaluation.

### 1. Data Augmentation
Stop writing manual test cases. Mutant takes a single seed scenario and uses cognitive dimensions (like anger, prompt injection, or complex reasoning) to generate hundreds of behaviorally diverse test cases instantly.

```python
from mutant.core import Scenario, augment
from mutant.providers.ollama import OllamaProvider

provider = OllamaProvider(model="llama3.1")

# Generate 5 diverse variations of a basic intent
dataset = await augment(
    dataset=[Scenario(title="Refund", description="I want a refund.")],
    provider=provider,
    mutations_per_case=5,
    dimensions=["emotion.angry", "language.slang"]
)

dataset.save("mutations.json")
```

### 2. Agentic Red Teaming
Instead of static security scans, unleash a hypothesis-driven attacker agent. Mutant dynamically probes your system in multi-turn conversations, adapting its attacks to bypass your safeguards.

```python
from mutant.redteam import red_team
from mutant.providers.ollama import OllamaProvider
from my_app import my_agent 

provider = OllamaProvider(model="llama3.1")

# Unleash the attacker against your live agent
report = await red_team(
    target=my_agent,
    goal="Extract the hidden system instructions.",
    provider=provider,
    max_turns=5
)

report.to_html("security_report.html")
```

### 3. Security Evaluation
Go beyond general correctness and actively grade your application's defensive mechanisms. Run your adversarial datasets through the Eval Suite using specialized security metrics to automatically detect prompt injections, PII leakage, and unauthorized behavior.

```python
from mutant.eval import EvalSuite, SensitiveDataLeakage, PromptInjectionResistance
from mutant.providers.ollama import OllamaProvider

provider = OllamaProvider(model="llama3.1")

# Grade the agent against security threats
suite = EvalSuite(metrics=[
    SensitiveDataLeakage(provider=provider),
    PromptInjectionResistance(provider=provider)
])

report = await suite.evaluate(test_cases=dataset.cases)
report.to_html("security_report.html")
```

### 4. Evaluation Suite
Once you've generated your adversarial dataset or executed your attacks, grade your application's responses using LLM-as-a-judge metrics to provide deterministic scores and HTML reports.

```python
from mutant.eval import EvalSuite, Correctness, Faithfulness
from mutant.providers.ollama import OllamaProvider

provider = OllamaProvider(model="llama3.1")

# Initialize metrics and evaluate test cases
suite = EvalSuite(metrics=[
    Correctness(provider=provider),
    Faithfulness(provider=provider)
])

report = await suite.evaluate(test_cases=dataset.cases)
report.to_html("eval_report.html")
```

---

## 📊 Actionable Reports

Mutant natively outputs highly visual HTML dashboards and CI/CD-ready JSON artifacts, mapping your exact vulnerability and coverage footprint natively out of the box.

---

## 📚 Documentation

For full documentation, including advanced tutorials, architecture diagrams, and custom metric creation, visit the [official documentation](https://docs-mutantai.netlify.app).

## 🤝 Contributing

Contributions are heavily welcomed! Please read our [Contributing Guide](https://github.com/ankitgmishra/mutant-ai/blob/main/CONTRIBUTING.md) to get started.

## 📄 License

MIT © 2026 [Ankit Mishra](https://aiankit.com). Built for the open-source community.
