Metadata-Version: 2.2
Name: llamator
Version: 2.0.0
Summary: Framework for testing vulnerabilities of large language models (LLM).
License: Attribution 4.0 International
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Science/Research
Classifier: License :: Other/Proprietary License
Classifier: Natural Language :: English
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Information Analysis
Classifier: Topic :: Security
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE.md
Requires-Dist: python-dotenv>=0.5.1
Requires-Dist: httpx==0.27.2
Requires-Dist: openai==1.6.1
Requires-Dist: langchain==0.2.16
Requires-Dist: langchain-community==0.2.16
Requires-Dist: langchain-core==0.2.38
Requires-Dist: tqdm==4.66.3
Requires-Dist: colorama==0.4.6
Requires-Dist: prettytable==3.10.0
Requires-Dist: pandas==2.2.2
Requires-Dist: inquirer==3.2.4
Requires-Dist: prompt-toolkit==3.0.43
Requires-Dist: fastparquet==2024.2.0
Requires-Dist: yandexcloud==0.316.0
Requires-Dist: openpyxl==3.1.5
Requires-Dist: python-docx==1.1.2
Requires-Dist: datetime==5.5

# LLAMATOR

Red Teaming python-framework for testing chatbots and LLM-systems

[![License: CC BY-NC-SA 4.0](https://img.shields.io/badge/License-CC_BY--NC--SA_4.0-lightgrey.svg)](https://creativecommons.org/licenses/by-nc-sa/4.0/)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/llamator)](https://pypi.org/project/llamator)
[![PyPI](https://badge.fury.io/py/llamator.svg)](https://badge.fury.io/py/llamator)
[![Downloads](https://pepy.tech/badge/llamator)](https://pepy.tech/project/llamator)
[![Downloads](https://pepy.tech/badge/llamator/month)](https://pepy.tech/project/llamator)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)

## Install 🚀

```bash
pip install llamator==2.0.0
```

## Documentation 📚

Documentation Link: [https://romiconez.github.io/llamator](https://romiconez.github.io/llamator/)

## Examples 💡

* 📄 [RAG bot testing via REST API](https://github.com/RomiconEZ/llamator/blob/release/examples/llamator-api.ipynb)
* 🧙‍♂️ [Gandalf web bot testing via Selenium](https://github.com/RomiconEZ/llamator/blob/release/examples/llamator-selenium.ipynb)
* 💬 [Telegram bot testing via Telethon](https://github.com/RomiconEZ/llamator/blob/release/examples/llamator-telegram.ipynb)
* 📱 [WhatsApp bot testing via Selenium](https://github.com/RomiconEZ/llamator/blob/release/examples/llamator-whatsapp.ipynb)
* 🔗 [LangChain client testing with custom attack](https://github.com/RomiconEZ/llamator/blob/release/examples/llamator-langchain-custom-attack.ipynb)

## Supported Clients 🛠️

* 🌐 All LangChain clients
* 🧠 OpenAI-like API
* ⚙️ Custom Class (Telegram, WhatsApp, Selenium, etc.)

## Unique Features 🌟

* ️🗡 Support for custom attacks from the user
* 👜 Large selection of attacks on RAG / Agent / Prompt in English and Russian
* 🛡 Custom configuration of chat clients
* 📊 History of attack requests and responses in Excel and CSV format
* 📄 Test report document in DOCX format

## OWASP Classification 🔒

* 💉 [LLM01: Prompt Injection and Jailbreaks](https://github.com/OWASP/www-project-top-10-for-large-language-model-applications/blob/main/2_0_vulns/LLM01_PromptInjection.md)
* 🕵 [LLM07: System Prompt Leakage](https://github.com/OWASP/www-project-top-10-for-large-language-model-applications/blob/main/2_0_vulns/LLM07_SystemPromptLeakage.md)
* 🎭 [LLM09: Misinformation](https://github.com/OWASP/www-project-top-10-for-large-language-model-applications/blob/main/2_0_vulns/LLM09_Misinformation.md)

## Community 🌍

* 📣 [Telegram Channel — AI Security Lab](https://t.me/aisecuritylab)
* 💬 [Telegram Chat — LLAMATOR | AI Red Team Community](https://t.me/llamator)

## Supported by 🚀

* [AI Security Lab ITMO](https://ai.itmo.ru/aisecuritylab)
* [Raft Security](https://raftds.ru/)
* [AI Talent Hub](https://ai.itmo.ru/)

## License 📜

© Roman Neronov, Timur Nizamov, Nikita Ivanov

This project is licensed under the terms of the **Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International** license. See the [LICENSE](LICENSE) file for details.

[![Creative Commons License](https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png)](https://creativecommons.org/licenses/by-nc-sa/4.0/)
