Metadata-Version: 2.4
Name: garak-multilingual
Version: 0.1.0
Summary: Turkish and multilingual prompt-injection / jailbreak probes for NVIDIA garak (LLM vulnerability scanner).
Author-email: Fevzi Ege Yurtsevenler <egeyurtsevenler@gmail.com>
License: Apache-2.0
Project-URL: Homepage, https://github.com/fevziegeyurtsevenler/garak-multilingual-probes
Project-URL: Source, https://github.com/fevziegeyurtsevenler/garak-multilingual-probes
Project-URL: garak PR #1997, https://github.com/NVIDIA/garak/pull/1997
Keywords: garak,garak turkish probes,llm security,multilingual llm vulnerability scanner,non-english prompt injection testing,prompt injection,jailbreak,red team,owasp llm top 10,mitre atlas
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Information Technology
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Security
Classifier: Natural Language :: Turkish
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
License-File: NOTICE
Requires-Dist: garak
Provides-Extra: test
Requires-Dist: pytest; extra == "test"
Dynamic: license-file

# garak Turkish probes — a multilingual LLM vulnerability scanner plugin

**garak-multilingual** is a pip-installable plugin pack for
[NVIDIA garak](https://github.com/NVIDIA/garak) that adds **Turkish and
multilingual prompt-injection, jailbreak and Turkish-casefold-evasion probes**.
It exists to make garak a better **multilingual LLM vulnerability scanner** and
to give defenders a way to run **non-English prompt injection testing** against
their own models — a surface that garak's English-first probes underserve.

If you already use garak to red-team your LLM, this pack lets you answer:
*does my guardrail hold up when the attack is written in Turkish, German,
Spanish, Russian, Arabic, Chinese, Persian or French — or when an attacker
abuses the Turkish dotted/dotless-I to slip a keyword past a filter?*

This is authorized **defensive security-testing** tooling. Every payload is a
probe you send to a model you own or are permitted to assess. See
[Responsible Use](#responsible-use).

> Turkish casefolding context: the maintainer's open garak contribution,
> [NVIDIA/garak PR #1997](https://github.com/NVIDIA/garak/pull/1997)
> (Turkish dotted/dotless-I casefold buff), motivated the casefold-evasion
> probe in this pack.

## Keywords

garak turkish probes · multilingual llm vulnerability scanner ·
non-english prompt injection testing · garak plugin · llm red teaming ·
turkish prompt injection · jailbreak probes · OWASP LLM Top 10 · MITRE ATLAS

## Install

```bash
pip install garak-multilingual        # once published
# or, from a clone:
pip install .
```

This declares `garak` as a dependency. The **payload corpus itself
(`garak_multilingual/payloads.py`) is dependency-free** and can be imported and
audited without installing garak.

## Usage

Run a single probe against any garak-supported model, e.g. the built-in
`test.Blank` generator or your own endpoint:

```bash
garak --model_type test.Blank \
      --probes garak_multilingual.probes_ml.TurkishInjection
```

Other probes in this pack:

```bash
garak --model_type <your_model> --probes garak_multilingual.probes_ml.MultilingualInjection
garak --model_type <your_model> --probes garak_multilingual.probes_ml.TurkishCasefoldEvasion
garak --model_type <your_model> --probes garak_multilingual.probes_ml.TurkishJailbreak
```

Run all four:

```bash
garak --model_type <your_model> \
  --probes garak_multilingual.probes_ml.TurkishInjection,garak_multilingual.probes_ml.MultilingualInjection,garak_multilingual.probes_ml.TurkishCasefoldEvasion,garak_multilingual.probes_ml.TurkishJailbreak
```

## Probes → OWASP LLM Top 10 → MITRE ATLAS

| Probe class | Language(s) | What it tests | OWASP LLM Top 10 | MITRE ATLAS |
|---|---|---|---|---|
| `TurkishInjection` | `tr` | Idiomatic Turkish instruction-override, role reassignment, system-prompt exfiltration ("Önceki tüm talimatları unut …") | LLM01: Prompt Injection | AML.T0051 |
| `MultilingualInjection` | `tr,de,es,ru,ar,zh,fr,fa` | Same injection intent across ≥4 languages garak underserves | LLM01: Prompt Injection | AML.T0051 |
| `TurkishCasefoldEvasion` | `tr` | Dotted/dotless-I keyword evasion (İGNORE / ıgnore) against ASCII lower() filters | LLM01: Prompt Injection | AML.T0051 |
| `TurkishJailbreak` | `tr` | Turkish DAN / grandma / opposite-mode / affirmative-prefix jailbreaks | LLM01: Prompt Injection | AML.T0054 (Jailbreak) |

Each probe carries garak `tags` (including `owasp:llm01` and the ATLAS
technique IDs above) and a `recommended_detector`
(`mitigation.MitigationBypass`).

## Honesty, coverage and provenance

This section is deliberate — the value of a security tool is undermined if it
overclaims.

- **Coverage, not superiority.** These probes cover languages and a
  Turkish-specific casefold trick that garak's built-in, English-first probe
  families (`promptinject`, `dan`, `latentinjection`) do not focus on. They are
  a **complement to**, not a replacement for, those probes.
- **Not "stronger" than English probes.** Non-English payloads are not
  inherently more powerful. A robust guardrail may block all of these; a weak,
  English-only guardrail may fail them precisely *because* they are non-English.
  The point is to measure that gap, not to claim a better attack.
- **Per-payload provenance.** Every payload in `payloads.py` includes a `note`
  documenting the linguistic trick or archetype it exercises (e.g. which
  keyword carries a dotted `İ`, or which jailbreak family it adapts), so you can
  audit coverage rather than trust an opaque list. Counts: Turkish injection
  16, multilingual 14 (8 languages), casefold evasion 12, Turkish jailbreak 14.
- **Prior art / credit.** Built on and for
  [NVIDIA garak](https://github.com/NVIDIA/garak) (Apache-2.0). The Turkish
  casefold work references the maintainer's open
  [garak PR #1997](https://github.com/NVIDIA/garak/pull/1997). Jailbreak
  archetypes (DAN, "grandma exploit", opposite-mode, affirmative-prefix) are
  widely documented community techniques, here adapted into Turkish as *framing
  only* — no operational harmful content is included.
- **No benchmark claims.** This pack ships payloads and probe wiring. It does
  **not** ship success-rate numbers against any named model; measured
  bypass/refusal rates depend entirely on the target you run it against.
- **Detectors.** The recommended detector is a heuristic
  (`mitigation.MitigationBypass`); treat flagged hits as candidates to review,
  not ground truth. Tune detection to your own policy.

## Responsible Use

These payloads are adversarial by design and provided **only** for authorized,
defensive LLM security testing:

- Run them **only** against models and systems you own or have explicit written
  permission to test.
- The goal is to **find and fix** guardrail gaps (OWASP LLM01 / LLM02, MITRE
  ATLAS AML.T0051 / AML.T0054), not to produce harmful output. Payloads are
  framing/override attempts and do not contain operational instructions for
  causing harm.
- Follow responsible-disclosure practice for any third-party service you assess,
  and comply with all applicable law and platform terms.

## Development & tests

The test suite is stdlib-only and **does not import garak**, so you can validate
the corpus and probe structure without the heavy dependency:

```bash
python -m pytest -q
# or, without pytest installed:
python tests/run_tests.py
```

Tests assert: `payloads.py` imports with no third-party deps, every payload set
is non-empty, all prompts are non-empty strings with provenance notes, BCP-47
tags are present and valid, the multilingual set covers ≥4 languages, every
casefold payload actually contains a Turkish `İ`/`ı`, and `probes_ml.py`
byte-compiles and declares the expected OWASP/ATLAS tags.

## License

Apache-2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).

---

## İlgili AltaySec Kaynakları

- 📖 [Türkçe Prompt Injection: Saha Verisinden 5 Saldırı Kalıbı](https://altaysec.com.tr/arastirmalar/turkce-prompt-injection-5-saldiri-kalibi) — konunun derinlemesine Türkçe analizi
- 🌐 [AltaySec Araştırmalar](https://altaysec.com.tr/arastirmalar/) — Türkçe yapay zekâ güvenliği yazıları

## Atıf

```bibtex
@software{altaysec_garak_multilingual_probes_2026,
  author = {{AltaySec}},
  title  = {garak-multilingual-probes},
  year   = {2026},
  url    = {https://github.com/fevziegeyurtsevenler/garak-multilingual-probes}
}
```
