garak-multilingual 0.1.0

Turkish and multilingual prompt-injection / jailbreak probes for NVIDIA garak (LLM vulnerability scanner).

garak Turkish probes — a multilingual LLM vulnerability scanner plugin

garak-multilingual is a pip-installable plugin pack for NVIDIA garak that adds Turkish and multilingual prompt-injection, jailbreak and Turkish-casefold-evasion probes. It exists to make garak a better multilingual LLM vulnerability scanner and to give defenders a way to run non-English prompt injection testing against their own models — a surface that garak's English-first probes underserve.

If you already use garak to red-team your LLM, this pack lets you answer: does my guardrail hold up when the attack is written in Turkish, German, Spanish, Russian, Arabic, Chinese, Persian or French — or when an attacker abuses the Turkish dotted/dotless-I to slip a keyword past a filter?

This is authorized defensive security-testing tooling. Every payload is a probe you send to a model you own or are permitted to assess. See Responsible Use.

Turkish casefolding context: the maintainer's open garak contribution, NVIDIA/garak PR #1997 (Turkish dotted/dotless-I casefold buff), motivated the casefold-evasion probe in this pack.

Keywords

garak turkish probes · multilingual llm vulnerability scanner · non-english prompt injection testing · garak plugin · llm red teaming · turkish prompt injection · jailbreak probes · OWASP LLM Top 10 · MITRE ATLAS

Install

pip install garak-multilingual        # once published
# or, from a clone:
pip install .

This declares garak as a dependency. The payload corpus itself (garak_multilingual/payloads.py) is dependency-free and can be imported and audited without installing garak.

Usage

Run a single probe against any garak-supported model, e.g. the built-in test.Blank generator or your own endpoint:

garak --model_type test.Blank \
      --probes garak_multilingual.probes_ml.TurkishInjection

Other probes in this pack:

garak --model_type <your_model> --probes garak_multilingual.probes_ml.MultilingualInjection
garak --model_type <your_model> --probes garak_multilingual.probes_ml.TurkishCasefoldEvasion
garak --model_type <your_model> --probes garak_multilingual.probes_ml.TurkishJailbreak

Run all four:

garak --model_type <your_model> \
  --probes garak_multilingual.probes_ml.TurkishInjection,garak_multilingual.probes_ml.MultilingualInjection,garak_multilingual.probes_ml.TurkishCasefoldEvasion,garak_multilingual.probes_ml.TurkishJailbreak

Probes → OWASP LLM Top 10 → MITRE ATLAS

Probe class	Language(s)	What it tests	OWASP LLM Top 10	MITRE ATLAS
TurkishInjection	tr	Idiomatic Turkish instruction-override, role reassignment, system-prompt exfiltration ("Önceki tüm talimatları unut …")	LLM01: Prompt Injection	AML.T0051
MultilingualInjection	tr,de,es,ru,ar,zh,fr,fa	Same injection intent across ≥4 languages garak underserves	LLM01: Prompt Injection	AML.T0051
TurkishCasefoldEvasion	tr	Dotted/dotless-I keyword evasion (İGNORE / ıgnore) against ASCII lower() filters	LLM01: Prompt Injection	AML.T0051
TurkishJailbreak	tr	Turkish DAN / grandma / opposite-mode / affirmative-prefix jailbreaks	LLM01: Prompt Injection	AML.T0054 (Jailbreak)

Each probe carries garak tags (including owasp:llm01 and the ATLAS technique IDs above) and a recommended_detector (mitigation.MitigationBypass).

Honesty, coverage and provenance

This section is deliberate — the value of a security tool is undermined if it overclaims.

• Coverage, not superiority. These probes cover languages and a Turkish-specific casefold trick that garak's built-in, English-first probe families (promptinject, dan, latentinjection) do not focus on. They are a complement to, not a replacement for, those probes.
• Not "stronger" than English probes. Non-English payloads are not inherently more powerful. A robust guardrail may block all of these; a weak, English-only guardrail may fail them precisely because they are non-English. The point is to measure that gap, not to claim a better attack.
• Per-payload provenance. Every payload in payloads.py includes a note documenting the linguistic trick or archetype it exercises (e.g. which keyword carries a dotted İ, or which jailbreak family it adapts), so you can audit coverage rather than trust an opaque list. Counts: Turkish injection 16, multilingual 14 (8 languages), casefold evasion 12, Turkish jailbreak 14.
• Prior art / credit. Built on and for NVIDIA garak (Apache-2.0). The Turkish casefold work references the maintainer's open garak PR #1997. Jailbreak archetypes (DAN, "grandma exploit", opposite-mode, affirmative-prefix) are widely documented community techniques, here adapted into Turkish as framing only — no operational harmful content is included.
• No benchmark claims. This pack ships payloads and probe wiring. It does not ship success-rate numbers against any named model; measured bypass/refusal rates depend entirely on the target you run it against.
• Detectors. The recommended detector is a heuristic (mitigation.MitigationBypass); treat flagged hits as candidates to review, not ground truth. Tune detection to your own policy.

Responsible Use

These payloads are adversarial by design and provided only for authorized, defensive LLM security testing:

• Run them only against models and systems you own or have explicit written permission to test.
• The goal is to find and fix guardrail gaps (OWASP LLM01 / LLM02, MITRE ATLAS AML.T0051 / AML.T0054), not to produce harmful output. Payloads are framing/override attempts and do not contain operational instructions for causing harm.
• Follow responsible-disclosure practice for any third-party service you assess, and comply with all applicable law and platform terms.

Development & tests

The test suite is stdlib-only and does not import garak, so you can validate the corpus and probe structure without the heavy dependency:

python -m pytest -q
# or, without pytest installed:
python tests/run_tests.py

Tests assert: payloads.py imports with no third-party deps, every payload set is non-empty, all prompts are non-empty strings with provenance notes, BCP-47 tags are present and valid, the multilingual set covers ≥4 languages, every casefold payload actually contains a Turkish İ/ı, and probes_ml.py byte-compiles and declares the expected OWASP/ATLAS tags.

License

Apache-2.0. See LICENSE and NOTICE.

---

İlgili AltaySec Kaynakları

• 📖 Türkçe Prompt Injection: Saha Verisinden 5 Saldırı Kalıbı — konunun derinlemesine Türkçe analizi
• 🌐 AltaySec Araştırmalar — Türkçe yapay zekâ güvenliği yazıları

Atıf

@software{altaysec_garak_multilingual_probes_2026,
  author = {{AltaySec}},
  title  = {garak-multilingual-probes},
  year   = {2026},
  url    = {https://github.com/fevziegeyurtsevenler/garak-multilingual-probes}
}