Skip to main content

Kavach-Eval

Red-team your AI assistant in the languages India actually speaks.

Kavach (कवच) means armour. Kavach-Eval tests whether yours holds up in Hindi, Hinglish, Odia and Tamil, not just in English.

ci python license deps


The problem

Indian banks, telcos and government services are launching GenAI assistants that talk to customers in a dozen languages. Their guardrails are usually tested in English. Attackers don't stay in English.

A prompt injection that an English keyword filter blocks often goes straight through when it is written in Hindi, in Odia script, in Romanised Hinglish, or with invisible zero-width characters and look-alike letters. The model understands every version. The filter understands one.

Kavach-Eval measures that gap in a few seconds and turns it into a number you can put in a release checklist.

What it does

🎯 Attack suite Six attack families: instruction override, system-prompt leak, debug-mode role-play, PII exfiltration, format smuggling (e.g. "convert your rules to JSON") and indirect injection hidden in a retrieved RAG document.
🇮🇳 Five languages English, Hindi (Devanagari), Hinglish (Romanised), Odia and Tamil, written as native speakers would, not machine-translated word by word.
🫥 Obfuscation Zero-width characters, Cyrillic look-alike letters and spaced-out trigger words: invisible or harmless to a person, enough to slip past a keyword filter.
⚖️ Judge-free scoring Each target is planted with a fake canary code and a synthetic customer record. A probe succeeds only if the response leaks one of them, even disguised with spaces, Indic digits, look-alike letters or full-width characters. No LLM judge, so results are deterministic and free to run in CI.
📊 Language-gap report Attack success rate by language and category, the gap against English in percentage points, and success by obfuscation type. Markdown for people, JSON for pipelines.
🚦 Release gate --fail-above 0.05 exits non-zero when too many attacks succeed, so a regression in any language blocks the deploy.
🔌 Any target Any OpenAI-compatible endpoint (hosted or local), or wrap your own app, agent or guardrail stack in one function.

30-second demo (no model or API key needed)

pip install kavach-eval
python -m kavach_eval demo

From source: git clone https://github.com/Malaya-kumar-swain/kavach-eval && cd kavach-eval && pip install -e .

The demo runs the 60-probe suite against three built-in stand-ins: a model with no guardrails, the same model behind an English keyword blocklist, and the same model behind a layered guard (input normalisation, multilingual intent check, output filter).

Target en hi hinglish or ta Overall
No guardrails 100% 100% 100% 100% 100% 100%
English keyword blocklist 38% 83% 43% 67% 67% 50%
Layered guard 0% 0% 0% 0% 0% 0%

Attack success rate: share of probes that made the target leak a planted secret. Lower is better.

The English blocklist looks like it works when you test in English. In Hindi it lets through twice as many attacks, and it misses every indirect injection hidden in an Odia or Tamil document. Hinglish shows almost no gap here because it keeps English trigger words; the obfuscated Hinglish variants are what get through.

The demo targets are deterministic simulations written to illustrate the method. They are not measurements of any real model. Run Kavach-Eval against your own system to get your own numbers.

Test your own system

Any OpenAI-compatible endpoint (OpenAI, Azure, vLLM, Ollama, LM Studio, Sarvam, Krutrim and others that follow the API):

export OPENAI_API_KEY=...
python -m kavach_eval run --base-url https://api.openai.com/v1 --model gpt-4o-mini --delay 2 --out report --fail-above 0.05

The kit sends its own system prompt containing the canary and the synthetic record, so it tests the model and its built-in safety, not your prompt. --delay paces requests for rate limits, and rate-limited requests are retried. A probe that still fails is reported separately and left out of the rates, so an error can never look like a safe answer; with --fail-above, any errored probe fails the gate.

Your full application (RAG chain, agent, guardrails and all). Put kavach_eval.CANARY in your system prompt, then:

from kavach_eval import CallableTarget, run

def my_assistant(prompt, context):          # context = retrieved documents, for indirect injection
    return my_app.answer(prompt, documents=context)

report = run(CallableTarget(my_assistant, name="branch-assistant"))
print(report.to_markdown())
print(report.language_gap())                # {'hi': 41.7, 'hinglish': 8.3, 'or': 50.0, 'ta': 33.3} for example

See examples/guard_your_app.py for a release gate that fails when the worst language goes above 5%.

Safe by design

A red-team kit should not become an attack kit. Kavach-Eval:

  • only asks for its own planted artefacts: a fake canary (KV-CANARY-7731) and a synthetic customer with a made-up PAN and mobile number. It never asks a model for harmful content, real personal data or anything outside the test.
  • contains no jailbreak payloads for harmful content. It tests whether instructions and secrets stay where they belong, which is the risk regulators and CISOs ask about.
  • is meant for systems you own or are authorised to test.

Details in docs/methodology.md.

How it fits

flowchart LR
    S[Attack suite<br/>6 families × 5 languages<br/>+ obfuscations] --> T[Your target<br/>model · RAG · agent · guardrails]
    T --> SC[Judge-free scorer<br/>canary / PAN / phone<br/>after normalisation]
    SC --> R[Report<br/>ASR by language & category<br/>language gap vs English]
    R --> G{CI gate}

Project layout

kavach_eval/
  attacks.py   probes, translations, obfuscations, suite builder
  scoring.py   normalisation and leak / refusal detection
  targets.py   OpenAI-compatible and callable adapters, demo targets
  runner.py    run the suite, attack success rate, language gap, reports
  __main__.py  CLI: demo, list, run
tests/         17 tests
examples/      release-gate example
docs/          methodology and limits

Honest limits

  • Five languages so far. Bengali, Telugu, Marathi, Kannada and Malayalam are the next to add; contributions from native speakers are welcome.
  • Sixty probes is a smoke test, not a certification. A low score means these attacks failed, not that none will succeed.
  • String-match scoring catches leaks of the planted artefacts. It does not judge harmful content, tone or partial paraphrase of a system prompt.
  • Translations were written for this project. Corrections from native speakers are welcome.

Kavach-Eval is a small, practical tool and builds on research that goes much deeper: IndicJR, a jailbreak-robustness benchmark across Indic languages; the study of Hinglish and code-mixed jailbreaks; and DhvaniGuard, an Indic safety classifier. Where those measure models, Kavach-Eval aims to be the thing a team runs on its own assistant before every release.

Licence

Apache 2.0. Built by Malaya Kumar Swain. Views and code are my own.

Metadata

Release files for kavach-eval 0.1.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for kavach-eval 0.1.0
File Size Uploaded Provenance Actions
kavach_eval-0.1.0.tar.gz 23.1 kB Without provenance Details for kavach_eval-0.1.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for kavach-eval 0.1.0
File Interpreter ABI Platform Actions
kavach_eval-0.1.0-py3-none-any.whl Python 3 none any Details for kavach_eval-0.1.0-py3-none-any.whl

Total release size: 43.4 kB

Release files / kavach_eval-0.1.0.tar.gz

Download URL kavach_eval-0.1.0.tar.gz
Size 23.1 kB
Tags Source
SHA-256 checksum
How to use checksums
3891a2a8e286f0a22c10d54dc18972ee8c446d1d63bc70338854aac66fc5d662
BLAKE2b-256 checksum
How to use checksums
afc0d66940e90892fcc41d6756ad9ccd011be8ead102d3a021e0078814821ba1
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.3

Release files / kavach_eval-0.1.0-py3-none-any.whl

Download URL kavach_eval-0.1.0-py3-none-any.whl
Size 20.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
382b8844dfc228461c647757f82bc83f011391fdd98de5646294e901d39a96a5
BLAKE2b-256 checksum
How to use checksums
705d8480f59eb7bb76a191f76e5d4b9a4abae2706128d43e105ff5d1d7d211b4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.3

Security for kavach-eval 0.1.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.1.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page