">

🛡️ Static Hugging Face Space · runs entirely in your browser

GuardArena Live — a two-axis EN/TR prompt-injection guardrail benchmark & leaderboard

A bring-your-own-guardrail leaderboard for prompt-injection guard comparison in English and Turkish. Score any guard on the two error rates that matter in production — jailbreak miss-rate and benign over-refusal — reported overall and split EN vs TR.

Add a guard to the board

Guard source
Blocklist mode

Security: this page runs no code you supply. A blocklist is matched only as a substring or a regular expression, entirely in your browser. Nested-quantifier (catastrophic-backtracking) regexes are rejected. See the security note below.

Run a guard to see its scorecard. Baselines are pre-seeded below.
Two-axis trade-off — bottom-left corner is ideal (few misses, little over-refusal).

Leaderboard

Ranked by the mean of the two error axes — a stated convenience sort key, not a “best” verdict. Lower is better on every axis. Read both columns.

Per-example breakdown (last-run guard — mistakes first)

Methodology, security & honest limitations

Why two axes, split by language

Most public guardrail numbers report attack detection alone — half a metric, since a guard that blocks everything gets a perfect miss-rate and is useless. The real question is the trade-off: how few attacks you miss for a given tolerance of over-refusal, and whether that holds up outside English. Rates are kept separate for EN and TR because a guard strong in English routinely collapses on Turkish morphology, and a single blended number hides exactly that.

Held setA fixed, dated snapshot of fevziegeyurtsevenler/guardrail-arena (split train), embedded in this page.
Attack (label 1)Prompt-injection / jailbreak / exfiltration attempt.
Benign (label 0)Harmless prompt, including security-adjacent text that merely discusses attacks.
Jailbreak miss-rateattacks_missed / attacks — false-negative rate. Lower is better.
Over-refusal ratebenign_blocked / benign — false-positive rate. Lower is better.
SplitEvery rate is also computed for language == "en" and language == "tr".

Security model — no arbitrary code execution

  • A guard is only a substring or regular-expression blocklist. There is no eval, no code execution, no network call — everything runs locally in your browser.
  • Blocklists are bounded (max patterns, max pattern length, capped scan length).
  • Regex denial-of-service (ReDoS): the classic nested-quantifier family (e.g. (a+)+, (a*)*, (.*x){2,}) is rejected before it runs, and the scanned text is capped. This is a heuristic targeting the dominant catastrophic-backtracking shape — not a proof of safety. Prefer literal mode, which never treats input as a regex.
  • All dataset text and guard names are rendered as inert text (never as HTML), so attack strings in the data cannot execute in the page.

Responsible use

This is authorized, defensive security-testing tooling — it measures how well a guardrail resists prompt-injection and how much it over-blocks legitimate users. It maps to OWASP LLM01 (Prompt Injection) and MITRE ATLAS AML.T0051 / AML.T0054. Evaluate defences; do not weaponize. Use only against systems you own or are authorized to test.

Honesty & limitations (please read)

  • No “best” / “first” / “only” claims. The tool reports two error rates and a stated sort key; the right operating point depends on your risk tolerance.
  • Dated snapshot. Scores are only meaningful against the embedded snapshot; the upstream dataset can change.
  • Small held set (337 rows). Rates are estimates with real sampling error — treat single-percent differences as noise.
  • Regex/substring guards are a lower bound on what a real (model-based / semantic) guard can do. Beating a baseline here does not make a guard production-ready.
  • Turkish rules are heuristic (suffix-aware regex, not a morphological analyzer); they miss paraphrase and heavy obfuscation.
  • The board is per-session and lives only in this browser tab; reloading reseeds the baselines.
  • Reproducible: the browser mirrors the audited Python in scoring.py/guards.py; the built-in baselines below reproduce the numbers pytest and that code produce on this snapshot.

Related public efforts (complemented, not replaced)

  • Lakera Gandalf and Gray Swan Arena — live human red-team arenas (interactive attack, not a fixed scored held set).
  • OpenAI moderation, Meta Llama Guard / Prompt Guard — model-based guards; this project ships no model of its own.
  • garak (NVIDIA) and promptfoo — probe/red-team harnesses; GuardArena is narrower on purpose: one dataset, two axes, EN/TR split, no code execution.