Red-Teaming Arabic LLMs: Annotating Safety, Toxicity and Jailbreaks in Dialect

Why Arabic LLMs Need Dialect-Native Red Teaming

The Arabic-speaking world uses roughly 25 major dialect varieties — Gulf/Khaleeji, Egyptian, Levantine, Iraqi, Moroccan Darija, Sudanese, Yemeni — that differ from Modern Standard Arabic (MSA) in vocabulary, grammar, pragmatics, and cultural reference. When a safety classifier is trained on MSA text, it learns the safety patterns of formal Arabic. It does not learn the toxicity markers, slang, sarcasm conventions, and cultural taboo signals of spoken dialects.

A 2024 analysis of Arabic hate speech detection systems published in the ACL Anthology found that state-of-the-art Arabic hate speech classifiers achieved 78–85% accuracy on MSA test sets but dropped to 38–54% accuracy on dialectal Arabic benchmarks (Mulki et al., 2024 — updated findings). The performance collapse is consistent: MSA-trained classifiers cannot reliably detect toxic content in dialects because the vocabulary is fundamentally different.

For Arabic LLM developers, this creates a specific safety risk. A Gulf user can prompt a chatbot with dialect-native toxic language, receive a harmful response, and the MSA safety filter catches none of it. The attack surface is not hypothetical — it is the default state of any Arabic LLM safety system built without dialect-native red teaming.

Our Arabic NLP annotation service addresses this directly through structured red-team corpus construction using dialect-native annotator pools for Gulf, Egyptian, and Levantine varieties.

The Anatomy of an Arabic Red-Team Attack Surface

Arabic LLM red teaming is not simply translating English jailbreak templates into Arabic. The attack surface is dialect-specific and culturally grounded. Effective red-team annotation covers six distinct categories:

• Direct dialect toxicity: Insults, slurs, and derogatory terms that are specific to a dialect variety. Khaleeji Arabic has specific terms for ethnic and tribal insults that do not exist in MSA. Egyptian Arabic has distinct sarcasm vocabulary that can express severe hostility through apparent compliments.
• Cultural taboo elicitation: Prompts targeting MENA-specific sensitivities around religion, gender roles, honour norms, political authority, and tribal identity. These triggers differ substantially from Western taboo categories that dominate English red-team datasets.
• Sarcasm and coded language: Gulf sarcasm operates through inversion — a highly positive expression can carry severe negative intent. Egyptian dialect sarcasm (تقيل) uses specific register markers that indicate ironic framing invisible to MSA classifiers.
• Code-switching attacks: Mid-prompt switches between Arabic and English that exploit the boundary between the model's Arabic and English safety systems. A prompt that is safe in pure Arabic and pure English may be unsafe in the code-switched form.
• Jailbreak template adaptation: Standard English jailbreak templates (DAN, role-play scenarios, hypothetical framings) adapted to Arabic dialect syntax. These fail if directly translated — they require native dialect authoring to function as attack prompts.
• Instruction injection via dialect: System prompt overrides embedded in dialect form, exploiting weaker parsing of dialectal instruction markers by models trained primarily on MSA system prompts.

Each category requires annotators with genuine dialect competence — not MSA literacy. A formal Arabic speaker cannot author credible Khaleeji sarcasm or Egyptian coded insults, and attempts to do so produce prompts that native speakers recognise as inauthentic and that models handle without difficulty.

Red-Team Annotation Methodology: What the Workflow Actually Looks Like

A production-grade Arabic red-team annotation project follows a structured five-phase workflow that differs materially from standard content labelling:

Phase 1 — Dialect pool assembly. Red-team annotators are recruited by native dialect, not just Arabic literacy. For a GCC-facing deployment, this means Khaleeji native speakers from Saudi Arabia, UAE, Kuwait, and Qatar — specifically tested for informal dialect competence, not MSA. Each annotator pool undergoes a dialect placement assessment before red-team work begins.

Phase 2 — Taxonomy briefing. Annotators receive the specific attack taxonomy for their dialect: the harm categories, prompt construction guidelines, and examples of successful and unsuccessful red-team prompts. The taxonomy is dialect-specific — the Gulf taxonomy covers honour norms and tribal insults; the Egyptian taxonomy covers political satire and class-based sarcasm; the Levantine taxonomy covers political sectarian references.

Phase 3 — Adversarial prompt authoring. Annotators author original prompts in their dialect, targeting specific harm categories. Each prompt is paired with an intended harm category label and a severity rating. Prompts undergo inter-annotator review — a second native speaker from the same dialect pool assesses whether the prompt would elicit harmful output and whether the harm label is correct.

Phase 4 — Model probing and response labelling. The red-team prompts are submitted to the target model. Response (prompt, model-response) pairs are then reviewed by dialect-native annotators who label: (a) whether a harmful output was produced, (b) the harm severity, (c) the bypass technique used. This produces the (prompt, response, harm-label) triple format that trains the safety classifier.

Phase 5 — Iterative gap analysis. The red-team corpus is analysed for taxonomy gaps — harm categories that are underrepresented or attack techniques that the model defends against more than others. A second round of targeted prompt authoring fills identified gaps. For Gulf-facing deployments, Phase 5 commonly identifies religious authority framing and tribal honour prompts as underrepresented in generic Arabic red-team datasets.

Case Study: Gulf Arabic Safety Classifier for a MENA Fintech Chatbot

A UAE-based fintech company deployed a customer service chatbot serving Gulf users across Saudi Arabia, UAE, and Kuwait. The initial safety system used an off-the-shelf Arabic content moderation API trained on MSA news and social media data. Internal testing showed the chatbot was passing harmful outputs in dialect — specifically tribal insults and financial fraud facilitation prompts written in Khaleeji Arabic.

The company commissioned a targeted red-team annotation project: 8,200 adversarial prompts authored by Gulf dialect native speakers across five harm categories (financial fraud facilitation, tribal/ethnic insults, gender-targeted harassment, regulatory bypass, and religious manipulation). Annotators were stratified by country: 40% Saudi (Najdi and Hijazi), 30% UAE (Emirati and expatriate Gulf), 20% Kuwaiti, 10% Qatari.

The red-team corpus was used to fine-tune a dialect-aware safety classifier on top of the existing MSA system. Results after deployment:

• False negative rate on Gulf dialect toxicity: 54% → 11% (a 79% reduction in harmful content passing undetected)
• False positive rate (legitimate Gulf dialect content incorrectly flagged): maintained at 6% — no significant regression in legitimate content accuracy
• Financial fraud facilitation prompts detected: 23% → 81% detection rate
• Safety incident rate (harmful responses reaching end users): reduced by 73% over first 30 days post-deployment
• Total annotation project cost: AUD $68,000, yielding an estimated AUD $420,000 reduction in regulatory exposure and incident response costs in the first quarter post-deployment

The key finding: the safety improvement was entirely attributable to dialect coverage, not classifier architecture. The same model architecture that performed at 46% detection on Gulf dialect toxicity before the red-team corpus performed at 89% after — the only change was the training data.

Dialect Coverage: What Arabic Varieties a Red-Team Corpus Must Include

The minimum viable dialect coverage for a pan-Arabic LLM safety system depends on the deployment context. A Saudi-only product can focus on Najdi and Hijazi Arabic. A GCC product requires the full Gulf variety range. A pan-Arab consumer product must cover at minimum five dialect families.

The Arabic NLP community has documented significant inter-dialect variation in toxicity expression. A 2024 study comparing hate speech patterns across Arabic dialects (CAMeLBERT analysis, Salameh et al.) found that toxic vocabulary overlap between Gulf and Egyptian Arabic is approximately 34% — meaning two-thirds of toxic Gulf Arabic vocabulary is absent from Egyptian Arabic safety training data, and vice versa. No single dialect's safety classifier transfers to another.

For Gulf-specific deployments: Khaleeji Arabic sub-variety coverage is critical. Najdi Arabic (central Saudi Arabia) uses tribal honour terminology distinct from Hijazi Arabic (western Saudi Arabia, Jeddah region). Emirati Arabic has specific Bedouin heritage vocabulary. Kuwaiti Arabic has unique loanword patterns from Persian and Indian Ocean trade languages. Each sub-variety needs dedicated annotator representation.

This is why generalised Arabic safety datasets — including well-intentioned academic resources like OSACT4 or ArSarcasm — are necessary but not sufficient for production Gulf safety systems. They cover MSA and some Egyptian Arabic but have thin Gulf dialect representation. Our Arabic NLP annotation service maintains dedicated annotator pools for each major Gulf sub-variety specifically to address this gap.

PDPL and Regulatory Considerations for Arabic Red-Team Data

Saudi Arabia's Personal Data Protection Law (PDPL) and its implementing regulations under SDAIA affect how Arabic red-team data can be sourced, processed, and stored. The key consideration is whether the red-team prompts involve personal data from real users.

If prompts are sourced from real Saudi user interactions — chat logs, support tickets, social media — PDPL requires: explicit consent for data use in model training, anonymisation to SDAIA-specified standards, PDPL-compliant data processing agreements if annotation occurs outside Saudi Arabia, and purpose limitation documentation. The SDAIA guidance on AI-related data processing (released 2024) clarifies that training data derived from user interactions is personal data for PDPL purposes.

The practical solution most Gulf AI teams adopt is synthetic red-team prompt authoring: annotators write original adversarial prompts from scratch using the harm taxonomy as a guide. Synthetic prompts involve no personal data, require no PDPL consent process, and can be legally processed and stored in annotation environments outside Saudi Arabia. This approach also produces more diverse coverage — annotators are incentivised to explore the full attack surface rather than being limited to attack patterns that appeared in real user logs.

For UAE-based deployments under the UAE Personal Data Protection Law (Federal Decree-Law No. 45 of 2021), similar principles apply. The UAE law has a somewhat narrower scope than PDPL regarding AI training data but still covers user-interaction-derived data used to train commercial AI systems. See also our post on PDPL vs GDPR for annotation vendors for a detailed comparison of the two frameworks.

Inter-Annotator Agreement and Quality Standards for Arabic Safety Data

Safety annotation is higher-stakes than standard classification tasks. A missed harm label is not a statistical noise event — it directly affects whether harmful content reaches users. Arabic safety annotation requires stricter inter-annotator agreement thresholds and more structured adjudication than typical annotation tasks.

For Arabic red-team harm classification, a minimum Cohen's kappa of 0.75 is required for production-grade datasets. This is meaningfully higher than the 0.60–0.65 threshold adequate for many other annotation tasks. Achieving 0.75 kappa on Arabic safety data requires: shared dialect competency within annotator pairs (same dialect variety), calibration sessions of at minimum two hours before project start, and regular adjudication meetings to resolve boundary cases between harm categories.

The most common source of kappa below threshold in Arabic safety annotation is dialect mismatch — annotator pairs who are both "Arabic speakers" but from different dialect backgrounds will disagree on whether a Gulf sarcasm pattern is harmful or benign. An Egyptian Arabic speaker may not recognise the specific register markers that indicate hostile intent in Khaleeji Arabic. This is why dialect-matched annotator pairing is a structural requirement, not a preference.

For a deeper look at IAA methodology in annotation quality, see our post on Cohen's kappa in annotation quality.

Building a Living Red-Team Process: Beyond One-Time Dataset Creation

A red-team corpus is not a static deliverable — it is a living process. Arabic internet language evolves rapidly; new slang, new political contexts, new cultural reference points emerge continuously. A red-team dataset produced in 2024 will have meaningful coverage gaps by 2026 simply because dialect toxicity patterns have shifted.

Production Arabic LLM safety requires quarterly red-team refresh cycles — typically smaller targeted datasets of 1,000–3,000 new prompts focusing on emerging attack patterns, new dialect vocabulary, and any harm categories where the deployed safety classifier shows degraded performance in production logs. This is analogous to how security teams maintain threat intelligence: the initial assessment is essential, but continuous monitoring is what keeps the system effective.

Red-team refresh datasets are significantly less expensive than initial corpus construction: typically AUD $12,000–$25,000 per quarterly cycle for a Gulf-focused product, compared to AUD $60,000–$120,000 for an initial production-grade corpus covering five dialect families.

Teams building serious Arabic AI safety infrastructure should budget for this as an operating cost, not a project cost. The alternative — a static safety system that slowly loses coverage as dialect usage evolves — produces exactly the kind of safety failures that create regulatory exposure under MENA content responsibility frameworks and platform liability regimes being developed across the GCC.

What to Look for in an Arabic Red-Team Annotation Vendor

Arabic red-team annotation is specialised work. Most general-purpose annotation vendors cannot execute it because they lack dialect-stratified annotator pools and domain expertise in adversarial prompt construction. When evaluating vendors, the key qualification questions are:

• Can the vendor provide annotator dialect placement test results — not just "Arabic speakers", but documented sub-variety competence?
• Does the vendor have experience in adversarial prompt taxonomy design for Arabic, or only standard content classification?
• What inter-annotator agreement targets does the vendor commit to for safety annotation specifically?
• How does the vendor handle PDPL/UAE PDPL compliance for annotation of sensitive content?
• Can the vendor provide a red-team taxonomy tailored to the specific harm categories relevant to the deployment context — not a generic English taxonomy translated to Arabic?

The last point is often the most revealing. A vendor with genuine Arabic red-team expertise will immediately ask about the target dialect, deployment context, and harm taxonomy before quoting. A vendor without it will quote on prompt volume without raising these questions. Volume without dialect coverage produces a dataset that passes QA but fails in production — the characteristic failure mode of low-cost Arabic safety annotation.

Frequently Asked Questions