Sudanese Arabic Content Moderation: What Models Get Wrong Without Native Annotators

Why Sudanese Arabic Content Moderation Fails Without Native Annotators

Content moderation for Arabic-language platforms has historically concentrated on MSA, Egyptian, and Gulf Arabic — the three varieties with the largest commercial user bases and the most available training data. Sudanese Arabic, spoken by approximately 33 million people across Sudan and the Sudanese diaspora, sits outside every major Arabic moderation training corpus. The result is moderation pipelines that simultaneously over-flag benign Sudanese content and under-detect genuine harm.

The over-flagging problem is driven by Nubian-origin vocabulary in Sudanese Arabic. Words borrowed from Nobiin, Dongolawi, Beja, and Nuba Mountains languages form a substantial layer of everyday Sudanese Arabic vocabulary. Several of these loanwords are phonetically close to MSA abusive terms — producing false positives when MSA-trained classifiers encounter common Sudanese expressions. Platform users whose benign posts are repeatedly removed lose trust in the platform; appeals queues fill with legitimate content.

The under-detection problem is structurally different. Research on under-resourced Arabic dialect moderation (Abdul-Mageed et al., PADIC corpus analysis, 2020) documents F1 degradation of 35–50% when MSA classifiers are applied to Nile Valley dialect content. Sudanese Arabic community-register harm expression — tribal-register indirect hostility, conflict-era political incitement, in-group slurs for Nile Valley ethnic groups — is absent from MSA training data entirely.

Our Arabic NLP annotation service provides Sudanese Arabic moderation annotation with native annotators across Khartoum urban, Northern Nile Valley, and regional varieties, with conflict-era vocabulary awareness and adjudicated review for politically sensitive content categories.

Four Sudanese Arabic Characteristics That Break MSA Moderation

1. Nubian-origin vocabulary and false positives

Sudanese Arabic has absorbed substantial vocabulary from Nile Nubian languages — particularly Nobiin and Dongolawi — across centuries of language contact in the Nile Valley. This substrate vocabulary covers food, household objects, family terms, greetings, and everyday activities. Several Nubian-origin words in common Sudanese Arabic use share phonological sequences with MSA terms that carry abusive or sexual connotations in Gulf and Egyptian moderation training corpora.

The practical consequence is that routine Sudanese Arabic social media posts — discussing food, family visits, regional customs — trigger keyword and n-gram features in MSA-trained classifiers that were trained to detect harm in different vocabulary contexts. A platform serving both Egyptian and Sudanese users will systematically over-moderate Sudanese content unless the moderation classifier has been trained on Sudanese Arabic-specific annotation that correctly labels these vocabulary items as benign in their Nile Valley context.

Non-Sudanese Arabic annotators from Egypt or the Gulf are not equipped to resolve these cases. Without knowledge of the Nubian-substrate vocabulary layer and its semantic range in Sudanese usage, a non-native reviewer is as likely to confirm a false positive as to override it — the word looks harmful in an MSA reading, even when Sudanese context makes its benign meaning obvious to a native speaker.

2. Tribal-register indirect hostility

Sudanese Arabic communication norms — particularly in contexts involving inter-tribal or inter-regional relations — encode hostility through indirect-speech patterns that avoid explicit abusive vocabulary. Tribal-register Sudanese Arabic uses formulaic praise constructions that function as insults through contextual reversal, rhetorical questions that imply ethnic or regional derogation without stating it, and proverb deployment that references tribal stereotypes opaque to non-Sudanese readers.

MSA moderation classifiers, trained predominantly on explicit-vocabulary harm signals, classify these constructions as neutral or positive — high-frequency praise words produce low harm scores. Human reviewers without Sudanese cultural context make the same error. Only native Sudanese annotators familiar with the specific formulaic patterns, the ethnic referents in the underlying tribal knowledge, and the community context of specific disputes can label this content accurately.

The inter-annotator agreement challenge is significant for this category. Two native Sudanese annotators from different tribal backgrounds may disagree on whether a specific indirect-speech construction crosses the harm threshold — particularly in political contexts where the tribal reference is contested. Robust moderation annotation requires an adjudication protocol where politically sensitive content with annotator disagreement above 20% goes to a third-pass review by a senior Sudanese annotator with cross-regional awareness.

3. Conflict-era vocabulary: 2019 revolution and post-2023 civil war

Sudanese Arabic has undergone rapid lexical change since the 2018–2019 revolution that ended Omar al-Bashir's three-decade rule, and again since the April 2023 outbreak of civil conflict between the Sudanese Armed Forces (SAF) and Rapid Support Forces (RSF). Both periods produced new political vocabulary — terms that acquired incitement, derogatory, or dehumanising connotations within the Sudanese community context that no pre-2019 training corpus can capture.

Several terms that were politically neutral before 2019 became coded signals of factional alignment, ethnic targeting, or incitement to violence in post-2023 Sudanese Arabic social media. MSA classifiers trained before 2023 — and non-Sudanese Arabic annotators without current cultural awareness — cannot reliably identify these terms as harmful. A Sudanese Arabic content moderation annotation taxonomy must include a conflict-era vocabulary module with regular updates to track neologisms and semantic drift in politically sensitive terms.

For platforms with Sudanese diaspora user bases — which are often more politically active than the in-Sudan population due to access to international platforms — conflict-era vocabulary monitoring is a pressing operational need. Diaspora Sudanese Arabic social media is a primary amplification channel for conflict information, making accurate moderation particularly high-stakes.

4. Code-switching and cross-pipeline gaps

Educated Khartoum Sudanese Arabic users — the demographic most active on international platforms — produce high rates of English sentence-level code-switching. In moderation contexts, this creates cross-pipeline gaps: English-embedded harmful content inside Sudanese Arabic grammatical structures presents to the Arabic moderation system as partially-Arabic text with unrecognised segments, while Sudanese Arabic abusive constructions using English vocabulary present to the English moderation system as Arabic-framed content below its detection threshold.

A specific pattern in Sudanese Arabic political social media is the use of English technical or news vocabulary within Arabic incitement constructions — exploiting the fact that Arabic content moderation is trained on Arabic vocabulary harm signals and English moderation is trained on English ones. Native Sudanese bilingual annotators, operating fluently in both registers, label these constructions accurately. A pure Arabic-language annotator pool cannot.

Case Study: Khartoum Social Commerce Platform — 43% False Positive Rate to 8%

A pan-African social commerce platform with approximately 180,000 Sudanese active users operated a unified Arabic moderation pipeline built on an MSA-trained classifier fine-tuned on Egyptian and Saudi Arabic content. The platform's Trust & Safety team observed a persistent pattern of Sudanese user complaints about moderation removals — posts that users insisted were benign receiving automated removal followed by upheld appeals, creating a backlog of manual review requests that consumed Trust & Safety capacity.

Before: An audit of 2,400 Sudanese Arabic posts that had been automatically removed revealed that 43.2% were false positives — content that native Sudanese reviewers unanimously labelled as benign on re-review. The false positive rate for posts containing Nubian-substrate vocabulary was 67.4%. Meanwhile, a human-reviewed sample of 800 posts that had passed the automated classifier found that 18.6% contained indirect tribal-register hostility or conflict-era political incitement that the classifier had not flagged. The platform's effective moderation recall on genuinely harmful Sudanese Arabic content was 31.2% — meaning roughly seven in ten harmful posts reached users unmoderated.

The annotation project built a Sudanese Arabic-specific moderation taxonomy covering five harm categories: explicit hate speech and slurs (Sudanese dialect vocabulary); indirect tribal-register hostility; conflict-era political incitement; harassment targeting ethnic and regional identity; and self-harm content in Sudanese Arabic expression. The annotation team comprised eight native Sudanese annotators — five Khartoum urban-native, two Northern Nile Valley-native, one Kordofan-native — plus a senior Sudanese Arabic reviewer for conflict-era political content adjudication. The team labelled 28,000 posts across the five categories, with 15% adjudicated by independent pair-review.

After fine-tuning the classifier on the Sudanese annotation: Overall false positive rate on the Sudanese Arabic user base fell from 43.2% to 8.1%. Nubian-vocabulary false positives fell from 67.4% to 11.3%. Moderation recall on genuinely harmful Sudanese Arabic content improved from 31.2% to 78.6%. Manual appeal volume from Sudanese users declined by 71% within 90 days of deployment, reducing Trust & Safety manual review hours by an estimated 340 hours per month. The annotation project cost AUD $38,400 in total; the Trust & Safety team attributed monthly operational savings of approximately AUD $42,000 in manual review staff hours at equivalent contractor rates.

Building a Sudanese Arabic Moderation Taxonomy

Effective Sudanese Arabic content moderation annotation requires a taxonomy that differs materially from standard Arabic moderation taxonomies. The following categories require Sudanese-specific treatment:

Nubian-vocabulary exception list. The taxonomy must include an explicit list of Nubian-origin Sudanese Arabic terms that overlap phonetically with MSA abusive vocabulary but are benign in Sudanese context. This list requires input from Sudanese Arabic linguists or senior Sudanese annotators with Nile Valley vocabulary expertise. The list should be version-controlled and reviewed quarterly as new Nubian-substrate vocabulary enters Sudanese digital usage.

Tribal-register harm definitions with examples. Indirect tribal-register harm cannot be labelled correctly without examples. The taxonomy must include worked examples of the specific formulaic constructions — the formulaic praise-as-insult patterns, the rhetorical question derogation frames, the proverb deployments with ethnic referents — that constitute indirect harm in Sudanese Arabic community context. Each example should include the Sudanese Arabic text, a plain-language explanation of the harm mechanism, and the correct label.

Conflict-era vocabulary module. Post-2019 and post-2023 vocabulary requires a dedicated module that is updated at least quarterly by a Sudanese Arabic annotator with current awareness of Sudanese political discourse. This module should track semantic drift — terms that have shifted from neutral to harmful — as well as entirely new terms coined during the conflict period.

For guidance on structuring the broader annotation pipeline and quality controls, see our posts on annotation QA and relabeling and writing annotation guidelines that survive production.

Regional Variety Coverage for Moderation Annotation

Sudanese Arabic moderation annotation is not uniform across the country's major dialect regions. Khartoum urban Arabic is the dominant digital register and the primary source of social media content, but Northern Nile Valley, Kordofan, and Darfur varieties each carry region-specific vocabulary and community-register harm patterns that Khartoum-only annotator pools cannot reliably classify.

Northern Nile Valley content — from the Dongola, Shendi, and Atbara corridor — shows stronger Nubian vocabulary density, higher frequency of tribal-register indirect speech, and different political vocabulary from Khartoum urban content. A moderation annotator pool drawn entirely from Khartoum backgrounds will produce higher error rates on Northern Nile Valley content than on Khartoum content, particularly for indirect-speech harm categories where community knowledge is geographically specific.

For platforms with national Sudanese user coverage, the minimum viable annotator pool stratification is Khartoum urban (50–60%), Northern Nile Valley (20–25%), and Kordofan or Darfur (15–20%). Diaspora Sudanese annotators — from the Australian, UK, US, and Gulf Sudanese communities — are suitable for most harm categories but require specific briefing on conflict-era vocabulary if they left Sudan before 2019.

Our Arabic NLP annotation service maintains Sudanese annotator pools across Khartoum, Northern Nile Valley, and regional varieties, with conflict-era vocabulary training and adjudication protocols for politically sensitive moderation categories.

Inter-Annotator Agreement and Quality Control

Sudanese Arabic moderation annotation presents distinctive inter-annotator agreement challenges. For explicit harm categories — clear slurs, direct threats — native Sudanese annotators achieve Fleiss's kappa of 0.78–0.85, comparable to well-specified moderation tasks in other languages. For indirect tribal-register hostility and conflict-era political content, kappa typically falls to 0.52–0.68 — reflecting genuine category ambiguity rather than annotator error.

The appropriate response to lower kappa in political content is not to lower the label quality threshold but to increase the adjudication protocol stringency. Content where two first-pass annotators disagree should go to a third-pass senior Sudanese reviewer with cross-regional political awareness. Items where the three-annotator result is still split should be escalated to a platform policy team decision — these are genuine category-boundary cases where the annotation reflects the policy question, not annotation error.

For moderation classifiers, tracking inter-annotator agreement by content category over time is a quality leading indicator. A decline in kappa for a specific category — for example, conflict-era political incitement — typically signals that the taxonomy needs updating to reflect vocabulary drift in that category, before the classifier performance degrades visibly in production.

See our Cohen's kappa annotation quality guide for a detailed treatment of IAA metrics and the misreadings that hide real quality problems.

Related Reading

• Sudanese Arabic Sentiment Analysis: What Models Get Wrong Without Native Annotators
• Sudanese Arabic Named Entity Recognition: What Models Get Wrong Without Native Annotators
• Gulf (Khaleeji) Arabic Content Moderation: What Models Get Wrong Without Native Annotators
• Arabic NLP Annotation Service

Frequently Asked Questions