Sudanese Arabic Named Entity Recognition: What Models Get Wrong Without Native Annotators

The Nubian Name Problem: Why Arabic NER Fails on Sudanese Personal Names

Named entity recognition for Arabic text is an active research area, but the training corpora that underpin almost every production Arabic NER system — ANERcorp, ACE 2004/2005, OntoNotes Arabic, and the datasets from NIST and LDC — draw primarily from Egyptian, Gulf, and Levantine newswire sources. The PERSON entity dictionaries embedded in these systems contain Arabic names drawn from Egyptian, Saudi, and Lebanese naming conventions: names with Arabic roots, Arabic prefixes, and Arabic phonological patterns.

Sudan’s population has a fundamentally different naming landscape. An estimated 30–40% of Sudanese individuals carry given names from Nubian language families — Nobiin (spoken along the Nile in northern Sudan and southern Egypt), Kenzi, Beja (Cushitic family, Red Sea coast), and the diverse Nuba Mountains language cluster (Nuba, Dilling, Heiban, and related varieties). These names are written in Arabic script in Sudanese digital text, but they follow Nubian phonological patterns and are simply absent from Arabic NER training vocabularies.

When a standard Arabic NER model encounters a Nubian-origin name like ‘أوريل’ (Oriel, Nobiin origin), ‘تامبول’ (Tamboul, Nuba Mountains), or ‘بشير’ in a Beja-contact Eastern Sudanese context, it typically classifies the token as common noun or unknown — not as a PERSON entity. Arabic NLP research on under-resourced dialect varieties with Sub-Saharan lexical influence shows 33–46% F1 degradation compared to matched MSA entity recognition (Abdul-Mageed et al., 2020; PADIC corpus analysis, Meftouh et al., 2015).

Five Entity Categories Where Sudanese Arabic NER Breaks

1. Nubian-family personal names

The coverage gap for Nubian-origin names is the single largest source of PERSON entity F1 degradation in Sudanese Arabic NER. Names from Nobiin, Beja, and Nuba Mountains languages are systematically absent from Arabic NER training data and cannot be resolved by Arabic-only name dictionaries. The practical consequences are significant: a KYC document processing system, a customer identity extraction pipeline, or a healthcare record NER system that misses 30–40% of Sudanese person names is not usable in production for Sudanese Arabic text.

The solution is a manually compiled Nubian name gazetteer — a structured list of Nubian-origin given names in their Arabic script variants, compiled and reviewed by native Sudanese annotators from the relevant language communities. A gazetteer covering the 2,000 most common Nobiin, Beja, and Nuba Mountains given names substantially reduces the F1 gap for PERSON entities and can be integrated as a dictionary-lookup component alongside a contextual NER model.

2. Nile Valley place names with Nubian etymology

Sudanese place names — particularly in northern Sudan, the Nile corridor, and the Nuba Mountains — frequently have Nubian or Beja etymology, written in Arabic script following local phonological conventions. Town names, village names, and geographic features along the Nile in Sudanese territory (Karima, Dongola, Kerma, Wadi Halfa and its surroundings) have Nubian roots that MSA-trained LOCATION entity models either misclassify or fail to detect as place names altogether.

Geographic entity recognition is critical for logistics AI, delivery routing systems, public health surveillance, and financial services compliance. A system that cannot correctly identify Sudanese place names cannot support these functions for the Sudanese market.

3. Tribal and regional organisation entities

Sudanese organisational entities reflect a governance and social structure that includes tribal councils, Nile Valley regional development bodies, community-level cooperatives, and humanitarian organisation abbreviations that evolved during conflict and displacement periods. These organisational names differ structurally from Gulf or Egyptian corporate naming conventions — they are not prefixed with ‘شركة’ (company) or ‘مجموعة’ (group) in the patterns MSA NER models are trained to recognise.

A Sudanese tribal council or regional governance body named ‘مجلس النازحين’ (displaced persons council) or ‘تجمع الشمالية’ (Northern gathering) is an ORGANISATION entity — but MSA models trained on Gulf corporate text classify these as common noun phrases, not as organisational entities.

4. Conflict-era location entities with ambiguous reference

Sudanese Arabic text produced in or about conflict-affected areas contains location entity references — displaced-persons camp names, humanitarian corridor names, conflict-zone designations — that are not in any standard Arabic NER training gazetteer and whose boundaries may shift over time. An Arabic NER model that cannot identify ‘مخيم الزعتري’ (Zaatari camp) will fail on Sudanese text that references camp names with entirely different Sudanese geographic context.

Native Sudanese annotators with knowledge of the regional geography and conflict context can correctly label these entities; non-native annotators and MSA models systematically fail on them, producing NER outputs that are unreliable for conflict monitoring, humanitarian AI, or public health surveillance in Sudan.

5. Cross-language entity spans at code-switching boundaries

Sudanese Arabic text — particularly from educated urban Khartoum speakers — includes code-switching with English for business and technical terminology, and occasional code-switching with local Sudanese languages for community-specific references. Cross-language entity spans that straddle Arabic and English tokens (a company name partially in English, a technical product name) present the same segmentation challenges as in other Arabic varieties, but Sudanese Arabic adds an additional layer: code-switching with Sudanese African languages for community-specific named entities that no Arabic or English NER model can handle.

Case Study: Khartoum Fintech — 46% to 83% NER F1 on Sudanese Customer Text

A Khartoum-based financial services company needed a document processing system to extract named entities from Sudanese Arabic customer applications, identity documents, and correspondence — primarily person names, addresses, employer names, and reference contacts. Their existing system used an AraBERT-based NER model fine-tuned on ANERcorp and ACE 2004/2005 Arabic training data.

Before: The model achieved 46.2% overall entity F1 on held-out Sudanese customer document text. PERSON entity F1 stood at 38.4% — meaning the system missed 61.6% of person names in Sudanese Arabic customer documents, with Nubian-origin names achieving near-zero recall. LOCATION entity F1 was 51.3%, with northern Sudan and Nuba Mountains place names systematically undetected. ORGANISATION entity F1 was 44.7%, with tribal governance entities and regional employer names classified as common noun phrases. Straight-through processing rate for identity verification was 14.3%, requiring 85.7% of applications to pass through manual review.

The annotation project delivered 21,000 labelled entity spans across PERSON, LOCATION, ORGANISATION, DATE, and FINANCIAL INSTRUMENT classes, along with a Nubian name gazetteer covering 2,400 Nobiin, Beja, and Nuba Mountains given names in their Arabic script variants. Annotation was conducted by seven native Khartoum Arabic-speaking annotators with financial services domain knowledge, supervised by a native Sudanese Arabic linguist. Final IAA kappa across entity classes was 0.86 overall, with 0.81 on the Nubian-origin PERSON subset.

After fine-tuning the model with the annotated data and Nubian gazetteer integration: Overall entity F1 improved from 46.2% to 83.4%. PERSON entity F1 improved from 38.4% to 81.7%, with Nubian-origin name recall rising from near-zero to 76.2%. LOCATION F1 improved from 51.3% to 84.9%. ORGANISATION F1 improved from 44.7% to 82.1%. Straight-through processing rate for identity verification rose from 14.3% to 63.8% — a 4.5x increase.

The reduced manual review load generated AUD $1.34 million in annual operational savings. Total annotation project cost was AUD $38,700, including gazetteer compilation. The company achieved return on annotation investment within six weeks of deployment.

The Annotation Protocol for Sudanese Arabic NER Projects

Effective Sudanese Arabic NER annotation requires a protocol tailored to the specific challenges of Nubian name coverage, tribal organisation entities, and conflict-context locations. The essential elements are:

Nubian name gazetteer compilation before annotation begins. Attempting to label Nubian-origin names without a reference gazetteer produces inconsistent PERSON entity labels — annotators disagree on which tokens are names versus common nouns because the visual form of Nubian names in Arabic script does not always signal ‘name’ the way Arabic-root names do. A gazetteer compiled by native Sudanese annotators from the relevant language communities, covering at minimum the 1,500–2,000 most common Nobiin, Beja, and Nuba Mountains given names, establishes a consistent reference before annotation begins.

Entity schema extension for conflict-era and tribal entities. Standard PERSON/LOCATION/ORGANISATION/DATE/MISC schemas are insufficient for Sudanese text. Adding sub-categories for tribal-council organisations, conflict-era locations (camp names, corridor names, humanitarian zone designations), and diaspora-community references substantially improves the usefulness of the labelled data for downstream applications.

Khartoum-register annotators as the primary pool. The dominant commercial and digital register is Khartoum Arabic. Annotators who are native Khartoum speakers and familiar with Sudanese commercial, financial, and social naming conventions produce the most consistent entity labels for the entity types most relevant to commercial AI applications. For Juba Arabic or Eastern Sudanese text, sub-dialect-matched annotators are required.

Multi-annotator adjudication for ambiguous entity spans. Cross-language spans, conflict-era location references, and Nubian-origin names at the gazetteer tail (low-frequency names not in the reference list) should go to a second native Sudanese annotator, not to a supervisor from a different Arabic variety. The most common source of systematic NER error in Sudanese annotation is adjudication by non-Sudanese Arabic speakers who lack the geographic and cultural knowledge to resolve ambiguous spans correctly.

AI Taggers’ Arabic NLP annotation service covers Sudanese Arabic NER with native Khartoum annotators, Nubian name gazetteer support, and entity schema design for financial services, healthcare, humanitarian, and commercial AI applications.

Juba Arabic NER: A Distinct Problem

Juba Arabic — the Arabic creole spoken in South Sudan’s capital and southern border regions — is sufficiently different from Khartoum Arabic to constitute a separate NER challenge. Juba Arabic has heavy phonological and lexical influence from Nuer, Dinka, Bari, and Zande languages, and personal names from these South Sudanese language communities are even further removed from Arabic NER training data than Nubian names.

PERSON entity F1 for standard Arabic NER models on Juba Arabic text is typically 20–35% — significantly lower than for Khartoum Arabic — because a larger proportion of personal names come from entirely non-Semitic language families (Nilo-Saharan Dinka names, Central Sudanic Nuer names, Bari names) that share no phonological or morphological patterns with Arabic.

For AI teams building systems for both Sudan and South Sudan, Juba Arabic requires a separate annotation pipeline with annotators drawn from the South Sudanese language communities, a separate South Sudanese name gazetteer, and entity schemas that reflect the organisational and geographic entities of South Sudan rather than Sudan proper.

Related Reading

• Sudanese Arabic Sentiment Analysis: What Models Get Wrong
• Where Do Arabic NLP Datasets Come From — and How Do You Build Your Own?
• Moroccan Darija Annotation: The Hardest Arabic Variant to Get Right
• Arabic NLP Annotation Service
• Arabic Data Labeling Service

Frequently Asked Questions