Urdu NLP Data Annotation: What Makes It Hard and How to Get It Right

Why Urdu Breaks Both Arabic and Hindi Annotation Pipelines

Urdu occupies a unique position in the NLP landscape: it shares its script tradition with Arabic and Persian but its grammar with Hindi, creating a language that breaks workflows designed for either family. Teams that try to annotate Urdu using Arabic NLP infrastructure fail on the grammar and vocabulary side; teams that use Hindi annotation infrastructure fail on the script side. The only reliable approach is a dedicated Urdu annotation pipeline built for Urdu's specific properties.

Pakistan's AI sector context matters for understanding annotation demand. The NCAI (National Centre of Artificial Intelligence), established under the National AI Policy 2018, has directed investment toward Urdu language AI as a national priority. Pakistan's tech sector — including HBL's Konnect digital banking, Jazz's JazzCash, and Telenor's Easypaisa — is building conversational AI products in Urdu for a market of 70 million native speakers. These products require annotation capability that Pakistan's research institutions have partially built but that commercial annotation at production scale still imports through specialist vendors.

Urdu is spoken by approximately 70 million native speakers and 170 million second-language speakers globally (Ethnologue, 2024). The diaspora dimension adds UK (500K Urdu speakers), Canada (200K), UAE (1.5M+ Pakistani workers), and Saudi Arabia (2M+ Pakistani workers) — all markets where diaspora-serving fintech and government services require Urdu NLP capability that existing Arabic or English annotation pipelines cannot serve.

The Four Structural Challenges of Urdu NLP Annotation

1. Nastaliq script rendering and annotation fatigue

Urdu is written in the Nastaliq style of the Perso-Arabic script — a calligraphic tradition where characters are arranged in a diagonal, flowing baseline with extensive ligatures and contextual glyph variations. Nastaliq is visually distinct from Arabic Naskh: characters in Nastaliq sit on a descending diagonal rather than a horizontal baseline, and the overall text flow is significantly denser and more visually complex than equivalent Naskh text.

For annotation platforms, this creates a rendering requirement that most Arabic-configured tools do not meet. When Urdu text is displayed in an Arabic Naskh font (as happens when an Arabic annotation platform is used without Urdu-specific configuration), native Urdu readers can technically decode the text but find it significantly harder to read — comparable to reading English in an unfamiliar sans-serif at very small point size. Annotation fatigue accelerates, error rates on character-dense passages increase by 30–50%, and annotators report boundary uncertainty on vowel-marked text that they would resolve instantly in native Nastaliq rendering.

A 2022 study by Hussain et al. at the Center for Language Engineering (CLE), Lahore — the leading Urdu NLP research group — measured annotation speed and error rates for native Urdu speakers working in Nastaliq versus Naskh rendering on an NER task. Native Nastaliq rendering produced 23% fewer span boundary errors and 41% faster annotation throughput on passages containing more than 15 characters per line. The rendering environment is not a cosmetic choice for Urdu annotation — it directly affects data quality.

2. Extensive Perso-Arabic vocabulary and NER complexity

Urdu's formal and literary vocabulary is heavily derived from Persian and Arabic — far more so than Hindi, which draws its formal vocabulary from Sanskrit. A formal Urdu text can contain 60–70% Persian and Arabic loanwords in the lexicon, with Indo-Aryan roots (shared with Hindi) dominant only in everyday conversational vocabulary and grammatical function words.

For NER annotation, this creates a distinctive challenge: many high-frequency Urdu words are also Arabic or Persian proper nouns. The word فرید (Fareed) is both a common Arabic-derived Urdu adjective meaning "unique" and a very common Muslim first name. The word کریم (Karim) means both "generous" (an Arabic adjective used in Urdu) and is a common given name and part of Allah's 99 names. Disambiguating these as named entities versus common vocabulary requires deep contextual reading that non-native annotators cannot perform reliably.

Published Urdu NER benchmarks from the CLE Urdu Corpus (Hussain, 2010; updated with modern benchmarks by Nawaz et al., 2019) show that native-speaker annotators achieve NER F1 of 0.84 on standard person/organisation/location categories versus 0.63 for annotators with intermediate Urdu proficiency — a gap of 21 percentage points driven primarily by person-name vs adjective disambiguation on Arabic-derived vocabulary.

3. Urdlish: two-script code-switching

Urdu digital communication in Pakistan and in the Pakistani diaspora involves a phenomenon unique in annotated language corpora: two-script code-switching within a single text. "Urdlish" (also written Urdinglish) is the informal mixing of Urdu and English that is standard in Pakistani urban digital communication — but where the same text may contain Urdu words in Nastaliq script interspersed with English words in Roman script, within a single sentence.

A typical WhatsApp or social-media message in Pakistani Urdlish might read: اج meeting cancel ہوگئی، کل آجانا (Aaj meeting cancel ho gayi, kal aajana — "Today the meeting was cancelled, come tomorrow") — mixing Urdu Nastaliq characters with the English Roman-script word "meeting." For annotation tools, this creates a mixed-script token stream where RTL and LTR text segments alternate within a sentence. Most annotation platforms handle this poorly — they either force the entire sentence to RTL (making the English word appear mirrored) or break the text flow at the script boundary.

For intent and sentiment annotation, Urdlish requires annotators to understand both Urdu grammar (which governs sentence structure) and English vocabulary (which provides the semantic content in code-switched segments). An annotator who is strong in Urdu but not English, or vice versa, cannot annotate Urdlish corpora reliably. Pakistani Urdu annotators who are educated in English-medium schools — the standard for Pakistan's urban professional class — naturally handle Urdlish; annotators sourced from Urdu-medium education backgrounds or from non-Pakistani Urdu-speaking communities often cannot.

4. Pakistan/India vocabulary and register divide

Urdu is the national language of Pakistan and is spoken by approximately 50 million native speakers in India (predominantly in Uttar Pradesh, Bihar, and the diaspora from those states). Pakistani and Indian Urdu differ in formal vocabulary, loanword preferences, and register conventions in ways that affect annotation quality for domain-specific tasks.

For most NER and sentiment annotation tasks, Pakistani and Indian native Urdu speakers are interchangeable. The differences become significant for: (1) formal administrative vocabulary (Pakistani government Urdu uses different Persian-origin terms for some roles and processes than Indian Urdu); (2) colloquial expressions that are regional to Pakistan versus India; and (3) the Urdlish code-switching pattern, which is characteristic of Pakistani digital communication and less common in Indian Urdu digital text. For annotation projects targeting Pakistani products (e-government, fintech, telco chatbots), Pakistani native speakers are the higher-quality annotator source.

The Urdu NLP Tooling Ecosystem

Urdu NLP has a growing open-source tooling ecosystem, centred on the Center for Language Engineering (CLE) in Lahore and international research groups focused on South Asian languages. The critical tools for annotation pipelines are:

Case Study: Pakistani Telecom — Customer Service Chatbot Intent Recovery

In early 2026, a major Pakistani telecom operator (similar scale to Jazz, Telenor Pakistan, or Zong) needed 50,000 annotated customer service messages for an intent classification model to power their Urdu-language chatbot. Messages were written in a mix of Nastaliq Urdu and Urdlish (Roman-script code-switching), reflecting the realistic distribution of their digital customer base.

The initial annotation run used a multilingual crowdsourcing platform. After 12,000 records, an internal Urdu NLP engineer reviewed a validation sample and found:

• Intent accuracy of 66.8% on a 500-record gold standard evaluated by native Pakistani Urdu speakers — against a 85% target
• 34% of Urdlish messages were partially or fully misclassified — the crowdsource platform's annotators could not read the mixed-script text and defaulted to "other" or guessed from context
• Nastaliq rendering had not been configured in the annotation interface — annotators received Naskh Arabic rendering and were working at reduced reading speed
• Unicode normalisation had not been applied, causing 9.3% of records to have split-vocabulary tokenisation errors from Arabic/Urdu character variant mixing

The team rebuilt the annotation pipeline with native Pakistani Urdu-speaking annotators and proper Urdu pre-processing:

• urduhack normalisation on all 50,000 messages to resolve Arabic/Urdu Unicode character variants
• Label Studio configured with Jameel Noori Nastaleeq font for Urdu text rendering and proper RTL display
• Native Pakistani Urdu annotators (Lahore and Karachi-based, educated in English-medium schools for Urdlish competence), with a calibration set of 100 Urdlish messages designed to test intent classification of mixed-script constructions
• Separate annotation track for Urdlish messages (25% of the corpus) with enhanced per-record pay reflecting the dual-script reading requirement
• Double annotation on 15% of records with kappa measurement per intent class

Results on the re-annotated corpus:

The native-speaker annotation cost was 2.7× higher per record. The 12,000 crowd-annotated records — particularly the Urdlish subset — were structurally unusable and were discarded. The sunk cost on the discarded records exceeded the premium for the entire 38,000-record re-annotation. The downstream intent model, trained on native-speaker data, achieved 86.2% accuracy on production holdout, enabling the telecom to route 71% of customer service queries automatically at a false-routing rate of 3.8%.

Urdu Annotation Guidelines: What Generic Templates Miss

Urdu annotation guidelines adapted from Arabic or Hindi templates routinely omit the language-specific instructions that prevent the most common errors. Critical inclusions are:

• Urdlish policy: Define explicitly how to handle mixed-script sentences. Specify whether English tokens in Nastaliq-Urdu text should be annotated with their semantic label (recommended for intent/sentiment) or treated as a separate "other" class. For Urdlish-dominant corpora, the policy is to annotate by semantic content regardless of script — so the English word "cancel" in a Urdu sentence receives the same cancellation-intent label as its Urdu equivalent منسوخ.
• Arabic loanword NER disambiguation: Provide a reference list of the 40–60 most common Arabic-derived Urdu words that are also Muslim given names, with context rules for disambiguation. The context rules must be illustrated with positive examples (the word used as a name) and negative examples (the same word as an adjective). This is the highest-value single addition to any Urdu NER annotation guideline.
• Nastaliq rendering confirmation: Require annotators to verify that their annotation interface is rendering text in Nastaliq before beginning — either through a screenshot check or a calibration task designed to fail visibly in Naskh rendering. This prevents annotators from working at reduced speed without reporting the rendering issue.
• Nun ghunna (ں) handling: Specify that the Urdu-specific character ں (nun ghunna, U+06BA) must be preserved in annotation output and not mapped to ن (noon, U+0646). These characters are visually similar in some fonts but phonologically distinct in Urdu — this matters for ASR and speech synthesis annotation tasks.
• Izafat construction: Like Persian, Urdu uses the izafat (Urdu: اضافت, equivalent to Persian ezafe) to link nouns to modifiers via a vowel -e/-ye that is usually unwritten. Annotation guidelines for parsing and coreference tasks must include izafat chain handling rules with illustrated examples.

The Urdu AI Market and Annotation Demand

Pakistan's NCAI has identified Urdu NLP as a national priority, funding research at NUST (National University of Sciences and Technology), LUMS (Lahore University of Management Sciences), and IBA Karachi. This research investment is creating domestic annotation capability but commercial annotation at production scale — particularly for Urdlish and domain-specific applications — still relies heavily on specialist vendors.

The international annotation demand for Urdu comes from Pakistan-headquartered fintech (HBL, UBL Digital, JazzCash), international humanitarian organisations operating in Pakistan (UN, World Bank, NGO digital services), and global AI product teams adding Urdu to multilingual rosters. The Pakistani diaspora context adds UAE and Saudi Arabia — both markets with 1–2 million Pakistani workers who use Urdu-language digital banking and government services.

Our multilingual annotation services include native Pakistani and Indian Urdu-speaking annotators for NER, sentiment, intent, and text classification tasks, with Nastaliq rendering configuration, Unicode normalisation, and Urdlish annotation protocols included as standard. For teams building Arabic and Urdu capabilities simultaneously — a common requirement for products targeting the Pakistani diaspora in Gulf states — our Arabic data labeling services and Urdu annotation share the same RTL QA infrastructure and can be coordinated in a single managed workflow. For broad multilingual coverage, our native-speaker annotation network covers Urdu alongside 120+ languages with consistent quality standards.

Related Reading

If you are building Urdu NLP annotation alongside other language requirements, these posts cover annotation challenges and strategies for related contexts:

• Persian (Farsi) NLP Data Annotation: What Makes It Hard and How to Get It Right — Closely related Perso-Arabic script challenges for teams working across both languages
• Where Do Arabic NLP Datasets Come From — and How Do You Build Your Own? — Parallel challenges of RTL dataset sourcing for teams working across Arabic, Urdu, and Persian
• How Does Multilingual Annotation and Localization Work for Global AI? — Multi-language annotation workflows when Urdu is one of several target languages

Frequently Asked Questions