Choosing an Arabic Data Annotation Company: 10 Checks Before You Sign

Why Generic Annotation Vendors Fail at Arabic

Arabic is spoken by over 400 million people across 22 countries (Ethnologue, 2024), but it is not a single language. It operates as a diglossic system: Modern Standard Arabic (MSA) is used in formal writing, news, and government documents, while regional spoken dialects — Khaleeji, Egyptian, Levantine, Moroccan Darija — differ substantially in vocabulary, grammar, and pragmatics. A native speaker of Egyptian Arabic does not reliably label Emirati colloquial chat transcripts at production-grade accuracy.

Generic annotation platforms address this by recruiting bilingual annotators — people with strong English and formal Arabic literacy. For classification tasks on MSA news text, this is serviceable. For conversational AI, sentiment analysis on social media, or any task requiring dialect judgement, the gap becomes significant.

Published NLP benchmarking research has consistently found that named entity recognition and intent classification models trained on MSA-annotated data show substantially lower F1 scores on Gulf dialect test sets compared to models trained on dialect-native annotated data. The gap is not a modelling artefact — it is a labelling provenance problem.

Our Arabic data labelling service works from a structured annotator pool where each annotator carries a verified dialect tag, has passed a task-specific qualification test, and is calibrated against gold sets for ongoing quality measurement. The 10 checks below are designed to surface whether any vendor you evaluate has that same infrastructure in place.

The 10 Checks Before You Sign

These checks apply whether you are buying NLP annotation, speech transcription, sentiment labelling, or multimodal Arabic annotation. Request written responses to each — verbal assurances rarely survive the handover to the delivery team.

Case Study: UAE Digital Services Platform

A government-aligned digital services platform in the UAE needed to annotate 50,000 citizen-service chat transcripts in a mix of MSA and Emirati dialect for a virtual assistant rollout. Their first vendor — a generalist platform using bilingual but non-native annotators — returned data with an inter-annotator agreement of 68% on intent labels and a 22% error rate on dialect-specific named entities. Neither figure was flagged by the vendor. The problems only surfaced when the downstream model underperformed significantly on held-out UAE test data.

After switching to a native-speaker annotation pipeline with per-dialect routing and a double-blind adjudication layer, IAA improved to 91% on intent classification and 88% on NER. The dialect-specific entity error rate dropped from 22% to 4%. The downstream virtual assistant achieved an 18-point improvement in intent accuracy on held-out Emirati citizen queries.

The post-mortem finding was simple: the original vendor had no Emirati-native annotators at all. The work had been routed to MSA-trained annotators who consistently miscategorised Emirati-specific lexical patterns. None of the 10 checks above had been applied at procurement.

PDPL, Data Residency, and What Most Vendors Miss

Saudi Arabia's Personal Data Protection Law (PDPL) came into full effect in September 2023 and is enforced by SDAIA (the Saudi Data and Artificial Intelligence Authority). The law applies to any organisation — domestic or foreign — that processes personal data about Saudi residents. For annotation vendors, this creates specific obligations that many offshore platforms have not yet structured their Middle East operations to address:

• A data processing agreement (DPA) must be signed before any personal data is transferred to the vendor
• Cross-border data transfers require either explicit data-subject consent or a SDAIA-approved transfer mechanism — or annotation work must be performed within the Kingdom
• Data must be deleted or irreversibly anonymised at the end of the engagement, with documented evidence of deletion
• Breach notification to SDAIA is required within 72 hours of discovery

Parallel requirements apply for UAE data: the Federal UAE Personal Data Protection Law (enacted 2021) applies nationally, with the Dubai International Financial Centre Data Protection Law (2020) and Abu Dhabi Global Market Data Protection Regulations applying in their respective free zones.

Many global annotation platforms handle GDPR compliance competently but have not structured their operations for PDPL. When evaluating vendors, ask specifically: where is annotation work physically performed? Do they have signed DPAs with GCC clients on record? What is their deletion process at project end? For healthcare, financial, or government data, request evidence of compliance process — not just a security checkbox on a vendor portal.

What a Useful Pilot Includes

A pilot is your quality audit at manageable scale. For Arabic annotation, a useful pilot delivers more than labelled data — it reveals the quality infrastructure. A well-structured Arabic annotation pilot should include:

For audio tasks, also request: timestamp accuracy on a 30-second sample clip, a speaker-diarisation example for a two-speaker conversation, and the transcription output in both buckwalter and Unicode Arabic formats if your pipeline requires both. See our guide to end-to-end Arabic data labelling pipelines for how production annotation workflows are structured from task brief to delivery.

Cost Drivers and Realistic Turnaround

Arabic annotation pricing varies by task complexity, dialect requirement, and quality tier. The major cost drivers are dialect routing (Khaleeji and Darija specialists command a premium over MSA annotators), adjudication layers (adds approximately 25–35% to base cost), domain expertise (clinical and legal Arabic requires subject-matter-qualified annotators), and volume (per-unit rates drop substantially above 50,000 records). For specific pricing, see our annotation pricing page.

Turnaround for a 10,000-record Arabic text annotation project is typically 5–7 business days at standard quality. Dialect-specific work requiring adjudication and gold-set calibration takes 8–12 days. Audio transcription at 10,000 minutes of Gulf-dialect speech typically takes 10–15 days with native-speaker QA.

For further reading on Arabic annotation tooling and software evaluation, see our guide to Arabic text annotation software. For Gulf-dialect sentiment specifically, see our deep dive on Khaleeji Arabic sentiment annotation. If you are building an LLM on Arabic data, our Arabic text annotation service covers SFT and RLHF data preparation workflows.

Frequently Asked Questions