Arabic RLHF: Building Preference Data That Aligns Models to Gulf Users

Why Arabic RLHF Is Not Just Translated English RLHF

Reinforcement learning from human feedback works by training a reward model on human preference judgements, then using that reward signal to align the base LLM. The quality of the alignment is bounded entirely by the quality of those human preference judgements. In English, this is relatively tractable: native English speakers from diverse backgrounds tend to agree on what a helpful, harmless, and honest response looks like for a given domain.

Arabic breaks that assumption immediately. Arabic is not one language for alignment purposes — it is five major dialect clusters (Khaleeji, Egyptian, Levantine, Maghrebi, Iraqi) plus Modern Standard Arabic (MSA), each with distinct norms for politeness, directness, formality, and what counts as a culturally appropriate response. A Khaleeji user asking a chatbot for advice on a business contract expects a response register that is significantly more formal and relationship-oriented than an Egyptian user asking the same question. An MSA-trained annotator judging preference for a Khaleeji customer service bot will apply the wrong reward signal — not because they are wrong, but because they are calibrating on a different cultural model.

According to research from MBZUAI (Mohammed bin Zayed University of AI), Arabic preference datasets built using MSA or Egyptian annotators showed less than 30% agreement with Gulf/Khaleeji native speakers on whether a model response was culturally appropriate (MBZUAI Multilingual Alignment Study, 2024). This means the reward model trained on that data is, by definition, optimising for the wrong target for Gulf products.

The Dialect Problem in Arabic Preference Annotation

Gulf Arabic (Khaleeji) is the dialect spoken in Saudi Arabia, the UAE, Kuwait, Qatar, Bahrain, and Oman. It differs from Egyptian Arabic — by far the most common dialect in annotation pools — not just in vocabulary but in pragmatics: the way requests are framed, how disagreement is expressed, how much context is assumed, and what formality register is appropriate for which relationship type.

RLHF preference judgements are fundamentally pragmatic assessments. "Is this response helpful?" means "does it match what a fluent speaker of this dialect would find natural and appropriate in this context?" An annotator who does not speak Gulf Arabic natively cannot reliably answer that question for Gulf-facing products — no matter how linguistically skilled they are in MSA or Egyptian.

The Open Arabic LLM Leaderboard (OALL) 2025 evaluation found that frontier models trained on translated English RLHF data score 18–22 percentage points lower on Khaleeji cultural alignment tasks than on equivalent English tasks. This is not a small gap — it means models that perform well in English and reasonably in MSA are substantially misaligned for the 100 million+ Arabic speakers in the Gulf region (OALL Technical Report, 2025).

The practical implication: if you are building a customer-facing product for Saudi Arabia, the UAE, or the broader GCC, your RLHF annotator pool must be majority Khaleeji native speakers. Egyptian or Levantine annotators can supplement but should not dominate the preference judgement task.

What Culturally Aligned Arabic Preference Data Looks Like

High-quality Arabic RLHF preference data for Gulf applications has several characteristics that distinguish it from generic RLHF data:

• Dialect stratification: Tasks are routed to annotators by dialect match. Khaleeji prompts go to Khaleeji native-speaker annotators; MSA formal prompts can go to a broader pool; code-switching prompts (Arabic-English mix) require bilingual Khaleeji speakers.
• Domain-specific cultural guidelines: Gulf business norms, Islamic etiquette considerations, gender-address conventions, and formality gradients are documented explicitly in annotation guidelines — not left to annotator intuition.
• Chosen-rejected pair format: For DPO training, each prompt is paired with two model responses and annotators select the preferred one. For PPO reward modelling, annotators assign scalar quality scores across multiple dimensions (helpfulness, safety, cultural fit, factual accuracy).
• Code-switching coverage: Gulf business text routinely mixes Arabic and English within sentences. Preference data must include a representative proportion of mixed-language prompts with preference judgements from bilingual Khaleeji annotators.
• PDPL-compliant handling: For KSA projects, any prompts sourced from real user conversations must meet Saudi Personal Data Protection Law requirements for consent, anonymisation, and data residency.

The Arabic NLP annotation workflows that work for RLHF differ from those used for NER or sentiment labelling: they require higher annotator seniority, more complex calibration sessions, and adjudication protocols when Khaleeji annotators disagree on preference rankings.

Case Study: Riyadh Fintech LLM — Before and After Gulf-Aligned RLHF

A Riyadh-based digital bank launched a customer service LLM in early 2025 using GPT-4 fine-tuned with 28,000 translated English RLHF preference pairs. Initial deployment metrics were poor: user satisfaction scores averaged 3.1/5, with qualitative feedback consistently citing responses as "too formal", "sounds like a news article", and "doesn't understand what I actually want". Intent completion rate — the proportion of customer queries resolved without escalation to a human agent — sat at 41%.

The bank engaged AI Taggers to rebuild the RLHF dataset with dialect-native annotation. The replacement dataset comprised 22,000 preference pairs: 60% Khaleeji/Saudi Arabic prompts annotated by native Najdi and Hijazi speakers, 25% Gulf business code-switching (Arabic-English), and 15% MSA formal prompts for regulatory communication contexts. Annotators completed a two-day calibration session covering Saudi banking formality norms, Islamic finance terminology conventions, and the bank's brand voice guidelines. PDPL compliance documentation was produced for all data handling stages.

After fine-tuning the same base model on the Gulf-aligned RLHF dataset, results over a 90-day post-deployment window showed:

• Customer satisfaction score: 3.1 → 4.4 / 5 (+42%)
• Intent completion rate: 41% → 74% (+80%)
• Escalation to human agent: 59% → 26% of interactions
• Complaint about "unnatural language" in user feedback: 34% → 4% of responses

The base model, infrastructure, and deployment pipeline were identical. The only change was the dialect-native preference dataset. This is the clearest illustration of why Arabic RLHF data quality is not a minor optimisation — it is the primary driver of Gulf product alignment.

How Many Preference Pairs Does Arabic RLHF Need?

Volume requirements for Arabic RLHF vary significantly by model size, domain, and target dialect mix. As a practical guide:

• Customer-facing Gulf chatbot (DPO): 15,000–50,000 preference pairs, dialect-stratified. Minimum 40% Khaleeji, 25% MSA formal, 20% Egyptian (for pan-MENA products), 15% code-switching.
• Domain-specific expert model (legal, medical, government): 5,000–15,000 high-quality expert-judged pairs. Volume matters less than annotator credentialing — for Islamic finance or Saudi healthcare contexts, expert annotators with domain knowledge are non-negotiable.
• Research / benchmark evaluation: 1,000–3,000 carefully curated pairs covering all major Gulf dialect sub-variants (Najdi, Hijazi, Emirati, Kuwaiti) and domain topics.

For comparison, leading English RLHF datasets (Anthropic HH-RLHF, OpenAI InstructGPT preference data) contain 160,000–340,000 pairs. Arabic has a data scarcity problem compounded by dialect fragmentation — which is why quality per pair matters more than raw volume, and why native-speaker annotation is not optional.

This also intersects with general RLHF data collection principles — prompt diversity, annotator calibration, and reward model validation — which apply to Arabic RLHF with additional dialect-specific constraints layered on top.

PDPL, Saudi Vision 2030, and the Regulatory Dimension of Arabic RLHF

Saudi Arabia's push to build sovereign Arabic AI under Vision 2030 is creating significant demand for large-scale Arabic RLHF data — from SDAIA's national AI strategy, Aramco's internal AI programmes, and PIF-backed ventures developing Arabic foundation models. This demand is inseparable from the PDPL compliance requirement.

PDPL (the Saudi Personal Data Protection Law) restricts how personal data — including conversational data used for RLHF — is collected, processed, and transferred. Key implications for Arabic RLHF projects:

• User conversation data: If prompts are sourced from real user interactions, explicit consent is required under PDPL Article 5. Anonymisation must meet SDAIA standards, not just pseudonymisation.
• Cross-border transfer: RLHF annotation performed outside Saudi Arabia on KSA personal data requires a PDPL-compliant data transfer agreement with the processing entity.
• Audit trail: PDPL compliance requires documentation of what data was used, who processed it, and under what consent basis — exactly the kind of provenance trail that annotation workflows must produce.

Teams building Arabic LLMs for the Saudi market should review what Vision 2030 means for Arabic AI data alongside the specific PDPL requirements for RLHF data handling. The regulatory and commercial pressures are now inseparable for Saudi AI development.

Evaluating Arabic RLHF Quality: Beyond Perplexity

Standard RLHF evaluation metrics (reward model accuracy, perplexity reduction, win rate on held-out pairs) are necessary but not sufficient for Gulf Arabic alignment. They measure consistency with the training preference judgements — but if those judgements were made by non-native Khaleeji annotators, consistency with them is not alignment with Gulf users.

Effective evaluation of Arabic RLHF quality requires:

• Native speaker preference evaluation: Hold-out test set evaluated by Khaleeji native speakers who did not annotate the training data. Win rate against the pre-RLHF baseline and against models fine-tuned on translated RLHF data.
• Cultural alignment benchmarks: The ACVA (Arabic Cultural Values Alignment) subset of the OALL benchmark tests model responses against Gulf cultural norms explicitly — including scenarios around formality, religiosity, and business etiquette.
• Domain-specific red-teaming: For banking, healthcare, and government products, adversarial testing by native speakers in the target domain to identify culturally inappropriate responses the reward model missed.

For a full picture of how Arabic models are evaluated, the Arabic LLM evaluation benchmarks guide covers ArabicMMLU, AlGhafa, and how to build custom evaluation that surfaces real product weaknesses rather than benchmark artefacts.

If you are building or evaluating Arabic RLHF data at scale, our Arabic NLP annotation service provides dialect-routed annotator pools, PDPL-compliant data workflows, and QA protocols designed specifically for preference and alignment datasets.

Frequently Asked Questions