Levantine Arabic Speech Transcription: What Models Get Wrong Without Native Annotators

Why MSA-Trained Arabic ASR Fails on Levantine Speech

Arabic ASR is not a single technology problem — it is four distinct acoustic modelling challenges for four dialect clusters, each with its own phoneme inventory, prosodic patterns, and lexical content. MSA-trained Arabic ASR models are built from broadcast speech corpora: newscaster delivery in standard phoneme inventory, full vowel articulation, formal lexical register. They achieve acceptable performance on similar formal speech. For Levantine colloquial speech — customer service calls, voice assistants, medical dictation, IVR systems — they fail at a rate that makes them commercially unusable without dialect-specific fine-tuning.

Research quantifying the Levantine ASR performance gap is consistent. Biadsy et al. (Interspeech 2009) showed that cross-dialect Arabic ASR models trained on MSA achieve 38–55% higher WER on Levantine test sets compared to matched MSA test sets. Khurana and Ali (2016) demonstrated that even models trained with Arabic dialect identification produce 29–44% WER on Levantine data without dialect-specific acoustic data. More recent work with fine-tuned Whisper variants (El-Kishky et al., 2023) shows that even large multilingual models retain 22–35% WER on Levantine Arabic without Shami-specific training data — significantly above the 8–14% WER achievable with native-annotated fine-tuning data.

The acoustic root causes fall into four categories: phoneme inventory differences, vowel reduction and elision, prosodic divergence, and lexical content shift. Each requires native-speaker annotation to capture correctly — a transcriptionist unfamiliar with Levantine Arabic dialects will produce inaccurate transcripts that teach the model incorrect acoustic-to-text mappings, compounding the WER problem rather than resolving it.

Five Acoustic and Linguistic Challenges Specific to Levantine ASR

1. The Lebanese qaf→hamza phoneme shift

The most distinctive phonological feature of Lebanese Arabic is the systematic replacement of qaf (ق — a uvular stop in classical Arabic and MSA) with a glottal stop (hamza, ء) in casual and colloquial speech. This is not a speaker-level variation — it is a productive rule in Lebanese Arabic phonology affecting the entire qaf vocabulary class. High-frequency words change fundamentally at the phoneme level: ‘قلب’ (heart) is pronounced ‘ʔalb’; ‘قال’ (said) is ‘ʔāl’; ‘قبل’ (before) is ‘ʔabl’.

For an MSA-trained ASR model, the glottal stop realisation of qaf is acoustically similar to hamza words (‘أمر’, ‘أخذ’, etc.) rather than the qaf words the model expects. The acoustic model either transcribes the Lebanese speech as the hamza-initial MSA equivalent (producing wrong-word errors) or introduces insertion errors as it tries to reconcile the unfamiliar acoustic pattern. A Lebanese service call containing fifty qaf-vocabulary words will produce fifty acoustic mismatches — the aggregate WER impact is substantial.

Native Lebanese transcriptionists normalise this phoneme shift automatically — they write the MSA orthography (‘قلب’ not ‘ألب’) for the Lebanese phoneme realisation, producing training data where the acoustic signal maps to the correct lexical form. Non-native transcriptionists, or automated transcription pre-annotation systems trained on MSA, produce the hamza-form orthography, creating phoneme-to-wrong-grapheme mappings that the fine-tuned model learns as errors.

2. Syrian qaf variation and uvular realisation patterns

Syrian Arabic has its own qaf realisation pattern that differs from both Lebanese and MSA. Educated urban Syrian Arabic (Damascene) retains the uvular stop closer to MSA qaf but with a more anterior, lighter realisation than Gulf or Egyptian Arabic. Rural and traditional Syrian Arabic — particularly from rural Homs, Hama, and Deir ez-Zor — may retain the classical uvular stop more strongly. Palestinian Arabic (across West Bank, Gaza, and Palestinian communities in Lebanon and Jordan) has yet another realisation: qaf is frequently pronounced as a glottal stop similar to Lebanese Arabic in urban speech, but with different lexical conditioning — some qaf words retain the uvular stop in Palestinian Arabic where the Lebanese equivalent would use glottal stop.

The variation means that a single “Levantine Arabic” acoustic model trained without sub-dialect annotation routing will perform sub-optimally on all four Levantine varieties. Production Levantine ASR datasets should include annotator metadata tagging the speaker's home region (Lebanon, Syria, Palestine, Jordan) and urban/rural register, so that acoustic model training can include sub-dialect conditioning features.

3. Lebanese vowel elision and fast-speech reduction

Lebanese Arabic in conversational speech applies pervasive vowel elision and consonant cluster reduction that is absent from MSA training data. Short vowels between consonants are frequently deleted: ‘كتير’ (ktīr, “a lot”) loses the initial vowel; sequences like ‘بقدر’ (“I can”) become ‘bʔdr’ in fast speech with the medial vowel deleted. Phrase-final vowels are regularly elided, merging the end of one word with the beginning of the next in ways that challenge standard acoustic word boundary detection.

These elision patterns are not phonological errors — they are productive fast-speech rules that every native Lebanese speaker applies and every native Lebanese listener compensates for automatically. MSA-trained ASR models produce insertion errors (inserting expected vowels that are not acoustically present) and word boundary errors at high rates on Lebanese fast speech, because the acoustic models were built on speech where the written vowels are acoustically realised.

4. French lexical borrowings in Lebanese speech

Lebanese Arabic has the most extensive French lexical borrowing of any Arabic dialect, reflecting Lebanon's French Mandate history and bilingual education system. French borrowings in Lebanese speech are not marginal — they occur in everyday vocabulary across domains. Common items: ‘merci’ (thank you), ‘bonjour’ (hello), ‘problème’ (problem), ‘numéro’ (number), ‘compte’ (account), ‘livraison’ (delivery), ‘téléphone’ (phone). In customer service contexts, French borrowings for account management, product names, and transaction terminology are extremely common.

Arabic ASR models trained on Arabic-only data have no acoustic model for French phonemes. French /r/ (uvular fricative), French nasal vowels, and French vowel quality distinctions are all outside the standard Arabic phoneme inventory. Lebanese speakers producing French-origin words in Arabic speech generate acoustic patterns that Arabic ASR models either fail to transcribe, transcribe as similar-sounding Arabic words, or transcribe as insertion errors with noise tags. For Lebanese contact centre ASR — where account management terminology is heavily French-borrowed — this produces consistent transcription failures on the most commercially important vocabulary class.

Case Study: Lebanese Contact Centre ASR — 46.2% to 18.7% WER

A Lebanese insurance company operating a voice-channel customer service centre needed to automate call transcription for quality assurance, agent coaching, and customer complaint tracking. The deployed ASR system used a Whisper-large model fine-tuned on Arabic broadcast speech — the most commonly available Arabic ASR starting point for commercial deployments.

Before: Word error rate on held-out Lebanese contact centre audio was 46.2%. Qaf-vocabulary words — a substantial proportion of everyday Lebanese service vocabulary — showed 61.4% WER due to the Lebanese qaf→hamza shift producing systematic mismatches. French-borrowed terminology (‘compte’, ‘police d'assurance’, ‘sinistre’, ‘livraison’) produced 73.8% WER — the French-origin insurance terminology that was most important for QA flagging was the category most consistently transcribed incorrectly. Segment boundary detection was also poor, with 18.3% of turn boundaries mis-detected due to fast-speech elision patterns causing the VAD to produce merged or split segments. Automated complaint identification from transcripts was operating at 31.2% recall — most complaint calls were not being flagged by the QA system, defeating the purpose of the transcription deployment.

The annotation project delivered 340 hours of transcribed Lebanese contact centre audio. Transcriptionists included eight native Lebanese Arabic speakers — six Beirut-native bilingual (Arabic-French), two Lebanese-South and Bekaa-native for regional accent coverage. The transcription convention document defined: phoneme normalisation rules (write MSA orthography for qaf-vocabulary words regardless of qaf→hamza realisation; write French words in French orthography for French-borrowed terms), fast-speech elision handling (transcribe at word-form level, not phoneme level — do not insert missing vowels), speaker diarisation protocol for overlapping speech in call centre contexts, and domain vocabulary lists covering Lebanese insurance terminology with French equivalents. All transcripts received a second-pass QA review by a different native Beiruti annotator, with disagreements resolved by the project lead. Transcript-level IAA on a held-out QA set reached 94.7% token agreement.

After fine-tuning on the annotated dataset: Overall WER improved from 46.2% to 18.7% on held-out Lebanese contact centre audio. Qaf-vocabulary WER improved from 61.4% to 22.3%, reflecting the acoustic model's ability to map Lebanese glottal-stop realisations to the correct qaf-vocabulary orthography. French-borrowed terminology WER improved from 73.8% to 24.1% — still higher than standard Arabic vocabulary WER, but reduced to a commercially acceptable level for QA use. Segment boundary detection accuracy improved from 81.7% to 94.8%. Automated complaint identification recall improved from 31.2% to 73.4% — more than doubling the effective QA coverage of the contact centre operation. The annotation project cost AUD $52,000 for 340 hours of transcription, QA, and delivery. The company estimated AUD $890,000 in annual QA operational savings from the improved automated complaint flagging and agent coaching, plus a 22.7% reduction in repeat contacts attributed to improved complaint identification and resolution.

Building the Annotation Protocol for Levantine ASR Projects

Levantine Arabic ASR annotation requires explicit protocol design decisions that most speech annotation projects skip. The essential elements are:

Phoneme normalisation conventions before transcription begins. The single most important protocol decision for Lebanese ASR is whether transcriptionists write the phoneme realisation or the MSA orthographic form for qaf-vocabulary words. For ASR model training, MSA orthographic form is almost always correct — the acoustic model needs to learn that the Lebanese glottal stop realisation maps to the qaf-orthographic form, not to a hamza-orthographic form. This must be documented and tested in the pilot before full-scale transcription begins.

French-token transcription protocol for Lebanese speech. The protocol must define explicitly whether French-borrowed words in Lebanese speech are transcribed in French orthography, Arabic transliteration, or both. For downstream ASR model use, French orthography is typically preferable if the language model includes French vocabulary; Arabic transliteration is preferable if the model is Arabic-only. For QA and search use cases, French orthography provides better downstream searchability for key insurance and banking terms.

Sub-dialect speaker tagging. Each audio segment should be tagged with the speaker's home region (Lebanese, Syrian, Palestinian, Jordanian) and urban/rural classification where identifiable. This metadata enables sub-dialect conditioning during model training and allows WER analysis to identify which sub-dialect varieties are most poorly covered by the current training set — enabling targeted data collection to address the largest remaining WER gaps.

Domain vocabulary supplement. Contact centre, medical, and government Levantine ASR require domain vocabulary lists provided to transcriptionists before annotation begins. Insurance terminology, medical drug names, government service names, and banking product terms all require consistent transcription conventions — particularly for French-Arabic hybrid terms where different transcriptionists may produce different orthographic choices.

AI Taggers’ Levantine Arabic NLP annotation service covers the full Shami dialect cluster for speech transcription, with native Beiruti and Damascene transcriptionists, French-Arabic bilingual protocol, sub-dialect speaker tagging, and domain vocabulary supplements for contact centre, medical, and government ASR deployments.

Compliance for Levantine Speech Annotation Projects

Recorded speech is personal data — and often sensitive personal data — under Lebanese Law No. 81 of 2018 and Jordan's Personal Data Protection Law of 2023. Voice is a biometric identifier under both laws. Contact centre recordings additionally contain financial account information, health disclosures, and complaint records that may constitute sensitive category data.

For ASR annotation projects using live contact centre recordings, the compliance framework requires: speaker consent or documented legitimate interest assessment for secondary use of the recordings in model training; annotator data access agreements and NDA covering the recorded speech content; data residency compliance for Levantine recordings subject to Lebanese or Jordanian data localisation requirements; and source audio deletion after transcript validation — the annotated transcripts, not the source recordings, are the deliverable retained for training.

For de-identified speech data collection — where audio is recorded fresh from consenting speakers — the compliance burden is lower but speaker consent must cover the use of their voice data for model training, specifically including the right to use the recordings in commercial AI products. See our post on PDPL vs GDPR for annotation vendors for a detailed comparison of Lebanese and Jordanian data obligations.

Related Reading

• Levantine Arabic Chatbot Intent Annotation: What Models Get Wrong Without Native Annotators
• Levantine Arabic Named Entity Recognition: What Models Get Wrong Without Native Annotators
• Gulf (Khaleeji) Arabic Speech Transcription: What Models Get Wrong Without Native Annotators
• Arabic NLP Annotation Service

Frequently Asked Questions