Turkish NLP Data Annotation: What Makes It Hard and How to Get It Right

Why Turkish Breaks Generic NLP Annotation Pipelines

Turkish is an agglutinative SOV (subject-object-verb) language — typologically distant from English and from every major Western European language. The famous example used in Turkish linguistics classrooms: a single Turkish word, çekoslovakyalılaştıramadıklarımızdanmışsınızcasına, translates approximately as "as if you are one of those whom we could not manage to make Czechoslovakian."

For NLP annotation, this agglutinative property creates a fundamental problem: what is the "token" in Turkish? Each suffix added to a root creates a new surface form with a distinct meaning. Segmenting those suffixes correctly is a prerequisite for span annotation — but Turkish morphological segmentation requires native-speaker intuition or a specialised morphological analyser, not just a standard whitespace tokeniser.

Turkey's AI market was valued at USD 1.1 billion in 2023 and is projected to reach USD 4.8 billion by 2030 (Grand View Research, 2024) — driven by government mandate, large domestic tech sector (Trendyol, Getir, BiTaksi), and Turkish diaspora markets in Germany and the Netherlands. This growth is creating annotation demand that outpaces the available native-speaker annotator pool, leading to the same quality shortcuts seen in Arabic and Hebrew annotation markets.

The Four Structural Challenges of Turkish NLP Annotation

1. Agglutinative morphology and token segmentation

Turkish builds words by stacking derivational and inflectional suffixes onto a root in a specific order. The Turkish root ev (house) generates: evler (houses), evlerde (in the houses), evlerden (from the houses), evlerimizde (in our houses), evlerinizde (in your houses [plural polite]), evlerimizden (from our houses) — and dozens more forms, each a single orthographic word.

For NER annotation, the span boundary problem is acute. If a person's name appears in the dative case — Ahmet'e (to Ahmet) — the entity span should cover only Ahmet, not the case suffix -e. But if the annotation tool presents the whole orthographic word as a single token without morphological segmentation, the annotator must manually split the token — and different annotators split at different positions, producing inconsistent training data.

Published Turkish NER benchmarks show that span boundary consistency improves from approximately κ 0.63 to κ 0.84 when text is pre-segmented with a Turkish morphological analyser before annotation (Tür et al., 2003; results replicated on modern corpora by Yıldız et al., 2019 with zemberek-NLP). This is the single highest-value step in any Turkish NLP annotation pipeline.

2. Vowel harmony and surface form proliferation

Turkish vowel harmony means that each suffix appears in multiple allomorphic forms determined by the vowel of the preceding syllable. The plural suffix, for instance, appears as -lar after back vowels (a, ı, o, u) and as -ler after front vowels (e, i, ö, ü). The same suffix pattern extends to case endings, tense markers, and agreement suffixes — each with two to eight surface variants.

For annotation, vowel harmony means that string matching on suffixes is unreliable — you cannot write a rule-based system that finds all instances of the past-tense marker because it appears as -dı, -di, -du, -dü, -tı, -ti, -tu, -tü depending on the stem and consonant environment. NLP models must generalise across these surface variants, and training data annotations must be consistent about where suffix boundaries lie — which requires annotators who understand the harmony rules, not just annotators who can read Turkish characters.

3. Pro-drop and zero subjects

Turkish is a pro-drop language — subjects are routinely omitted because verb agreement suffixes encode the subject person and number. The sentence Geldi (literally "came-3SG") is complete and means "He/she/it came" without any expressed subject. In a conversation, the referent of that zero subject can only be recovered from context.

For coreference and intent annotation, this means annotators must resolve zero subjects across sentence boundaries based on discourse context alone. This is a task that requires strong comprehension of Turkish discourse structure — not achievable by annotators working from translation or with intermediate proficiency.

In sentiment annotation, pro-drop creates ambiguity that even native-speaker annotators debate. The sentence Beğenmedi ("didn't like [it/him/her/them]") is a complete negative sentiment expression — but without context, the annotator cannot determine the target of the sentiment. Turkish sentiment datasets without careful context-window design suffer from higher-than-expected annotator disagreement on this class of sentence.

4. Social media orthography and code-switching

Turkish social media text has a distinctive orthographic pattern: users frequently substitute ş with s, ğ with g, ü with u, and ö with o — the special characters introduced by Atatürk's 1928 alphabet reform that are slightly awkward on some keyboard layouts. This non-standard orthography produces surface forms that morphological analysers trained on standard Turkish cannot segment correctly.

Turkish digital text also features heavy English code-switching in technology, startup, and youth contexts: Meeting bugün cancel edildi ("The meeting today was cancelled") mixes English nouns with Turkish verb inflection. Annotation guidelines for Turkish social media and conversational AI must specify how to handle these mixed forms — particularly when an English word receives a Turkish suffix that creates a genuinely new Turkish surface form.

The Turkish NLP Tooling Ecosystem

Turkish NLP has a well-developed open-source tooling ecosystem centred on Turkish university research groups. The key components for annotation pipelines are:

Case Study: Turkish E-Commerce Platform — Product Review Sentiment Recovery

In late 2025, a Turkish e-commerce platform (comparable scale to Trendyol or Hepsiburada) needed 80,000 annotated product reviews for a sentiment and aspect classification model. Reviews spanned electronics, fashion, and home goods categories, with a high proportion of informal Turkish and mixed Turkish-English text.

The initial annotation run used a multilingual crowdsourcing platform. After 18,000 records, a Turkish NLP engineer reviewed a sample and found:

• Sentiment accuracy of 71.4% on a 500-record gold standard evaluated by native Turkish speakers — well below the 88% target
• 32% of aspect-level annotations were on incorrect token spans due to morphological segmentation inconsistencies
• Non-standard orthography (begen instead of beğen, gorme instead of görme) was being labelled as "other" class by non-native annotators who did not recognise the vowel harmony-dropped forms
• Turkish negation patterns — which use a suffix on the verb rather than a separate "not" word — were causing non-native annotators to misclassify negative sentences as neutral at a 23% rate

The team rebuilt the annotation pipeline with native Turkish-speaking annotators and proper morphological pre-processing:

• zemberek-NLP pre-processing on all 80,000 reviews to resolve morphological boundaries before annotation
• A Turkish social-media normalisation step that mapped common non-standard orthography variants to canonical forms for the annotation interface (while preserving originals in the output)
• Native Istanbul Turkish-speaking annotators, with a calibration set of 200 "negation trap" sentences designed to test correct negation interpretation
• Double annotation on 15% of records with kappa measurement per sentiment class
• Aspect taxonomy developed collaboratively with the client's Turkish-language product team

Results on the re-annotated corpus:

The native-speaker annotation cost was 2.4× higher per record. But the 18,000 crowd-annotated records were completely unusable — representing a sunk cost that exceeded the premium for the entire 80,000-record re-annotation project. The downstream sentiment model, trained on native-speaker data, achieved 89.8% accuracy on the production holdout set, enabling the platform to deploy aspect-based product review summaries with reliable Turkish negative/positive classification.

Turkish Annotation Guidelines: What English Templates Miss

Turkish annotation guidelines adapted from English templates routinely omit the language-specific instructions that prevent the most common errors. Critical inclusions are:

• Morpheme boundary rules: Define explicitly whether entity spans should cover bare roots, roots plus case suffixes, or roots plus all suffixes. For person and organisation NER, the standard is to span the root and possessive suffixes (-in, -ın, -un, -ün) but not case suffixes (-e, -a, -de, -da, -den, -dan). This must be illustrated with 10+ examples in the guidelines.
• Negation annotation: Turkish negation uses the suffix -mA- (with vowel harmony) on the verb, not a separate negative particle. Guidelines must include a section on recognising morphological negation and must not assume annotators will infer this from English annotation experience.
• Non-standard orthography: Provide a mapping table of common social-media orthography variants (s→ş, g→ğ, u→ü, o→ö) and specify that annotators should apply the semantic label appropriate to the intended standard form, not treat non-standard forms as "unknown."
• Code-switching boundaries: Specify whether English-origin words that have received Turkish suffixes (meetingi = "the meeting [accusative]") should be annotated as Turkish tokens, English tokens, or a special class.
• Pro-drop context windows: For intent or coreference annotation, specify the context window (typically 3–5 sentences) that annotators must consider before making a label decision on a sentence with a zero subject.

The Turkish AI Market and Annotation Demand

Turkey's National AI Strategy 2021–2025 explicitly identified Turkish language AI as a priority investment area, funding Turkish NLP research at TUBITAK (Scientific and Technological Research Council of Turkey) and establishing the AI Institute at Sabancı University. This investment is creating demand for Turkish training data across government AI services, Turkish-language chatbot development, and the domestic tech sector.

The Turkish diaspora dimension adds a second market: approximately 3 million Turkish speakers in Germany, 400,000 in the Netherlands, and 350,000 in Austria create demand for Turkish-language capabilities in European banking, insurance, and government AI applications. German and Dutch companies building multilingual AI products increasingly need Turkish annotation alongside their core language requirements.

Our Turkish data annotation service covers NER, sentiment, intent, morphological segmentation, and document classification — with native Istanbul Turkish-speaking annotators and the zemberek-NLP pre-processing pipeline included. For teams working across multiple language markets simultaneously, our multilingual localization annotation services handle Turkish alongside Arabic, Hebrew, Persian, and 120+ other languages in a single managed workflow, with consistent QA protocols across all languages.

Related Reading

If you are building Turkish NLP annotation capability alongside other language requirements, these posts cover the annotation challenges and strategies for closely related contexts:

• Hebrew NLP Data Annotation: What Makes It Hard and How to Get It Right — Semitic morphology, RTL, and the parallel challenges of a different root-based language system
• How Does Turkish Data Annotation Work for AI? (Native-Speaker Case Study) — Earlier case study focused on the initial project scoping and annotator selection decisions
• How Does Multilingual Annotation and Localization Work for Global AI? — Multi-language annotation workflows when Turkish is one of several target languages

Frequently Asked Questions