How Is Retinal Image Annotation Used to Detect Eye Disease with AI?

The Scale of Preventable Blindness Driving Retinal AI Investment

Diabetic retinopathy affects approximately 103 million adults globally (Teo et al., The Lancet Diabetes & Endocrinology, 2023) and is the leading cause of new blindness in working-age adults in developed economies. An estimated 45% of DR cases go undetected in routine care due to inadequate screening coverage — a gap that AI-based retinal screening is designed to close. The global retinal imaging AI market was valued at USD $189 million in 2024 and is projected to reach USD $1.4 billion by 2032 (MarketsandMarkets, 2024), making retinal imaging one of the highest-growth segments in diagnostic AI.

Our retinal image annotation service covers the four primary annotation modalities driving this market:

ICDR Grading: The Foundation of DR Screening AI

The International Clinical Diabetic Retinopathy (ICDR) severity scale is the standard annotation taxonomy for DR screening AI datasets. The five-grade scale provides a clinically actionable classification that maps to screening programme referral thresholds:

The clinically critical cutpoint for screening AI is the referable/non-referable binary decision — Grade 0/1 vs Grade 2/3/4. AI systems deployed for DR screening must achieve sensitivity above 90% for referable DR at a specificity above 85% to meet the clinical standard set by the landmark IDx-DR FDA clearance in 2018 (the first autonomous AI diagnostic cleared by FDA for any indication). Training data quality at this binary cutpoint — specifically, the consistency of Grade 2 vs Grade 1 classification on cases near the boundary — is the dominant determinant of where the AI ROC curve sits.

Annotation guidelines must specify exactly what constitutes a microaneurysm (to distinguish Grade 0 from Grade 1), and what constitutes "more than microaneurysms" to distinguish Grade 1 from Grade 2. Without explicit worked examples for the Grade 1/2 boundary — where dot haemorrhages are indistinguishable from microaneurysms on non-dilated fundus photography — inter-annotator variability at this boundary is consistently the largest source of training label noise in DR datasets.

AMD and OCT Annotation: The Volume Problem

Age-related macular degeneration affects approximately 196 million people globally (GBD 2019 Blindness and Vision Impairment Collaborators, The Lancet Global Health, 2021) and is the leading cause of visual impairment in high-income countries. OCT imaging is the primary modality for AMD monitoring, and OCT annotation is among the most data-intensive tasks in medical image AI.

A single macular OCT volume at standard clinical resolution (512 × 512 × 49 A-scans or 512 × 1024 × 97 B-scans) contains 49–97 individual B-scan slices, each of which requires annotation for retinal layer boundaries and fluid compartments. A dataset of 1,000 OCT volumes therefore contains 49,000–97,000 annotatable slices — a volume that makes fully manual annotation prohibitively expensive without tool-assisted workflows.

For AMD specifically, drusen annotation is the primary training task for early and intermediate AMD AI. Drusen appear as bright deposits beneath the retinal pigment epithelium (RPE) on OCT, visible as elevations of the RPE-Bruch's membrane complex. Annotation guidelines must specify the minimum drusen size for labelling (to exclude drusenoid PED from soft drusen annotations), whether confluent drusen areas are annotated as a single region or individual deposits, and how the transition from soft drusen to geographic atrophy (GA) is handled when both are present in the same scan.

Retinal fluid classification in OCT — distinguishing intraretinal fluid (IRF) from subretinal fluid (SRF) from sub-RPE fluid — is the annotation task most sensitive to annotator qualification. The distinction between IRF and SRF determines treatment eligibility in anti-VEGF therapy protocols (IRF is associated with active exudative AMD requiring treatment; SRF management is more nuanced), and annotation errors at this boundary directly affect the clinical appropriateness of treatment decisions the AI supports.

Case Study: Diabetic Retinopathy Screening AI for an Australian Community Pharmacy Programme

An Australian digital health company was developing a DR screening AI system for deployment in community pharmacy settings — environments where fundus cameras are operated by pharmacy technicians without ophthalmology training. The intended clinical application was autonomous triage: Grade 0–1 images routed to 12-month recall with no clinical review, Grade 2+ images flagged for teleophthalmology review within 48 hours.

The company had developed an initial model using 12,000 fundus images annotated via a crowdsourced platform with general medical annotators. Evaluation on a 1,200-image held-out test set graded by ophthalmologists showed sensitivity for referable DR of 74.3% at 85% specificity — well below the 90% sensitivity threshold required for autonomous screening deployment. The false negative rate on Grade 2 moderate NPDR was 31.2%, driven almost entirely by Grade 1/2 boundary confusion in the training data.

Root cause analysis: Review of the crowdsourced annotations against ophthalmologist reference grades on a 500-image diagnostic subsample showed kappa of 0.41 at the Grade 1/2 boundary — effectively chance agreement on the clinically critical cutpoint. The cause was the absence of a microaneurysm vs dot haemorrhage distinction rule: both are small red dots on non-dilated fundus photography, but dot haemorrhages signal more severe vascular damage (Grade 2) while microaneurysms alone indicate Grade 1. Without fluorescein angiography (FA) — which is not performed in screening settings — this distinction is probabilistic and requires ophthalmologist-calibrated clinical judgement or a specific protocol for how annotators should handle the ambiguous cases.

Our approach: We developed a grading protocol in collaboration with two Australian ophthalmologists that explicitly addressed the microaneurysm-vs-haemorrhage boundary. Where the distinction was genuinely ambiguous on non-dilated fundus photography (estimated to affect 12–18% of Grade 1/2 boundary cases), annotators applied a "borderline Grade 2" flag that triggered mandatory dual ophthalmologist grading with formal adjudication. This produced an ophthalmologist-resolved grade for every ambiguous case rather than propagating crowd disagreement as noisy training signal. All 28,000 images were graded by trained annotators working from the protocol, with 100% ophthalmologist review on Grade 2–4 images and 15% random sampling on Grade 0–1 images. The complete ICDR grade distribution, image quality flag rates, and annotator IAA statistics were documented for the TGA technical evidence file.

Sensitivity improvement from 74.3% to 92.1% — crossing the 90% clinical deployment threshold — came primarily from the borderline Grade 2 adjudication protocol, which removed the Grade 1/2 boundary noise that had been the dominant failure mode. The ungradeable false pass rate reduction from 8.9% to 1.2% came from explicit image quality grading criteria (specifying minimum field illumination, acceptable pupil dilation, and artefact exclusion rules), which ensured that technically inadequate images did not contribute false-negative signal to the training distribution. See our retinal image tagging service for capability details and workflow options.

Annotator Qualification and Quality Standards for Retinal AI

Retinal image annotation spans a wider range of acceptable annotator qualification levels than most medical imaging tasks, because the complexity of annotation tasks varies enormously across the disease spectrum. A structured decision framework:

For regulatory documentation requirements in ophthalmic SaMD submissions, our guide to FDA 21 CFR Part 11 annotation documentation covers the provenance log requirements that apply to retinal AI training data. For a broader perspective on credentialled clinical annotation across medical specialties, see what clinical-expert AI annotation involves. Our full radiology annotation service covers retinal and ophthalmic imaging alongside CT, MRI, and X-ray modalities.

Retinal Annotation Cost and Throughput

Fundus photograph annotation is cost-efficient at scale — each image is 2D and annotation tasks are well-defined once grading protocols are established. OCT volume annotation is significantly more expensive due to the slice count per volume.

For large-scale DR screening AI datasets (10,000+ images), volume pricing typically reduces per-image rates by 25–40%. Quality validation workflows — the IAA documentation, reference standard annotation, and per-class grading statistics required for TGA and FDA submissions — are included in our retinal image annotation service rather than invoiced separately. For more context on medical imaging annotation across modalities, see our case study on how radiology annotation is done for diagnostic AI.

Frequently Asked Questions