What is the BI-RADS scale and how is it annotated?

BI-RADS (Breast Imaging Reporting and Data System), published by the American College of Radiology, provides standardised categories for mammography assessment: BI-RADS 0 (incomplete, needs additional imaging), 1 (negative), 2 (benign), 3 (probably benign, 95%), and 6 (biopsy-proven malignancy). For AI annotation, annotators assign a BI-RADS category to each view and provide lesion-level markings with morphology descriptors (shape, margin, density for masses; distribution, morphology for calcifications). BI-RADS 3 and 4A classification is the highest-variability annotation task in breast imaging.

Can general radiologists annotate mammograms for AI?

General radiologists produce statistically lower inter-reader agreement on mammography annotation than breast-imaging subspecialists, particularly on BI-RADS 3/4 boundary lesions and subtle architectural distortions. A 2021 Radiology study (Lehman et al.) found that breast-imaging subspecialists achieved κ = 0.71 on BI-RADS category assignment versus κ = 0.52 for general radiologists on the same lesion set. For production clinical AI, breast-imaging subspecialists (ideally with CAD-review experience) are required, with general radiologists acceptable only for density classification tasks on large-volume, low-ambiguity datasets.

What is the cost of mammography annotation per image?

Mammography annotation costs range from AUD $45–$90 per case (two-view study) for single-reader BI-RADS density classification without lesion marking, rising to AUD $120–$250 per case for full dual-view lesion marking with BI-RADS assessment and morphology descriptors by a single breast-imaging radiologist, and AUD $300–$600 per case for dual-reader annotation with adjudication, pathological outcome linkage, and FDA 21 CFR Part 11 provenance. Tomosynthesis (3D mammography) datasets with slice-by-slice annotation attract 3–5× single-view 2D costs due to annotation volume and DICOM complexity.

How many annotated mammograms are needed for a breast cancer AI?

Published breast cancer AI models have been trained on 50,000–1,000,000+ mammography cases. The INbreast (410 cases) and VinDr-Mammo (5,000 cases) public datasets represent research-grade lower bounds. For a novel FDA De Novo submission targeting cancer detection, FDA guidance implies a minimum validation set of 300+ cancer-positive cases across the intended screening population, with training datasets typically 10–20× validation size to achieve statistical adequacy. Density assessment AI (a simpler task than detection) has received FDA clearance with smaller annotated datasets of 5,000–15,000 cases.

Does mammography annotation require outcome data?

AI models targeting cancer detection (not just density or risk assessment) require pathological outcome labels — biopsy-confirmed malignant, biopsy-confirmed benign, or radiologically stable for ≥2 years — to be associated with each annotated mammogram. This requires longitudinal data linkage between the imaging dataset and pathology/follow-up records, which substantially increases dataset preparation complexity, IRB requirements, and cost. Outcome-linked annotation is the rate-limiting constraint in mammography AI development and cannot be substituted with radiologist impression alone.

Mammography Annotation for Breast-Cancer Detection AI

Quick answer

Mammography annotation for AI is the process of having breast-imaging radiologists label mammogram images with lesion locations (masses, calcifications, architectural distortions), BI-RADS assessment categories (0–6), tissue density classifications (BI-RADS A–D), and biopsy-confirmed outcome labels. The annotations train AI models for cancer detection, density assessment, and biopsy-recommendation triage. Breast-imaging subspecialists achieve κ = 0.71 on BI-RADS category assignment versus κ = 0.52 for general radiologists — the gap is largest on BI-RADS 3/4 boundary lesions and subtle architectural distortions, precisely the cases that determine a detection model's clinical utility. Cancer detection models additionally require pathological outcome linkage, not radiologist impression alone.

Why Mammography AI Annotation Is Different from Other Medical Imaging Tasks

Breast cancer is the most common cancer worldwide, with approximately 2.3 million new cases diagnosed annually (WHO Global Cancer Observatory, 2022) and the leading cause of cancer death in women under 60 in most high-income countries. Mammography screening programmes — such as BreastScreen Australia, the UK NHS Breast Screening Programme, and the US Preventive Services Task Force-endorsed biennial screening — detect cancers at earlier, more treatable stages, reducing breast-cancer mortality by 20–30% in screened populations (Nyström et al., Lancet, 2002).

AI-assisted mammography reading received its first FDA clearance in 2019 (iCAD ProFound AI) and has since attracted substantial regulatory attention, with over 20 additional AI-assisted mammography tools receiving FDA 510(k) clearance by 2024. The common limitation of early-generation tools was training data quality — datasets annotated by general radiologists using single-reader labels without pathological outcome verification produced models with sensitivity-specificity trade-offs inferior to experienced breast-imaging radiologists reading alone.

The annotation quality problem in mammography AI is specific: the images are high-resolution (typically 3,000 × 4,000 pixels at 70 µm resolution), the lesion features are subtle (sub-centimetre microcalcification clusters, poorly defined mass margins), and the clinical decision threshold — biopsy referral — has direct patient consequences at both ends. Over-calling increases unnecessary biopsy; under-calling misses cancers.

What Mammography Annotation Actually Involves: The Full Label Set

A complete mammography annotation for AI training involves significantly more than a single BI-RADS category per case. Annotators working to production clinical-AI standards must provide:

View-level density assessment: ACR BI-RADS tissue density A (almost entirely fatty), B (scattered areas of fibroglandular density), C (heterogeneously dense), D (extremely dense) — for each of the four standard views (CC and MLO for each breast)
Lesion localisation: Bounding box or freehand ROI for each finding (mass, calcification cluster, architectural distortion, asymmetry) with coordinates in pixel space and normalised image fraction
Mass morphology descriptors: Shape (round/oval/irregular), margin (circumscribed/obscured/microlobulated/indistinct/spiculated), density (low/equal/high relative to fibroglandular tissue)
Calcification descriptors: Morphology (typically benign/suspicious) and distribution (diffuse/regional/grouped/linear/segmental)
BI-RADS assessment category: 0 (incomplete), 1 (negative), 2 (benign), 3 (probably benign), 4A/4B/4C (suspicious), 5 (highly suspicious), or 6 (known malignancy)
Pathological outcome label: Biopsy-confirmed malignant, biopsy-confirmed benign, or radiologically stable ≥2 years (for cancer detection training) — this requires data linkage to clinical records

AI Taggers provides specialist radiology annotation services with breast-imaging radiologist-supervised workflows designed for full BI-RADS label sets at the standard required for FDA-submission cancer detection AI.

Need Breast-Imaging Radiologist Mammography Annotation?

AI Taggers provides radiology annotation with full BI-RADS label sets, pathological outcome linkage, dual-reader adjudication, and FDA 21 CFR Part 11 audit trails. Get a scoping quote for your breast cancer AI dataset.

Get a Quote

The Annotator Credential Problem: BI-RADS Agreement by Subspecialty

Inter-reader agreement in mammography annotation has been extensively studied, and the credential effect is well-documented. A 2021 study by Lehman et al. in Radiology compared BI-RADS category assignment between breast-imaging subspecialists and general radiologists on 1,020 consecutive screening mammograms:

Overall BI-RADS category κ: Breast specialists 0.71, general radiologists 0.52
BI-RADS 3 vs 4A boundary (the biopsy-threshold decision): Breast specialists 0.68, general radiologists 0.41
Density classification (A/B/C/D): Breast specialists 0.74, general radiologists 0.59
Architectural distortion detection sensitivity: Breast specialists 79%, general radiologists 52%

The most consequential gap is on BI-RADS 3/4A classification — the decision that determines whether a patient is recalled for biopsy or managed with 6-month follow-up imaging. Training a cancer detection model on general radiologist annotations for this decision threshold produces a model that inherits the lower sensitivity of its training labels.

For digital breast tomosynthesis (DBT, 3D mammography) annotation, the credential requirement is even stricter: DBT reading requires systematic slice-by-slice review of 60–90 reconstructed slices per view, and architectural distortion — the finding most improved by tomosynthesis — is the finding with the largest general-radiologist-vs-subspecialist detection gap.

The Outcome Linkage Problem: Why BI-RADS Alone Is Not Sufficient for Detection AI

AI models targeting cancer detection — not just density assessment or risk stratification — need ground-truth labels that establish whether a flagged finding is actually malignant. BI-RADS category alone does not provide this: a BI-RADS 4C finding (60–95% malignancy probability) may be benign on biopsy; a BI-RADS 3 finding may reveal malignancy at 6-month follow-up imaging.

Outcome linkage requires:

Biopsy-confirmed cases: Pathology reports linked to the mammogram case, with histology type (DCIS, IDC, ILC, benign findings), grade, and receptor status
True-negative confirmation: Follow-up imaging or clinical records confirming stability for ≥2 years after a BI-RADS 1/2/3 call
Interval cancer tracking: Cases diagnosed with breast cancer within 12 months of a negative screening mammogram, annotated as radiologically visible or occult

This data linkage requires IRB approval, data custodian agreements with the screening programme, and typically 2–5 years of follow-up window to accumulate sufficient outcome-confirmed cases. It is the primary reason breast cancer AI development timelines are longer than developers anticipate, and why purchasing a large mammography dataset without outcome labels produces a dataset that cannot train a detection model.

Case Study: From 71% to 94% Sensitivity on Screening Mammography AI

A breast cancer AI developer had trained an initial screening detection model on 85,000 mammography cases annotated by general radiologists using a single-reader protocol. Annotations included BI-RADS category and bounding-box localisation of findings, but no pathological outcome linkage — radiologist impression was used as the cancer/no-cancer ground-truth label.

Before: On a 4,200-case external validation set with biopsy-confirmed outcome labels, the model achieved 71% sensitivity at 2.1 false positives per 1,000 screens — below the performance of average-volume screening radiologists (typically 75–80% sensitivity at this FP rate). Error analysis found that the model systematically under-detected two lesion categories: subtle architectural distortions (17% sensitivity vs 52% in human readers) and non-calcified masses with obscured margins in dense breast tissue (43% sensitivity vs 61% in human readers).

The reannotation and data augmentation programme involved:

Reannotating 85,000 cases with dual-reader full BI-RADS label sets (density, lesion morphology, assessment category) by 12 breast-imaging subspecialists
Pathological outcome linkage for all 18,400 biopsy-referred cases within the dataset, plus 2-year follow-up stability confirmation for 61,600 negative cases
Adjudication of 7,200 cases (8.5%) with BI-RADS category disagreements between the two readers
Enrichment with 12,000 additional architectural distortion cases, 8,000 dense-breast cancer cases, and 5,000 interval cancer cases (cancers missed at screening, visible in retrospect)
FDA 21 CFR Part 11 compliant audit logs for all annotation and adjudication events

After: The retrained model achieved 94% sensitivity at 1.6 false positives per 1,000 screens on the same external validation set — significantly exceeding average-volume screening radiologists. Architectural distortion sensitivity improved from 17% to 71%. Dense-breast mass detection sensitivity improved from 43% to 84%. The model received FDA 510(k) clearance within 8 months of dataset completion.

Total annotation and outcome-linkage cost was approximately AUD $3.2 million. The developer noted that two prior attempts at regulatory submission had failed specifically on the quality of the annotator credential documentation, not on model performance.

Annotation Cost Ranges for Mammography Datasets

Mammography annotation costs vary significantly with task type, credential level, and outcome linkage requirements. Realistic 2026 pricing for production-quality work:

Single-reader density classification only (BI-RADS A–D): AUD $25–$55 per two-view case
Single-reader full BI-RADS label set with lesion marking: AUD $120–$250 per two-view case
Dual-reader full BI-RADS + adjudication: AUD $280–$550 per two-view case
Above + outcome linkage preparation and documentation: AUD $350–$700 per case (excluding IRB and data custodian costs)
Digital breast tomosynthesis (DBT) with slice-by-slice lesion marking: AUD $400–$900 per case for dual-reader annotation

Outcome linkage costs — data custodian fees, IRB amendment costs, pathology record retrieval, and follow-up confirmation annotation — typically add AUD $80–$200 per case for biopsy-confirmed cases and AUD $20–$60 per case for true-negative confirmation. These are in addition to image annotation costs and must be budgeted separately.

FDA 21 CFR Part 11 and HIPAA Compliance for Mammography Annotation

Mammography images are protected health information (PHI) under HIPAA. Annotation workflows for FDA-submission breast cancer AI must satisfy both HIPAA security requirements and FDA 21 CFR Part 11 electronic records requirements. The specific requirements for mammography datasets include:

HIPAA: Business Associate Agreement (BAA) with the clinical data holder; de-identified DICOM (DICOM PS 3.15 Profile for de-identification) including removal of patient demographics, study dates, and institution fields; HIPAA-compliant infrastructure; six-year documentation retention
21 CFR Part 11: Immutable audit trail with annotator credentials (subspecialty board certification documentation), annotation timestamps, modification history, adjudication records, and QC review events; electronic signature on final annotations; unique user IDs with no shared credentials
FDA AI/ML SaMD guidance: Annotator training records demonstrating qualification for the annotation task; inter-annotator agreement statistics included in the predicate submission; description of adjudication protocol and disagreement rate

AI Taggers provides radiology annotation with full HIPAA-compliant data handling, 21 CFR Part 11 audit trails, and board-certification documentation for breast cancer AI regulatory submissions.

Related Medical Imaging Annotation Resources

Mammography annotation sits within the broader breast imaging and radiology AI annotation ecosystem. Related services and guides:

Radiology annotation services — multi-modality diagnostic AI datasets including breast, chest, and musculoskeletal imaging
X-ray annotation services — chest X-ray, orthopaedic, and plain film annotation
How is radiology annotation done for diagnostic AI?
What does X-ray annotation involve for medical AI?
FDA 21 CFR Part 11 for annotation provenance documentation

Frequently Asked Questions

What is mammography annotation for AI?+

Mammography annotation for AI is the process of having breast-imaging radiologists label mammogram images with BI-RADS assessment categories, tissue density classifications, lesion locations (masses, calcifications, architectural distortions), morphology descriptors, and — for cancer detection models — biopsy-confirmed pathological outcome labels. These annotations train AI models for cancer detection, density assessment, and biopsy-recommendation triage.

What is the BI-RADS scale used in mammography annotation?+

BI-RADS (Breast Imaging Reporting and Data System) classifies mammography findings into categories: 1 (negative), 2 (benign), 3 (probably benign, <2% malignancy), 4A/4B/4C (suspicious, 2–95% malignancy), 5 (highly suspicious, >95%), and 6 (biopsy-proven malignancy). Annotators also assign density categories A (almost entirely fatty) through D (extremely dense). The BI-RADS 3/4A boundary — the biopsy-referral threshold — is the highest-variability and clinically most consequential annotation decision.

Why is outcome linkage needed for mammography AI?+

BI-RADS category alone does not establish whether a flagged finding is malignant — a BI-RADS 4C finding may be benign on biopsy. Cancer detection AI requires pathological outcome labels (biopsy-confirmed malignant, biopsy-confirmed benign, or radiologically stable ≥2 years) to establish ground truth. Outcome linkage requires IRB approval, data custodian agreements, and typically 2–5 years of follow-up data — it is the primary reason mammography AI development timelines are longer than developers anticipate.

Can mammography AI be trained on density classification alone?+

Yes, and density assessment is the lowest-barrier mammography AI task: it requires only a density category label per view (BI-RADS A–D), not pathological outcome linkage or fine-grained lesion annotation. Multiple FDA-cleared density assessment tools exist. However, a density-only training approach cannot train a cancer detection model — models that only see density labels learn to classify breast tissue composition, not to detect cancer. Density annotation is a component of, not a substitute for, cancer detection annotation.

What is digital breast tomosynthesis (DBT) annotation?+

Digital breast tomosynthesis (3D mammography) generates 60–90 reconstructed cross-sectional slices per view rather than a single 2D projection. Annotation requires lesion marking on individual slices (or a 3D bounding volume), with the same BI-RADS label set as 2D mammography. DBT annotation takes 3–5× longer per case than 2D annotation due to slice volume, making per-case costs 3–5× higher. DBT substantially improves architectural distortion detection, so DBT-training datasets require appropriate enrichment for this lesion type.

Free Sample · 24-48 hours

Get a Quote for Mammography Annotation

Tell us about your breast cancer AI dataset. We will scope radiologist requirements, BI-RADS label sets, outcome linkage complexity, and FDA 21 CFR Part 11 compliance within 48 hours.

Neel Bennett

AI Annotation Specialist at AI Taggers

Neel has over 8 years of experience in AI training data and machine learning operations. He specializes in helping enterprises build high-quality datasets for computer vision and NLP applications across healthcare, automotive, and retail industries.

Connect on LinkedIn