How Is CT Scan Annotation Done for Radiology AI?

What Makes CT Annotation Fundamentally Different From Other Medical Imaging

A CT scan is not a single image — it is a three-dimensional volume reconstructed from hundreds or thousands of individual axial slices, each 0.5–5 mm thick. A chest CT typically contains 300–600 slices. An annotator who labels only the slice where a nodule appears largest has not annotated a CT scan; they have annotated one frame of it. Production CT annotation requires either 3D volumetric segmentation tools that operate across the full slice stack, or systematic multi-slice labelling with explicit cross-referencing between adjacent slices to maintain spatial continuity.

The second distinguishing feature is Hounsfield unit windowing. CT pixel values encode radiodensity on the Hounsfield scale, and a single CT volume contains tissue types spanning roughly 3,000 HU — from air at -1000 HU through soft tissue, blood, muscle, calcification, and cortical bone. No single display window makes all pathologies visible simultaneously. An annotator working on pulmonary nodule detection must apply the lung window (-600 to +1600 HU). Switching to the mediastinal window (-160 to +240 HU) for the same dataset will hide small nodules entirely. Annotation performed in the wrong window systematically misses findings and produces corrupted training labels.

Our CT scan annotation service covers thoracic (chest CT, pulmonary nodule, lung cancer screening), abdominal (liver, kidney, pancreas, colon), musculoskeletal (fracture, joint pathology), and neuroradiology (brain lesion, stroke, haemorrhage) modalities. Each task type has its own windowing protocol, annotator qualification standard, and QA framework.

The CT Annotation Workflow: Four Stages That Determine Data Quality

A well-designed CT annotation workflow is sequential and non-negotiable. Compressing or skipping stages to meet volume targets is the most common root cause of CT training datasets that fail to improve model performance.

Multi-Slice Consistency: The Hardest Part of CT Annotation to Get Right

The most common quality failure in CT annotation is slice-to-slice inconsistency: a nodule annotated as 8.3 mm on slice 142 but 6.1 mm on slice 141, a tumour boundary that jumps 3 mm between adjacent slices without biological justification, or a structure annotated as present on even-numbered slices and absent on odd-numbered slices due to annotator fatigue. These inconsistencies do not prevent the annotation from appearing visually acceptable in spot-check reviews — they only manifest when the 3D volume is rendered or when the training pipeline extracts slice-level crops.

Three practices enforce multi-slice consistency reliably. First, annotation must proceed in sequential slice order rather than jumping to the "most obvious" slice. Second, inter-slice measurement consistency must be quantitatively verified — lesion diameter variance across contiguous slices should not exceed the annotated structure's biological variability at the imaging resolution. Third, 3D rendering must be performed on every annotated volume before QA sign-off, since 3D renders reveal topological errors (disconnected segmentations, slice-boundary artefacts) that 2D slice review misses.

The global AI in radiology market is projected to reach USD$4.5 billion by 2030 (MarketsandMarkets, 2024), with CT-based applications — including lung cancer screening, cardiac risk assessment, and polyp detection — representing the largest segment. The data quality requirements for these applications are correspondingly high: a 2023 meta-analysis in Radiology: Artificial Intelligence found that AI models for pulmonary nodule detection trained on consistently windowed and inter-rater-validated CT datasets achieved 93% sensitivity at the 6 mm detection threshold, compared to 81% sensitivity for models trained on inconsistently annotated data from the same patient population.

Regulatory Compliance: FDA 21 CFR Part 11 and TGA Requirements

CT annotation for diagnostic AI that will be submitted to the FDA as Software as a Medical Device (SaMD) requires annotation provenance that satisfies FDA 21 CFR Part 11. This regulation governs electronic records and electronic signatures and mandates: tamper-evident audit trails recording who annotated each finding, when, and what changes were made; electronic signature controls preventing annotation identity spoofing; system access logs; and record retention for the duration of device clearance plus applicable post-market surveillance.

Beyond Part 11, FDA's 2021 guidance on AI/ML-based SaMD requires that the training dataset reference standard (ground truth) methodology be documented in the submission, including annotator qualification criteria, inter-rater agreement statistics, and the process for resolving disagreements. This means teams cannot retrospectively reconstruct annotation provenance — the documentation must be built into the annotation workflow from day one.

In Australia, the Therapeutic Goods Administration (TGA) applies similar requirements under the Software as a Medical Device framework for Class III devices. Australian medical AI teams should expect TGA to request annotation methodology documentation as part of conformity assessment for high-risk radiology AI. Our post on FDA 21 CFR Part 11 annotation documentation covers the specific provenance log fields required for regulatory submissions.

Case Study: Pulmonary Nodule Detection Model for Australian Hospital Networks

A radiology AI company developing a pulmonary nodule detection and risk stratification model for deployment across Australian hospital networks engaged AI Taggers to annotate their CT chest dataset. The client had initially attempted annotation with an offshore crowdsourced vendor; preliminary model benchmarking revealed sensitivity of 74% at the 3–6 mm nodule threshold — the most clinically critical size range for early lung cancer detection — which was insufficient for intended clinical use.

Root cause of the original quality problem: The offshore vendor had used non-radiologist annotators working in lung window only. Subsolid and ground-glass nodules — which require mediastinal window cross-referencing to characterise correctly — had been systematically mislabelled or missed. Inter-rater agreement on nodule malignancy classification was kappa = 0.41, indicating poor reliability on the most clinically significant task.

Our approach: We deployed a radiologist-led annotation team using OHIF v3 with custom window preset configurations for lung, mediastinal, and bone windows across every volume. Each CT volume was annotated by one radiologist and QA-reviewed by a second. Contested nodule characterisations — approximately 8% of all nodules — were adjudicated by a senior thoracic radiologist. Annotation provenance was logged in a Part 11-compliant audit system with electronic signature controls. Inter-rater agreement measurement was performed on a rolling 12% sample.

The sensitivity improvement from 74% to 91% on the 3–6 mm range — the most diagnostically critical — was driven primarily by the multi-window annotation protocol and the adjudication process for subsolid nodules. The false positive reduction from 4.2 to 1.6 per volume came from the radiologist QA layer catching non-nodule structures (vessel cross-sections, scar tissue, mucus plugging) that non-radiologist annotators had incorrectly flagged. The complete Part 11 audit trail made the 510(k) submission documentation straightforward; the client received FDA clearance 11 months after dataset completion.

CT Annotation Cost and Realistic Throughput

CT annotation cost is highly task-dependent. Detection tasks on straightforward findings — fracture presence in a long bone CT, pneumothorax in chest CT — can be performed by trained annotators under radiologist QA at reasonable throughput. Complex volumetric segmentation with characterisation requires radiologist primary annotation, which is slower and more expensive but unavoidable for regulatory-grade data.

For context on how CT annotation fits into broader radiology AI data programmes, our radiology AI annotation guide covers multi-modality workflows including MRI, X-ray, and pathology. Teams building multi-organ models may also need organ segmentation annotation services, which we offer in parallel with CT-level pathology labelling. For whole-slide histopathology workflows that complement CT-based cancer staging, see our overview in histopathology whole-slide annotation.

Pilot annotations — typically 100–300 CT volumes — are available within two to four weeks and allow teams to evaluate annotation quality and inter-rater agreement before committing to full-dataset production. Our CT scan annotation service includes a detailed annotation guideline co-development session before any production annotation begins, ensuring windowing protocols, finding definitions, and characterisation criteria are specified precisely enough to achieve consistent results at scale. We also support broader radiology annotation programmes across modalities.

Frequently Asked Questions