Working with DICOM: Medical Image Annotation Without Losing Metadata

What DICOM Is and Why It Matters for Annotation

DICOM — Digital Imaging and Communications in Medicine — is the international standard for storing, transmitting, and managing medical imaging data. A DICOM file is not just an image. It is an image plus a structured header containing several hundred metadata attributes: patient demographics, study identifiers, acquisition parameters, equipment details, and spatial calibration data (pixel spacing, slice thickness, image position) that define the physical coordinates of each pixel.

This metadata is what makes DICOM files clinically useful — and what makes them problematic for annotation without a proper workflow. A CT slice exported to JPEG retains only the pixel array. Pixel spacing (typically 0.5–0.9 mm for chest CT) and slice position are gone. Any measurement made on that JPEG cannot be converted to physical units. Any segmentation mask cannot be positioned back into the patient coordinate system. For AI applications where the model's output must be interpretable in physical space — nodule volume, lesion size, organ dimension — this loss is fatal.

The correct approach is to work with DICOM natively throughout the annotation pipeline, de-identify before annotation, apply appropriate windowing during annotation, and store results back in DICOM format. AI Taggers' radiology annotation service operates on DICOM natively across CT, MRI, X-ray, and ultrasound modalities, with HIPAA-compliant de-identification applied before any study reaches an annotator.

DICOM De-identification: What the 64 Tags Cover

DICOM PS3.15 Annex E specifies the de-identification action for each attribute in the standard DICOM data dictionary. Tags are categorised as 'D' (replace with dummy value), 'Z' (zero or replace), 'X' (remove), 'K' (keep), or 'C' (clean). The 64 'D' tags include the most sensitive identifiers: patient name (0010,0010), patient ID (0010,0020), patient birth date (0010,0030), accession number (0008,0050), institution name (0008,0080), and physician names.

In practice, a production-grade de-identification pipeline must go beyond the standard attribute dictionary. Acquisition systems from GE, Siemens, Philips, and Canon routinely embed PHI in private tags — vendor-specific attributes outside the standard DICOM namespace that the PS3.15 process does not address. A 2021 study examining de-identified DICOM datasets from five institutions found residual PHI in private tags in 34% of studies that had passed standard de-identification (Prior et al., Journal of Digital Imaging, 2021). Scanner-specific private tag dictionaries are required to achieve thorough de-identification.

For ultrasound and fluoroscopy, pixel-level PHI — patient name and accession number burned into the pixel array by the acquisition system — requires a separate de-identification step beyond DICOM header processing. See our guide on ultrasound annotation for the pixel-level de-identification workflow specific to that modality.

Windowing and Display: Why It Determines Annotation Accuracy

CT images store pixel values in Hounsfield Units (HU) — a continuous scale from approximately -1000 HU (air) to +3000 HU (dense bone). The human visual system can distinguish roughly 30–40 shades of grey simultaneously. To make clinically relevant detail visible, CT viewing software applies a window and level setting that maps a chosen HU range to the full grey scale. Outside that range, pixels render as either white or black.

Standard clinical presets exist for each anatomical region: lung parenchyma (W:1500, L:-600), mediastinum (W:350, L:40), bone (W:1500, L:300), and soft tissue (W:400, L:40). An annotator labelling a lung nodule with mediastinal windowing will miss or miscontour ground-glass components that are only visible in the lung window. A researcher who annotates the same dataset found that annotation errors attributable to incorrect windowing accounted for approximately 15% of documented mislabellings in publicly released radiology AI datasets (Oakden-Rayner, Lancet Digital Health, 2020).

Annotation guidelines for CT must specify the required window preset for each label class. For multi-task annotation projects that require labelling both lung and mediastinal structures on the same series, annotators must switch presets between label types — which requires explicit instruction and quality checks, since annotators working quickly default to a single window setting.

DICOM Output Formats: SR, SEG, and RT Struct

The choice of DICOM annotation output format should be determined by the annotation task and the downstream application, not by convenience or tool defaults. Three formats cover most clinical AI use cases.

Case Study: Multi-Site Lung CT Annotation on DICOM

A pulmonology AI company approached AI Taggers to annotate 28,000 chest CT studies sourced from 12 Australian and New Zealand clinical sites for a ground-glass opacity (GGO) and consolidation detection model. The client had previously run an annotation round using a JPEG-export workflow through a general-purpose annotation platform.

The problems with the initial annotation round were documented in the client's internal audit: JPEG exports from the PACS had been generated with mediastinal windowing applied uniformly across all studies. Ground-glass opacity — which is visible primarily in the lung window — was systematically under-detected. Three sites used legacy scanners that burned institution name and patient ID into the pixel array; these had not been de-identified before export. Annotations were stored as coordinate JSON files referencing JPEG filenames with no mapping back to DICOM study UIDs.

The reannotation workflow applied by AI Taggers:

1. Full DICOM de-identification using PS3.15 Annex E plus vendor-specific private tag removal for GE, Siemens, and Philips scanners represented in the dataset
2. Pixel-level PHI redaction for the three legacy scanner sites with burned-in identifiers
3. DICOM-native annotation with per-task window specification: lung window (W:1500, L:-600) for GGO and parenchymal labels, mediastinal window (W:350, L:40) for lymph node and vessel labels
4. Output stored as DICOM SEG objects, preserving physical coordinate system from source CT
5. FDA 21 CFR Part 11-compliant provenance logging: annotator ID, credential, annotation timestamp, adjudication records where dual-annotator disagreement exceeded 20% Dice overlap

Model performance comparison between the JPEG-export and DICOM-native annotation rounds:

The nodule volume MAE metric was not measurable from the JPEG-export round because JPEG exports had discarded the pixel spacing metadata needed to convert pixel areas to physical volumes. For related guidance on CT annotation see our detailed post on lung nodule CT annotation.

Multi-Site DICOM Projects: Handling Scanner Variability

Multi-site DICOM projects surface systematic differences between clinical sites that affect annotation accuracy. Scanner manufacturer, reconstruction kernel (sharp versus smooth), slice thickness, and acquisition voltage all change the appearance of the same pathology. A lung nodule on a GE scanner reconstructed with a bone kernel looks sharper with more apparent heterogeneity than the same nodule on a Siemens scanner with a soft kernel.

Before beginning annotation on a multi-site dataset, conduct a site characterisation pass: sample 50–100 studies from each site, document scanner manufacturer, model, reconstruction parameters, and note any site-specific artefacts. Common issues include: edge-enhancement artefacts from sharp kernels that mimic calcification, motion artefacts in cardiac-gated CT that create double-contour appearances, and beam-hardening artefacts near implants that occlude adjacent anatomy.

Window/level presets should be validated independently on a sample from each manufacturer. A lung window preset that works well on GE data may clip relevant signal on Siemens reconstructions with different HU calibration. The annotation brief should document approved window/level ranges per site, not assume universal transferability. For organ-level annotation, our organ segmentation guide covers multi-site consistency protocols in detail.

FDA 21 CFR Part 11 and DICOM Provenance

Medical AI intended for FDA submission requires that the annotation provenance record satisfies 21 CFR Part 11 — the electronic records and electronic signatures regulation. For DICOM annotation workflows, this means the annotation record must capture: the de-identification method applied to the source study, the annotator identity linked to a verified credential, the annotation tool and version, the display windowing applied during annotation, and any adjudication decisions with rationale.

DICOM SR and DICOM SEG objects can carry provenance metadata in their structured headers — the 'Content Creator Name' and 'Content Creator Identification Code Sequence' attributes record annotator identity. Annotation platforms that export DICOM SEG or SR should be configured to populate these fields from the annotator's identity record, not leave them blank.

HIPAA and 21 CFR Part 11 interact in DICOM workflows: the de-identified DICOM study must remain linkable to the original study via a secure mapping table for re-identification in audit scenarios, while the annotation record must carry a pseudonymous study identifier that allows FDA reviewers to trace any annotation back to the source image and annotator. This mapping table is itself regulated under 21 CFR Part 11. Teams building medical AI for regulatory submission should document their DICOM de-identification and provenance workflow before annotation begins, not after — retroactive compliance documentation is the most common cause of annotation rework in pre-submission review cycles.

AI Taggers' medical imaging annotation service includes full DICOM-native workflows with FDA 21 CFR Part 11-compliant provenance logging. For the broader HIPAA de-identification requirements across clinical data types, see our post on HIPAA de-identification for medical AI.

Frequently Asked Questions