From X-Rays to AI: Navigating US Regulations in Radiological Health
123
Table 8-2. Examples of Key Performance Metrics Evaluated by the FDA During the Clearance of
recent Computer-Assisted Detection (CADe) Devices
Device Name Premarket
Clearance
#(Class II)
Summarized Indications Imaging
Modality
Test
Cases (n)
Standalone Performance Metrics
BriefCase48 K214043 Triage device intended
for flagging and
communication of
suspect positive
pneumothorax findings
CXR 619 (139
positive
480
negative)
Sensitivity: 94.2% (95% CI: 89.9%, 97.8%)
Specificity: 90.8% (95% CI: 88.1%, 93.1%)
Area under the curve (AUC): 0.969 (95%
CI: 0.954, 0.985)
Time to Notification: 13.1 seconds
Annalise
Enterprise
CXR Triage
Trauma49
K222268 Software workflow tool
designed to aid the clinical
assessment of adult chest
x-ray cases with features
suggestive of vertebral
compression fracture in
the medical care
CXR 589
(272
positive
317
negative)
Sensitivity: 89.3% (95% CI :85.7%, 93.0%)a
Specificity: 89.0% (95% CI :85.8%, 92.1%)a
AUC: AUC: 0.954 (95% CI: 0.939-0.968)
Time to Notification:
30 seconds
Chest-CAD42 K210666 CADe software
that analyzes chest
radiographs to identify,
categorize and highlight
suspicious regions of
interest
CXR 20,000 Sensitivity: 90.8% (95% CI: 90.5%, 91.1%)
Specificity: 88.7% (95% CI: 88.5%, 88.9%)
AUC of receiver operating characteristic:
0.976% (0.975-0.976, 95% CI)
Other:
Clinical reader study: multiple-reader,
multiple-case (MRMC) cross-over study.
The study evaluated the performance of
24 clinical readers evaluating 238 cases to
determine whether the accuracy of readers
aided by Chest-CAD (“Aided”) was superior
to the accuracy of readers when unaided
by Chest-CAD (“Unaided”) by the case-
level, across-category aggregate
Brainomix
360
e-ASPECTS47
K221564 A computer-assisted
diagnosis (CADx)
software device used
to assist the clinician
in the assessment and
characterization of brain
tissue abnormalities using
CT image data
CT 256 non-
contrast
CT scans
Sensitivity: 68.0% (95% CI: 57.0%, 72.0%)
Specificity: 97% (95% CI: 86.0%, 98.0%)
AUC: 83.0% (95% CI: 81.0%, 85.0%I)
Digital phantom testing:
Other:
Clinical reader study: MRMC cross-
over study. The study evaluated the
performance of 10 clinical readers
evaluating 54 retrospective scans with
and without the aid of Brainomix 360
e-ASPECTS
AI-Rad
Companion38
K213713 Image-processing
software that provides
quantitative and
qualitative analysis from
previously acquired
CT DICOM images.
Supports radiologists
and physicians from
emergency medicine,
specialty care, urgent
care, and general
practice in the evaluation
and assessment of
disease of the lungs
CT 200 Sensitivity: 94.3%
Average positive predictive value: 99.1%
AUC, area under the receiver operative characteristic curve CAD, computer-assisted device CADx, computer-assisted diagnosis device
CADe, computer-assisted detection device, CT, computed tomography CXR, chest x-ray DICOM, Digital Imaging and Communications in
Medicine MRMC, multiple-reader, multiple-case
a Vertebral compression fraction at operating point 0.3849.
123
Table 8-2. Examples of Key Performance Metrics Evaluated by the FDA During the Clearance of
recent Computer-Assisted Detection (CADe) Devices
Device Name Premarket
Clearance
#(Class II)
Summarized Indications Imaging
Modality
Test
Cases (n)
Standalone Performance Metrics
BriefCase48 K214043 Triage device intended
for flagging and
communication of
suspect positive
pneumothorax findings
CXR 619 (139
positive
480
negative)
Sensitivity: 94.2% (95% CI: 89.9%, 97.8%)
Specificity: 90.8% (95% CI: 88.1%, 93.1%)
Area under the curve (AUC): 0.969 (95%
CI: 0.954, 0.985)
Time to Notification: 13.1 seconds
Annalise
Enterprise
CXR Triage
Trauma49
K222268 Software workflow tool
designed to aid the clinical
assessment of adult chest
x-ray cases with features
suggestive of vertebral
compression fracture in
the medical care
CXR 589
(272
positive
317
negative)
Sensitivity: 89.3% (95% CI :85.7%, 93.0%)a
Specificity: 89.0% (95% CI :85.8%, 92.1%)a
AUC: AUC: 0.954 (95% CI: 0.939-0.968)
Time to Notification:
30 seconds
Chest-CAD42 K210666 CADe software
that analyzes chest
radiographs to identify,
categorize and highlight
suspicious regions of
interest
CXR 20,000 Sensitivity: 90.8% (95% CI: 90.5%, 91.1%)
Specificity: 88.7% (95% CI: 88.5%, 88.9%)
AUC of receiver operating characteristic:
0.976% (0.975-0.976, 95% CI)
Other:
Clinical reader study: multiple-reader,
multiple-case (MRMC) cross-over study.
The study evaluated the performance of
24 clinical readers evaluating 238 cases to
determine whether the accuracy of readers
aided by Chest-CAD (“Aided”) was superior
to the accuracy of readers when unaided
by Chest-CAD (“Unaided”) by the case-
level, across-category aggregate
Brainomix
360
e-ASPECTS47
K221564 A computer-assisted
diagnosis (CADx)
software device used
to assist the clinician
in the assessment and
characterization of brain
tissue abnormalities using
CT image data
CT 256 non-
contrast
CT scans
Sensitivity: 68.0% (95% CI: 57.0%, 72.0%)
Specificity: 97% (95% CI: 86.0%, 98.0%)
AUC: 83.0% (95% CI: 81.0%, 85.0%I)
Digital phantom testing:
Other:
Clinical reader study: MRMC cross-
over study. The study evaluated the
performance of 10 clinical readers
evaluating 54 retrospective scans with
and without the aid of Brainomix 360
e-ASPECTS
AI-Rad
Companion38
K213713 Image-processing
software that provides
quantitative and
qualitative analysis from
previously acquired
CT DICOM images.
Supports radiologists
and physicians from
emergency medicine,
specialty care, urgent
care, and general
practice in the evaluation
and assessment of
disease of the lungs
CT 200 Sensitivity: 94.3%
Average positive predictive value: 99.1%
AUC, area under the receiver operative characteristic curve CAD, computer-assisted device CADx, computer-assisted diagnosis device
CADe, computer-assisted detection device, CT, computed tomography CXR, chest x-ray DICOM, Digital Imaging and Communications in
Medicine MRMC, multiple-reader, multiple-case
a Vertebral compression fraction at operating point 0.3849.