Chapter 8. Artificial Intelligence-based Software
122 Regulatory Affairs Professionals Society (RAPS)
It is recommended that manufacturers review
the American Association of Physicists in
Medicine TG Report 27341 Clinical Performance
Assessment: Considerations for Computer-
Assisted Detection Devices Applied to
Radiology Images and Radiology Device Data
in Premarket Notification (510(k)) Submissions
and Software as a Medical Device (SAMD):
Clinical Evaluation.18
CADt (21 CFR §892.2080): Radiological
CADt and Notification Software
Triage devices assist healthcare practitioners to
quickly identify and prioritize patients based on
the severity of their conditions. These devices
allow patients to receive appropriate care in a
timely manner. Some examples of cleared triage
devices include BriefCase (K214043),48 Annalise
Enterprise CXR Triage Trauma (K222268)49
and Syngo.CT Brain Hemorrhage (K203260).50
Current evaluation methods for AI/ML-based
triage devices mainly include standalone software
testing (e.g., including measurement of sensitiv-
ity, specificity, AUC, and time-to-notification). It
is very important that the manufacturer reiterate
in their submission that triage devices are only
for flagging and prioritizing patients and are not
intended for disease detections and/or diagnosis.
Additional CAD Regulations
The discussion here has been limited to radiology
image processing systems along with CADe,
CADx, and CADt devices. Table 8-2 gives
examples of key performance metrics evaluated
by the FDA during the clearance of recent CAD
devices. 38,42,47-49 There are additional regulations
pertaining to CAD that are outside the scope of
this chapter that developers should be aware of,
namely:
• The use of dual detection and diagnosis
software (CADe/x) for lesions suspicious for
cancer regulated under 21 CFR §892.2090
• Radiological Image Acquisition and/
or Optimization Guided by Artificial
Intelligence (CADa/o) regulated under 21
CFR §892.2100 and
• Radiological Machine Learning Based
Quantitative Imaging Software with
Change Control Plan regulated under 21
CFR §892.2050 and image reconstruction
algorithms.
The line between quantitative imaging and
CAD devices may occasionally have some
ambiguity. They both provide advanced image
analysis techniques for healthcare profession-
als. Quantitative imaging can go beyond the
anatomy into the molecular level whereas CAD
essentially relies mainly on the measurements
from medical images For example, the FDA
guidance Technical Performance Assessment of
Quantitative Imaging in Radiological Device
Premarket Submissions,51 states that while the
Agency would consider “a function that reports a
percent stenosis value from the ratio of two vessel
diameters… a quantitative imaging function,” a
device “linking the probability of a cardiac event
to the percentage of vessel stenosis would be
outside the scope of [the] guidance document,”
presumably because such a device falls outside
the boundary of what FDA considers simple
“quantitative imaging.”
To ensure a smooth regulatory process, it
is recommended that device manufacturers pay
close attention to the importance of identifying
the appropriate regulation for their device, with
consultation to the FDA via the Q submission
process as necessary to determine whether the
Agency considers the device in question a CAD.
Per the special controls, the FDA has different
expectations of performance data for CAD
devices than for simpler measurement tools that
do not provide detection or diagnosis informa-
tion. In particular, CAD devices often require
submission of a clinical performance assessment,
whereas technical performance assessment may
suffice for simple quantitative imaging or mea-
surement devices that do not perform detection
or diagnosis. Submitting without any clinical
performance assessment, may result in regula-
tory delays or an immediate Not Substantially
Equivalent” (NSE) determination.52
122 Regulatory Affairs Professionals Society (RAPS)
It is recommended that manufacturers review
the American Association of Physicists in
Medicine TG Report 27341 Clinical Performance
Assessment: Considerations for Computer-
Assisted Detection Devices Applied to
Radiology Images and Radiology Device Data
in Premarket Notification (510(k)) Submissions
and Software as a Medical Device (SAMD):
Clinical Evaluation.18
CADt (21 CFR §892.2080): Radiological
CADt and Notification Software
Triage devices assist healthcare practitioners to
quickly identify and prioritize patients based on
the severity of their conditions. These devices
allow patients to receive appropriate care in a
timely manner. Some examples of cleared triage
devices include BriefCase (K214043),48 Annalise
Enterprise CXR Triage Trauma (K222268)49
and Syngo.CT Brain Hemorrhage (K203260).50
Current evaluation methods for AI/ML-based
triage devices mainly include standalone software
testing (e.g., including measurement of sensitiv-
ity, specificity, AUC, and time-to-notification). It
is very important that the manufacturer reiterate
in their submission that triage devices are only
for flagging and prioritizing patients and are not
intended for disease detections and/or diagnosis.
Additional CAD Regulations
The discussion here has been limited to radiology
image processing systems along with CADe,
CADx, and CADt devices. Table 8-2 gives
examples of key performance metrics evaluated
by the FDA during the clearance of recent CAD
devices. 38,42,47-49 There are additional regulations
pertaining to CAD that are outside the scope of
this chapter that developers should be aware of,
namely:
• The use of dual detection and diagnosis
software (CADe/x) for lesions suspicious for
cancer regulated under 21 CFR §892.2090
• Radiological Image Acquisition and/
or Optimization Guided by Artificial
Intelligence (CADa/o) regulated under 21
CFR §892.2100 and
• Radiological Machine Learning Based
Quantitative Imaging Software with
Change Control Plan regulated under 21
CFR §892.2050 and image reconstruction
algorithms.
The line between quantitative imaging and
CAD devices may occasionally have some
ambiguity. They both provide advanced image
analysis techniques for healthcare profession-
als. Quantitative imaging can go beyond the
anatomy into the molecular level whereas CAD
essentially relies mainly on the measurements
from medical images For example, the FDA
guidance Technical Performance Assessment of
Quantitative Imaging in Radiological Device
Premarket Submissions,51 states that while the
Agency would consider “a function that reports a
percent stenosis value from the ratio of two vessel
diameters… a quantitative imaging function,” a
device “linking the probability of a cardiac event
to the percentage of vessel stenosis would be
outside the scope of [the] guidance document,”
presumably because such a device falls outside
the boundary of what FDA considers simple
“quantitative imaging.”
To ensure a smooth regulatory process, it
is recommended that device manufacturers pay
close attention to the importance of identifying
the appropriate regulation for their device, with
consultation to the FDA via the Q submission
process as necessary to determine whether the
Agency considers the device in question a CAD.
Per the special controls, the FDA has different
expectations of performance data for CAD
devices than for simpler measurement tools that
do not provide detection or diagnosis informa-
tion. In particular, CAD devices often require
submission of a clinical performance assessment,
whereas technical performance assessment may
suffice for simple quantitative imaging or mea-
surement devices that do not perform detection
or diagnosis. Submitting without any clinical
performance assessment, may result in regula-
tory delays or an immediate Not Substantially
Equivalent” (NSE) determination.52