From X-Rays to AI: Navigating US Regulations in Radiological Health
117
Regulation of SaMDs Incorporating
AI/ML
While the existing regulatory framework for
traditional medical devices is still applicable
to SaMD, the FDA is looking at innovative
approaches to regulate SaMD. In recent years the
Agency has issued guidance documents, white
papers, and public workshops to clarify their
current thinking on these technologies in order
to promote public health and safety.
During the concept phase of developing AI/
ML technologies, developers should first deter-
mine whether their software functions fall under
the FDA’s definition of a medical device. The
FDA’s Digital Health Policy Navigator (hereon
referred to as Navigator) provides a convenient
and efficient framework for developers to assess
their software’s functions.34 This interactive
seven-step process guides developers through the
most relevant FDA medical device regulatory
guidance documents and policies that may apply
to their software product. After determination
that the software function is in fact a medical
device, developers should look into all applicable
regulations for the subject device. Brief intro-
ductions to some examples of AI/ML software
devices follow.
Medical Image Management and
Processing Systems – 21 CFR §892.2050
Medical image management and processing sys-
tems (MIMPS) were previously known as picture
archiving and communications system (PACS).
PACS are ubiquitous in hospitals and these sys-
tems were intended to securely store, access and
process medical images in healthcare facilities.35
The 21st Century Cures Act amended the defi-
nition and identification description of a medical
device in the Food Drug &Cosmetic Act to
exclude certain functions.36 In accordance with this
change, the FDA amended 21 CFR §892.2050 on
19 April 2021, altering the title of the classification
Term Abbreviation Definition
Tuning data – A set of data used by manufacturers to evaluate a small number of trained
ML functions to explore different architecture or hyperparameters
(parameters used to control the learning process)19
Testing data – The data used to characterize the performance of the ML model. An
independent set of data that is never shown to the AI training algorithm
during training and is used to estimate the actual ML model performance
after training20
Verification – The confirmation that a system was built correctly and fulfils the spec-
ified requirements.17 Validation is the confirmation, through objective
evidence, that the requirements for a specific intended use have been
fulfilled
Validation – The confirmation, through objective evidence, that the requirements for a
specific intended use have been fulfilled17
Reference standard
(‘Gold standard’ or
‘ground truth’)
– An objectively determined benchmark that provides the expected result
for comparison, assessment, training, etc.21 In computer aided-detection
applications, a reference standard indicates whether a disease, condition,
abnormality, or all, are present, and if so, may include such attributes as
its extent or location22
Ground truthing – The characterization of the reference standard for a patient (disease
status) is known as the truthing process20
Locked algorithm – An algorithm that provides the same result each time the same input is
applied to it and does not change
Continuous learning
(adaptive)
– Incremental training of an AI system that takes place on an ongoing basis
during the operation19
Table 8-1. Terms, Abbreviations and Definitions Related to AI and ML Technologies (cont.)
117
Regulation of SaMDs Incorporating
AI/ML
While the existing regulatory framework for
traditional medical devices is still applicable
to SaMD, the FDA is looking at innovative
approaches to regulate SaMD. In recent years the
Agency has issued guidance documents, white
papers, and public workshops to clarify their
current thinking on these technologies in order
to promote public health and safety.
During the concept phase of developing AI/
ML technologies, developers should first deter-
mine whether their software functions fall under
the FDA’s definition of a medical device. The
FDA’s Digital Health Policy Navigator (hereon
referred to as Navigator) provides a convenient
and efficient framework for developers to assess
their software’s functions.34 This interactive
seven-step process guides developers through the
most relevant FDA medical device regulatory
guidance documents and policies that may apply
to their software product. After determination
that the software function is in fact a medical
device, developers should look into all applicable
regulations for the subject device. Brief intro-
ductions to some examples of AI/ML software
devices follow.
Medical Image Management and
Processing Systems – 21 CFR §892.2050
Medical image management and processing sys-
tems (MIMPS) were previously known as picture
archiving and communications system (PACS).
PACS are ubiquitous in hospitals and these sys-
tems were intended to securely store, access and
process medical images in healthcare facilities.35
The 21st Century Cures Act amended the defi-
nition and identification description of a medical
device in the Food Drug &Cosmetic Act to
exclude certain functions.36 In accordance with this
change, the FDA amended 21 CFR §892.2050 on
19 April 2021, altering the title of the classification
Term Abbreviation Definition
Tuning data – A set of data used by manufacturers to evaluate a small number of trained
ML functions to explore different architecture or hyperparameters
(parameters used to control the learning process)19
Testing data – The data used to characterize the performance of the ML model. An
independent set of data that is never shown to the AI training algorithm
during training and is used to estimate the actual ML model performance
after training20
Verification – The confirmation that a system was built correctly and fulfils the spec-
ified requirements.17 Validation is the confirmation, through objective
evidence, that the requirements for a specific intended use have been
fulfilled
Validation – The confirmation, through objective evidence, that the requirements for a
specific intended use have been fulfilled17
Reference standard
(‘Gold standard’ or
‘ground truth’)
– An objectively determined benchmark that provides the expected result
for comparison, assessment, training, etc.21 In computer aided-detection
applications, a reference standard indicates whether a disease, condition,
abnormality, or all, are present, and if so, may include such attributes as
its extent or location22
Ground truthing – The characterization of the reference standard for a patient (disease
status) is known as the truthing process20
Locked algorithm – An algorithm that provides the same result each time the same input is
applied to it and does not change
Continuous learning
(adaptive)
– Incremental training of an AI system that takes place on an ongoing basis
during the operation19
Table 8-1. Terms, Abbreviations and Definitions Related to AI and ML Technologies (cont.)