165
Software as a Medical Device: Regulatory and Market Access Implications
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
ity accepts the ‘new’ device’s technical documen-
tation containing just a reference to the master
file of the original device. That master file then
has to be made available by the original manufac-
turer to the notified body. Note: FDA provides
the possibility of using master files. Depending
on the technology used, the second manufacturer
or health institution operating under this sce-
nario may need to implement technical measures,
a quality agreement, and monitoring of original
device updates to ensure the new device is safe
and performant in light of state-of-the-art.
Change Boundaries and Algorithm
Change Protocol
The manufacturer must specify a change envelope
or change boundaries and an ACP. As long as the
device operates within the related change bound-
aries, its safety and performance are guaranteed
(e.g., minimum specificity and sensitivity). The
manufacturer can ensure this through procedural
(e.g., acceptance testing) or technical measures.
How change envelopes can be defined depends
on the technology used. The manufacturer should
demonstrate why the chosen parameters and
threshold values are valid in consideration of the
clinical state of the art and the intended purpose
of the device. Currently, no standard is available
that guides the drafting of an ACP.21
Manufacturers can increase the trust neces-
sary for implementation by informing regulators
and users about the following:
• How the AI learns over time
• What the allowed change envelope/
boundaries are of the AI
• What caused a significant change in the AI
behavior
• How the performance and safety is being
assured as it adapts
• How quality control of new training data is
assured
• What triggers algorithm change
• What performance and confidence levels22
are during a given timeframe
• Whether and how a user can reject an
algorithm change or roll-back to a previous state
For troubleshooting purposes, a manufacturer
also may want to monitor actual performance
and change/drift of an evolving algorithm to
detect performance deficits, e.g., by implement-
ing traceability through an audit trail.23
Figure 11-5. Visualization of Postmarket Significant Change
(4a) A manufacturer develops an AI-based device and
places it on the market.
The user intentionally changes the local model in a way not
allowed by the manufacturer’s change control plan, either
by making the change
(1) beyond the manufacturer’s pre-determined change
envelope (a.k.a. change boundaries, pre-specifications)
(2) while not complying with the manufacturer’s Algorithm
Change Protocol
Or by using the device as a component for a new device.
(4b) If the user places the transformed device on the
market, then the user is subject to manufacturer require-
ments of the medical device legislation, with the original
manufacturer its supplier.
(4c) If the user is an EU health institution having
transformed the device for in-house use purposes only,
then the health institution is an in-house manufacturer
subject to EU MDR/IVDR Art. 5(5).
(4d) If the user does not place the device on the market,
is not a health institution that changes the device for
in-house use-only, then the user is misusing the device and
not subject to medical device regulation.
user-initiated
significant
change
through learning
(c) put into service
by and within (EU) health institution
locked
(d) placed on the market and used by regular user
backend
(b) placed on the market (a) placed on the market
change
through learning
Set
operating
point change
change
through
learning
user-initiated
significant
change
through learning
locked
regular user
change
through
learning
Set
operating
point change
ASSESS
CURATE APPLY
Continuous
Change
through
learning
Manufacturer
User
Proceed?
© Koen Cobbaert 2021
Software as a Medical Device: Regulatory and Market Access Implications
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
ity accepts the ‘new’ device’s technical documen-
tation containing just a reference to the master
file of the original device. That master file then
has to be made available by the original manufac-
turer to the notified body. Note: FDA provides
the possibility of using master files. Depending
on the technology used, the second manufacturer
or health institution operating under this sce-
nario may need to implement technical measures,
a quality agreement, and monitoring of original
device updates to ensure the new device is safe
and performant in light of state-of-the-art.
Change Boundaries and Algorithm
Change Protocol
The manufacturer must specify a change envelope
or change boundaries and an ACP. As long as the
device operates within the related change bound-
aries, its safety and performance are guaranteed
(e.g., minimum specificity and sensitivity). The
manufacturer can ensure this through procedural
(e.g., acceptance testing) or technical measures.
How change envelopes can be defined depends
on the technology used. The manufacturer should
demonstrate why the chosen parameters and
threshold values are valid in consideration of the
clinical state of the art and the intended purpose
of the device. Currently, no standard is available
that guides the drafting of an ACP.21
Manufacturers can increase the trust neces-
sary for implementation by informing regulators
and users about the following:
• How the AI learns over time
• What the allowed change envelope/
boundaries are of the AI
• What caused a significant change in the AI
behavior
• How the performance and safety is being
assured as it adapts
• How quality control of new training data is
assured
• What triggers algorithm change
• What performance and confidence levels22
are during a given timeframe
• Whether and how a user can reject an
algorithm change or roll-back to a previous state
For troubleshooting purposes, a manufacturer
also may want to monitor actual performance
and change/drift of an evolving algorithm to
detect performance deficits, e.g., by implement-
ing traceability through an audit trail.23
Figure 11-5. Visualization of Postmarket Significant Change
(4a) A manufacturer develops an AI-based device and
places it on the market.
The user intentionally changes the local model in a way not
allowed by the manufacturer’s change control plan, either
by making the change
(1) beyond the manufacturer’s pre-determined change
envelope (a.k.a. change boundaries, pre-specifications)
(2) while not complying with the manufacturer’s Algorithm
Change Protocol
Or by using the device as a component for a new device.
(4b) If the user places the transformed device on the
market, then the user is subject to manufacturer require-
ments of the medical device legislation, with the original
manufacturer its supplier.
(4c) If the user is an EU health institution having
transformed the device for in-house use purposes only,
then the health institution is an in-house manufacturer
subject to EU MDR/IVDR Art. 5(5).
(4d) If the user does not place the device on the market,
is not a health institution that changes the device for
in-house use-only, then the user is misusing the device and
not subject to medical device regulation.
user-initiated
significant
change
through learning
(c) put into service
by and within (EU) health institution
locked
(d) placed on the market and used by regular user
backend
(b) placed on the market (a) placed on the market
change
through learning
Set
operating
point change
change
through
learning
user-initiated
significant
change
through learning
locked
regular user
change
through
learning
Set
operating
point change
ASSESS
CURATE APPLY
Continuous
Change
through
learning
Manufacturer
User
Proceed?
© Koen Cobbaert 2021