163
Software as a Medical Device: Regulatory and Market Access Implications
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
on the operating curve that is different from the
factory default, e.g., to trade-off increased sensi-
tivity with reduced specificity.
AI can change during runtime in various
other ways. In the second scenario (continuous
change), the model continues to learn within
the manufacturer’s boundaries. Consider, for
example, an AI for precision medicine. The AI
calculates the optimal medication dose to reduce
the tremor of a patient with Parkinson’s disease
while limiting the drug’s side effects. As the
disease progresses over months or years, symp-
toms change. The algorithm continues to evolve
together with the patient’s disease state. In this
scenario, the user may still be able to change the
working point on the operating curve. A patient
may, for example, on a particular day, decide to
pick a point on the operating curve that lowers
the dose, allowing for more tremor, but improv-
ing their cognitive performance. A patient may
prefer a different operating point because the
medication causes mild cognitive impairments,
such as distraction, disorganization, difficulty
planning, and accomplishing tasks.
In the third scenario (discrete changes), the
learning initially occurs under the manufacturer’s
control. The model is then handed over to the
user (or another party) for further calibration or
adjustment to the local context or to a specific
patient. The change occurs within the intended
Figure 11-4. Three Scenarios for Change of Machine Learning Devices in Relation to Their
Placement on the Market
placed on the market pre-market
Manufacturer
backend
health
use
Health Environment
1
ASSESS
CURATE
APPLY
Manufacturer
Release
Locked
The manufacturer analyzes and controls model changes
and decides on updates for release to the market. The
global model, i.e. the model at the manufacturer site,
may learn ‘offline’ from real-world data on the market.
The local model does not change during use, but the user
can optionally select the appropriate working point
locked change
through learning
Set
operating
point change
2
ASSESS
CURATE APPLY
Continuous
Change
through
learning
Learns in the field
The local model is updated without explicit
manufacturer or user interaction
Optionally the user can select the appropriate working
point during health or clinical use.
change
through learning
change
through learning
Set
operating
point change
The local model learns ‘offline’ from real-world data
generated through health or clinical use. A human,
such as a healthcare professional, service engineer or
patient, analyzes and controls the changes to the local
model ‘in the backend’, before putting a new state of
the local model into health or clinical use, returning to
a previous state or resetting it to the factory defaults.
This kind of change is for example used to calibrate
the model to changing data inputs or clinical practices
at the user site. The local model changes in the
‘backend’, but uses a ‘locked’ state in health or clinical
use. Optionally the user can select the appropriate
working point during health or clinical use.
change
through learning
backend health
use
locked change
through learning 3
Set
operating
point change
ASSESS
CURATE APPLY
Continuous
Change
through
learning
Manufacturer
User
Proceed?
© Koen Cobbaert 2021
Software as a Medical Device: Regulatory and Market Access Implications
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
on the operating curve that is different from the
factory default, e.g., to trade-off increased sensi-
tivity with reduced specificity.
AI can change during runtime in various
other ways. In the second scenario (continuous
change), the model continues to learn within
the manufacturer’s boundaries. Consider, for
example, an AI for precision medicine. The AI
calculates the optimal medication dose to reduce
the tremor of a patient with Parkinson’s disease
while limiting the drug’s side effects. As the
disease progresses over months or years, symp-
toms change. The algorithm continues to evolve
together with the patient’s disease state. In this
scenario, the user may still be able to change the
working point on the operating curve. A patient
may, for example, on a particular day, decide to
pick a point on the operating curve that lowers
the dose, allowing for more tremor, but improv-
ing their cognitive performance. A patient may
prefer a different operating point because the
medication causes mild cognitive impairments,
such as distraction, disorganization, difficulty
planning, and accomplishing tasks.
In the third scenario (discrete changes), the
learning initially occurs under the manufacturer’s
control. The model is then handed over to the
user (or another party) for further calibration or
adjustment to the local context or to a specific
patient. The change occurs within the intended
Figure 11-4. Three Scenarios for Change of Machine Learning Devices in Relation to Their
Placement on the Market
placed on the market pre-market
Manufacturer
backend
health
use
Health Environment
1
ASSESS
CURATE
APPLY
Manufacturer
Release
Locked
The manufacturer analyzes and controls model changes
and decides on updates for release to the market. The
global model, i.e. the model at the manufacturer site,
may learn ‘offline’ from real-world data on the market.
The local model does not change during use, but the user
can optionally select the appropriate working point
locked change
through learning
Set
operating
point change
2
ASSESS
CURATE APPLY
Continuous
Change
through
learning
Learns in the field
The local model is updated without explicit
manufacturer or user interaction
Optionally the user can select the appropriate working
point during health or clinical use.
change
through learning
change
through learning
Set
operating
point change
The local model learns ‘offline’ from real-world data
generated through health or clinical use. A human,
such as a healthcare professional, service engineer or
patient, analyzes and controls the changes to the local
model ‘in the backend’, before putting a new state of
the local model into health or clinical use, returning to
a previous state or resetting it to the factory defaults.
This kind of change is for example used to calibrate
the model to changing data inputs or clinical practices
at the user site. The local model changes in the
‘backend’, but uses a ‘locked’ state in health or clinical
use. Optionally the user can select the appropriate
working point during health or clinical use.
change
through learning
backend health
use
locked change
through learning 3
Set
operating
point change
ASSESS
CURATE APPLY
Continuous
Change
through
learning
Manufacturer
User
Proceed?
© Koen Cobbaert 2021