2 HUMAN IN THE CORE
Most industries can answer Amodei's implicit question about readiness with a
decisive and honest not yet. We cannot. For those of us inside regulated indus-
tries, that sentence lands differently. Our answer has to be yes.
We have spent careers in industries whose entire premise is that powerful
tools, applied without maturity, hurt people. The artifacts differ (drugs and de-
vices in my world, complex instruments in finance, airframes in aviation, reac-
tors in nuclear), but the architectural answer converges. We do not invent the
maturity to wield these things alongside their invention. We build it, slowly and
painfully, in the architecture of how we work: design controls and validated
systems, audit trails and ongoing surveillance, change control, escalation
paths, named accountable humans. A regulated enterprise is, more than any-
thing else, a system for not letting powerful things hurt people without our un-
derstanding why.
Now we are being handed something more powerful than anything in our his-
tory, and it does not arrive as a molecule, an aircraft, or a financial instrument.
It arrives as cognition. It thinks alongside us. It writes our procedures and drafts
our reports. It classifies the exceptions and suggests the responses. It is every-
where in the workflow at once, and every place it appears is a place where a sin-
gle decision lands on a person who did not make it.
Every executive over forty has lived through several transformations. The inter-
net. The mobile interface. The data lake. The shift from on-premise to SaaS and
Cloud. Each felt disorienting in its moment. Viewed in hindsight, each was a
substrate change: a shift in where the work runs, how it connects, what it costs,
and who maintains it, but not a shift in the work itself. A clinical trial in 2010,
run on paper case report forms and fax, is recognizable as the same activity as a
clinical trial in 2020 run on a cloud-hosted electronic data capture system. Only
the floor under it changed.
A I I S N O T A S U B S T R A T E C H A N G E
Artificial intelligence is not a substrate change. It is a cognition change, and that
is a different category of event entirely. The floor under the work was already
shifting in ways we had governance for. The work itself is now shifting in ways
we do not.
When you put cognition into a workflow, three things happen at once that no
prior transformation produced.
The locus of decisions moves. Choices that previously sat in the hands of
a particular reviewer (the medical monitor reviewing an adverse event,
the quality assurance (QA) reviewer dispositioning a deviation, the
Most industries can answer Amodei's implicit question about readiness with a
decisive and honest not yet. We cannot. For those of us inside regulated indus-
tries, that sentence lands differently. Our answer has to be yes.
We have spent careers in industries whose entire premise is that powerful
tools, applied without maturity, hurt people. The artifacts differ (drugs and de-
vices in my world, complex instruments in finance, airframes in aviation, reac-
tors in nuclear), but the architectural answer converges. We do not invent the
maturity to wield these things alongside their invention. We build it, slowly and
painfully, in the architecture of how we work: design controls and validated
systems, audit trails and ongoing surveillance, change control, escalation
paths, named accountable humans. A regulated enterprise is, more than any-
thing else, a system for not letting powerful things hurt people without our un-
derstanding why.
Now we are being handed something more powerful than anything in our his-
tory, and it does not arrive as a molecule, an aircraft, or a financial instrument.
It arrives as cognition. It thinks alongside us. It writes our procedures and drafts
our reports. It classifies the exceptions and suggests the responses. It is every-
where in the workflow at once, and every place it appears is a place where a sin-
gle decision lands on a person who did not make it.
Every executive over forty has lived through several transformations. The inter-
net. The mobile interface. The data lake. The shift from on-premise to SaaS and
Cloud. Each felt disorienting in its moment. Viewed in hindsight, each was a
substrate change: a shift in where the work runs, how it connects, what it costs,
and who maintains it, but not a shift in the work itself. A clinical trial in 2010,
run on paper case report forms and fax, is recognizable as the same activity as a
clinical trial in 2020 run on a cloud-hosted electronic data capture system. Only
the floor under it changed.
A I I S N O T A S U B S T R A T E C H A N G E
Artificial intelligence is not a substrate change. It is a cognition change, and that
is a different category of event entirely. The floor under the work was already
shifting in ways we had governance for. The work itself is now shifting in ways
we do not.
When you put cognition into a workflow, three things happen at once that no
prior transformation produced.
The locus of decisions moves. Choices that previously sat in the hands of
a particular reviewer (the medical monitor reviewing an adverse event,
the quality assurance (QA) reviewer dispositioning a deviation, the