CH 1 · CHALLENGE OF THE AGES 1
F O U N D A T I O N 1
Challenge of the Ages
Of all the transformations a CxO has led, this one is different.
1.1 Transformation Like No Other
On April 2, 2026, the U.S. Food and Drug Administration (FDA) issued the first
warning letter in its drug CGMP (current good manufacturing practice) history
with a section titled “Inappropriate Use of Artificial Intelligence.” The firm, a
homeopathic over-the-counter drug manufacturer, had used AI agents to draft
drug specifications, procedures, and master production records, and had dis-
tributed product without conducting required process validation. Asked why,
the firm told investigators the AI agent had not informed them the requirement
existed. The agency rejected the posture, cited U.S. 21 CFR 211.22(c), and made
clear that any AI output used in CGMP activities must be reviewed and cleared
by an authorized human representative of the firm's quality unit. The company
has since ceased drug production.
This is the first warning letter of its kind. It will not be the last. The architec-
tural commitments every regulated enterprise has to make from this point for-
ward must answer the question that quality unit presumably could not: when
AI produced this, who in this firm knew enough to recognize that it was wrong?
In a 2025 essay, Anthropic CEO Dario Amodei wrote a sentence that has stayed
with me longer than most things I have read about artificial intelligence.
“Humanity is about to be handed almost unimaginable power,
and it is deeply unclear whether our social, political, and
technological systems possess the maturity to wield it.”
DARIO AMODEI · ANTHROPIC CEO · 2025
F O U N D A T I O N 1
Challenge of the Ages
Of all the transformations a CxO has led, this one is different.
1.1 Transformation Like No Other
On April 2, 2026, the U.S. Food and Drug Administration (FDA) issued the first
warning letter in its drug CGMP (current good manufacturing practice) history
with a section titled “Inappropriate Use of Artificial Intelligence.” The firm, a
homeopathic over-the-counter drug manufacturer, had used AI agents to draft
drug specifications, procedures, and master production records, and had dis-
tributed product without conducting required process validation. Asked why,
the firm told investigators the AI agent had not informed them the requirement
existed. The agency rejected the posture, cited U.S. 21 CFR 211.22(c), and made
clear that any AI output used in CGMP activities must be reviewed and cleared
by an authorized human representative of the firm's quality unit. The company
has since ceased drug production.
This is the first warning letter of its kind. It will not be the last. The architec-
tural commitments every regulated enterprise has to make from this point for-
ward must answer the question that quality unit presumably could not: when
AI produced this, who in this firm knew enough to recognize that it was wrong?
In a 2025 essay, Anthropic CEO Dario Amodei wrote a sentence that has stayed
with me longer than most things I have read about artificial intelligence.
“Humanity is about to be handed almost unimaginable power,
and it is deeply unclear whether our social, political, and
technological systems possess the maturity to wield it.”
DARIO AMODEI · ANTHROPIC CEO · 2025