What safeguards are necessary to ensure the safe and effective deployment of AI in healthcare?
Most answers emphasize the need for strong governance, human oversight, and ongoing validation, while some highlight transparency, informed consent, and vendor accountability as additional safeguards.
16 answers
Answers

Chris Carmody
Chief Information Officer · Upmc
Governance and training
Chris Carmody says the right governance and processes must come first to identify and focus on the top problems, and that training users to understand the why behind the technology is just as important.
The answer starts at 7:04.
Becker’s Healthcare Podcast · Listen on Apple Podcasts

Munjal Shah
Senior Product Manager, Wearables · Google
Output testing
Munjal Shah says the key safeguard is output testing—testing what comes out of the model rather than trusting what went in—and advises looking for a history of it, comparing it to how the FDA tests drugs.
The answer starts at 11:54.
Becker’s Healthcare Podcast · Listen on Apple Podcasts
Hal Wolf
President and Chief Executive Officer · Himss
Governance, training, processes
Hal Wolf says deploying AI safely requires governance, training of the people, and processes in place, along with measuring impact on access and quality.
The answer starts at 6:50.
The Big Unlock · Listen on Apple Podcasts
Aaron Miri, DHA, FCHIME, CHCIO
EVP, Chief Digital & Information Officer · Baptist Health
Informed consent
Aaron Miri says a tolerance level for AI error is acceptable as long as patients choose with informed consent and responsibility, eyes wide open, rather than being unsafe.
The answer starts at 52:11.
"Second Opinion" with Christina Farr, Ash Zenooz MD & Luba Greenwood JD · Listen on Apple Podcasts

Katherine Adams, MS
Director, Clinic Operations, Primary Care · Commonspirit Health
Ongoing Safety Oversight
Katherine Adams says she needs tech partners' ongoing partnership after go-live to keep AI tools learning, avoid hallucinations and stay fine-tuned, and wants their commitment to patient safety post-implementation.
The answer starts at 5:20.
Becker’s Healthcare Podcast · Listen on Apple Podcasts

Colin LeClair
Chief Executive Officer · Connections Health Solutions
Vendor vetting and contracts
Colin LeClair says his organization vets vendors for proven track records and viability, and builds safeguards into contracts so it can absorb a technology if the company folds.
The answer starts at 13:46.
Becker’s Healthcare Podcast · Listen on Apple Podcasts

Amit Malik
Founder & CEO · Amaha
Human clinical accountability
Amit Malik says the human and clinical accountability angle is what is missing and should always be missing from an LLM or AI chatbot, and that AI should not be used for diagnosis, emergency risk, medication changes, or as a sole source of treatment.
The answer starts at 19:14.
Say Again?! With Amrita Tripathi · Listen on Apple Podcasts
David Niewolny
Sr. Dir, Global Head of Business Development - Healthcare / Medical · NVIDIA
Layered Governance
David Niewolny says AI governance in healthcare works in layers: bounded tools that are safe, secure and validated, controlling data access, and a clear human decision point.
The answer starts at 35:22.
Faces of Digital Health · Listen on Apple Podcasts
David Niewolny
Sr. Dir, Global Head of Business Development - Healthcare / Medical · NVIDIA
Bounded Tools
David Niewolny says AI governance in healthcare works in layers: bounded tools that are safe, secure and validated, controlling data access, and a clear human decision point.
The answer starts at 35:22.
Faces of Digital Health · Listen on Apple Podcasts
David Niewolny
Sr. Dir, Global Head of Business Development - Healthcare / Medical · NVIDIA
Data Control
David Niewolny says AI governance in healthcare works in layers: bounded tools that are safe, secure and validated, controlling data access, and a clear human decision point.
The answer starts at 35:22.
Faces of Digital Health · Listen on Apple Podcasts
David Niewolny
Sr. Dir, Global Head of Business Development - Healthcare / Medical · NVIDIA
Human Oversight
David Niewolny says AI governance in healthcare works in layers: bounded tools that are safe, secure and validated, controlling data access, and a clear human decision point.
The answer starts at 35:22.
Faces of Digital Health · Listen on Apple Podcasts

Rhett Alden
CTO, Elsevier Health Markets · Elsevier
Transparency and validation
Rhett Alden says transparency is critical for clinical adoption—both in how a response is arrived at and where its information comes from—and points to his team's robust validation framework as another requirement.
The answer starts at 3:24.
The AI in Business Podcast · Listen on Apple Podcasts

Anirban Maitra
Director, Perlmutter Cancer Center · Nyu Langone Health
Qualified physician oversight
Anirban Maitra says the most important safeguard is using AI safely, which means having qualified physicians analyze and check the results rather than letting AI diagnose without oversight, since the physician's judgment remains the captain of the ship.
The answer starts at 10:15.
Becker’s Healthcare Podcast · Listen on Apple Podcasts

Hannah Galvin, MD, FAAP, FAMIA, CHCIO
Chief Health Information Officer · Cambridge Health Alliance
Robust governance
Hannah Galvin says her organization was an early adopter of mature AI governance, which they have evolved over time to keep up with industry trends and the need to handle many more tools, and to move useful technology into practice safely.
The answer starts at 7:37.
Becker’s Healthcare Podcast · Listen on Apple Podcasts

Zeke Emanuel
Vice Provost for Global Initiatives · University Of Pennsylvania
Biannual review
Zeke Emanuel says AI in healthcare must be watched in hundreds of actual patient cases and reassessed on a regular basis, proposing biannual review every two years, with what it is authorized to do adjusted accordingly.
"Second Opinion" with Christina Farr, Ash Zenooz MD & Luba Greenwood JD · "Second Opinion" with Christina Farr, Ash Zenooz MD & Luba Greenwood JD · Listen at 32:56

Kimberly Powell
Vice President / General Manager, Healthcare at NVIDIA · Nvidia
Healthcare professionals in the loop
Kimberly Powell says it is so important to have healthcare professionals in the loop, meaning both that clinicians remain in the loop when using the tools and that companies build deep clinical understanding into their systems.
Second Opinion with Christina Farr · Watch on YouTube
More HealthTech questions
View all →- What practice ensures trust and safety scaling AI in healthcare?22 answers
- What practice is key for scaling healthcare services?17 answers
- What elements are essential for AI governance in healthcare?10 answers
- How can AI reduce admin burden for clinicians?15 answers
- What should early HealthTech founders prioritize?13 answers













