What principle or practice do you consider most important for ensuring trust in AI systems used in finance, and why?
Most answers emphasize the importance of human oversight, transparency, and explainability, though some focus on robust governance, verification, or understanding AI’s limits to ensure trust in financial AI systems.
10 answers
Answers


Hassan El Mghari
Director of Developer Experience · Together Ai
Human in the Loop
Hassan El Mghari says human in the loop is the key practice, because engineering checks alone missed domain-specific accounting checks that a human had to catch.
AI Engineer · Watch on YouTube

Garrett Lord
CEO - Co-Founder · Handshake
Interpretability
Garrett Lord says interpretability matters most for getting AI used in a bank, because banking is high-consequence and small mistakes can have large downstream effects, so the system must surface and interact with a human.
N/A · N/A · Listen at 1:39:52

Joe Welu
CEO and Founder · Total Expert
Human-in-the-loop
Joe Welu says keeping a human in the loop is the answer for the foreseeable future, because he thinks AI shouldn't invest money without human judgment somewhere in the process and the technology is too new.
Chrisman Commentary · Watch on YouTube

Nirav Prajapati
Co-Founder & CEO · Ignosis Ai
Robust governance
Nirav Prajapati says robust governance and transparent AI models are still needed to ensure automated lending decisions remain fair, explainable and accountable, which determines how much trust customers, institutions and regulators place in AI-driven lending.
Analytics Insight · Analytics Insight · Listen at 9:47

Maik Taro Wehmeyer
Co-Founder & CEO · Taktile
Human in the loop
Maik Taro Wehmeyer says you should always have the possibility of a human in the loop, because use cases differ: a loan default can cost a bank a lot of money, and for AML the regulator wants a human to look at it.
Fintech Leaders · Fintech Leaders · Read

Christian Catalini
Co-Founder and Chief Strategy Officer · Lightspark
Verification
Christian Catalini says the act of verification is the actual bottleneck, since AI can be applied to anything measured and automated but its output can't be trusted when important dimensions go unmeasured.
Faster, Please! · Faster, Please! · Read

Reid Hoffman
Co-Founder, Executive Board Chair · Manas Ai
Know the limits
Reid Hoffman says the important thing is to learn where AI tools' limits are and then figure out where they fit, making use of their strengths, since they lack shared contextual awareness and may get trapped in lacunas.
Reid Hoffman · Watch on YouTube

Ameya Bhatawdekar
VP, Field CTO · Braintrust
Robust evaluation discipline
Ameya Bhatawdekar says building a robust eval discipline with the right tools, automation and systems is paramount to manage generational model changes, so that systems keep improving while still working well for what they did before.
AI Engineer · Watch on YouTube

R "Ray" Wang
Principal Analyst, Founder & Chairman · Constellation Research, Inc.
Transparency
Ray Wang says his firm insists on transparency, telling clients where AI is and isn't used and encouraging them to be transparent with their own clients about what is AI-generated.
DisrupTV · DisrupTV · Listen at 13:49

Ajay Swamy
Senior Executive Director - GenAI Products, AIML Platform Management and Governance · Jpmorganchase
Auditability
Ajay Swamy says any model in a business-critical process must be auditable end to end, from the data it takes in to how outcomes occurred, so he can explain results to risk committees, customers and regulators.
The AI in Business Podcast · The AI in Business Podcast · Listen at 9:28