What principle or practice do you consider most important for ensuring trust in AI systems used in finance, and why?

Most answers emphasize the importance of human oversight, transparency, and explainability, though some focus on robust governance, verification, or understanding AI’s limits to ensure trust in financial AI systems.

10 answers

Answers

  • Hassan El Mghari

    Director of Developer Experience · Together Ai

    Human in the Loop

    Hassan El Mghari says human in the loop is the key practice, because engineering checks alone missed domain-specific accounting checks that a human had to catch.

    AI Engineer · Watch on YouTube
  • Garrett Lord

    CEO - Co-Founder · Handshake

    Interpretability

    Garrett Lord says interpretability matters most for getting AI used in a bank, because banking is high-consequence and small mistakes can have large downstream effects, so the system must surface and interact with a human.

    N/A · N/A · Listen at 1:39:52
  • Joe Welu

    CEO and Founder · Total Expert

    Human-in-the-loop

    Joe Welu says keeping a human in the loop is the answer for the foreseeable future, because he thinks AI shouldn't invest money without human judgment somewhere in the process and the technology is too new.

    Chrisman Commentary · Watch on YouTube
  • Nirav Prajapati

    Co-Founder & CEO · Ignosis Ai

    Robust governance

    Nirav Prajapati says robust governance and transparent AI models are still needed to ensure automated lending decisions remain fair, explainable and accountable, which determines how much trust customers, institutions and regulators place in AI-driven lending.

    Analytics Insight · Analytics Insight · Listen at 9:47
  • Maik Taro Wehmeyer

    Co-Founder & CEO · Taktile

    Human in the loop

    Maik Taro Wehmeyer says you should always have the possibility of a human in the loop, because use cases differ: a loan default can cost a bank a lot of money, and for AML the regulator wants a human to look at it.

    Fintech Leaders · Fintech Leaders · Read
  • Christian Catalini

    Co-Founder and Chief Strategy Officer · Lightspark

    Verification

    Christian Catalini says the act of verification is the actual bottleneck, since AI can be applied to anything measured and automated but its output can't be trusted when important dimensions go unmeasured.

    Faster, Please! · Faster, Please! · Read
  • Reid Hoffman

    Co-Founder, Executive Board Chair · Manas Ai

    Know the limits

    Reid Hoffman says the important thing is to learn where AI tools' limits are and then figure out where they fit, making use of their strengths, since they lack shared contextual awareness and may get trapped in lacunas.

    Reid Hoffman · Watch on YouTube
  • Ameya Bhatawdekar

    VP, Field CTO · Braintrust

    Robust evaluation discipline

    Ameya Bhatawdekar says building a robust eval discipline with the right tools, automation and systems is paramount to manage generational model changes, so that systems keep improving while still working well for what they did before.

    AI Engineer · Watch on YouTube
  • R "Ray" Wang

    Principal Analyst, Founder & Chairman · Constellation Research, Inc.

    Transparency

    Ray Wang says his firm insists on transparency, telling clients where AI is and isn't used and encouraging them to be transparent with their own clients about what is AI-generated.

    DisrupTV · DisrupTV · Listen at 13:49
  • Ajay Swamy

    Senior Executive Director - GenAI Products, AIML Platform Management and Governance · Jpmorganchase

    Auditability

    Ajay Swamy says any model in a business-critical process must be auditable end to end, from the data it takes in to how outcomes occurred, so he can explain results to risk committees, customers and regulators.

    The AI in Business Podcast · The AI in Business Podcast · Listen at 9:28