What practice best improved your AI product testing?

Richard Sikang Bian

Product & Growth, Ant Ling & InclusionAI · Ant Group

Strategic evaluation

Richard Sikang Bian says being strategic about evaluation—evaluating things differently—is a good sign, and says his model lab hires people who know how to evaluate models.

The answer starts at 23:42.

Marty Cagan

Partner · Silicon Valley Product Group

Watching users

Marty Cagan says watching users use the product is the practice: he describes bringing the team to watch one user, and says two weeks in they had improved the product drastically, calling it maybe the final cheat code in product.

Tamar Yehoshua

Chief Product and AI Officer · Atlassian

Focus on the customer

Tamar Yehoshua says the way to keep up with AI tools is to focus on the customer — figuring out what is actually working for their customers rather than believing what you read on X.

JE

Jonathan Evens

Product Lead · Google Deepmind

Defining product principles

Jonathan Evens says defining product principles is the real secret, since they come before evaluations: you must first decide what experience you're trying to create, and evaluations translate that into something teachable.

Ben Celebicic

Chief Product and Technology Officer (CPTO) · Hinge

Rapid feedback loop

Ben Celebicic says the biggest value is the feedback loop: he can prototype without engineering support and put it in users' hands to learn quickly whether an idea is worth building or should be left behind.

Matt Ausman

Chief Information Officer · Zebra Technologies

AI‑generated test cases

Matt Ausman says he wants to use AI to write more test cases, though he questions whether validating a million AI-generated cases is feasible. He says testing is what scares him most.

TO

Todd Olson

CEO / Founder · Pendo.Io

Running experiments

Todd Olson says running experiments—trying things out and having multiple people test different solutions to see what works best for their environment—is part of slowing down to speed up, and says teams need time to do it.

FF

Faith Forster

Chief Maker · Discoveree For Product Leaders & Managers

Real‑time user observation

Faith Forster says the practice is real-time user observation—working alongside customers and watching how they use the product—since AI makes building so quick that the definition of done is now adoption.