What improves user experience in AI assistants most?

Sam Stephenson

Founder · Granola

Sweat the details

Sam Stephenson says the practice that matters is sweating the details and going the extra mile, since most people stop too early and ship before making something actually useful.

Josh Elman

Director of Product Management · Apple

Do a few things well

Josh Elman says start by doing a few things really well and earn the right to do more, rather than claiming to handle everything; claiming everything doesn't earn trust to get a few things done.

The answer starts at 35:38.

JE

Jonathan Evens

Product Lead · Google Deepmind

User empathy

Jonathan Evens says user empathy—focusing on deep user problems and pain points and starting with the problem—is essential for AI product or feature PMs.

Amit Desai

Director Voice AI - leads UX, product, engineering · Roku

Smarter conversational behavior

Amit Desai points to a second lever beyond accuracy — smarter conversational behavior under uncertainty — saying that exploiting it can help AI assistants reach an acceptable user experience.

Dilani Kahawala

Co-Founder and CEO

Teaching users

Dilani Kahawala lists teaching users how to work with an agent among the three biggest challenges she's had to think about, alongside reliability and the interface.

Blaine Billingsley

Member of Design Staff · Openai

Diverse eval sets

Blaine Billingsley says the key is to continually create new, throwaway eval sets, because sharing them with others widens his aperture for what an assistant might be used for.

TO

Todd Olson

CEO / Founder · Pendo.Io

Measuring user interactions

Todd Olson says there's no replacement for really measuring how users interact with software, because an eval may score something well while the actual user has a different experience.

FF

Faith Forster

Chief Maker · Discoveree For Product Leaders & Managers

Continuous user observation

Faith Forster says teams should work alongside customers, watching how they use the product and finding friction, because AI makes improvements easier and shifts the definition of done to adoption.

Dianne Na Penn

Head of Product Management, Research · Anthropic

User feedback

Dianne Na Penn says getting user feedback has to be fundamental for everyone.