What practice helps you iteratively improve AI-driven tools?

The answers highlight a range of iterative improvement practices, from user feedback and evaluation frameworks to automation, planning, and parallelization, each emphasizing a distinct approach.

5 answers

Answers

  • Matt Taylor

    Founder · Trepo

    User Feedback

    Matt Taylor says his practice is user feedback: he and Zach ask users what's wrong, learn from that, and develop better solutions for them.

    NZ Tech Podcast · NZ Tech Podcast · Listen at 44:49

  • Nan Yu

    Head of Product · Linear

    Building Evals

    Nan Yu says his team builds out evals to set success criteria and optimize models down, since evals give a framework to ensure performance; much comes from iterating and adding unexpected user behavior to their dataset.

    Peter Yang · Watch on YouTube

  • Daniel Herrington

    Chief AI Officer · The Zebra

    Continuous Automated Monitoring

    Daniel Herrington says his team uses continuous automated monitoring: a system runs constantly checking for errors in customer connections and spawns an agent to write fixes, because they can't staff thousands of engineers to handle API changes.

    The Software Leaders Uncensored Podcast · The Software Leaders Uncensored Podcast · Listen at 14:10

  • 👨🏾‍💻 Ben Ilegbodu

    Senior Software Engineer, TV Client Foundations · Netflix

    Planning and Decision-Making

    Ben Ilegbodu points to planning and decision-making as the most durable practice, since agents and models aren't talking to customers and won't know what to build.

    Become an Epic Product Engineer · Become an Epic Product Engineer · Listen at 21:02

  • Jarred Sumner

    Creator of Bun · Anthropic

    Fanning Out Work

    Jarred Sumner says he fans out a lot of work to many Claudes at the same time, an approach he says works very well for him and is underused.

    The Pragmatic Engineer · The Pragmatic Engineer · Read

More Software Engineering questions

View all →