What is one specific practice your team uses to create a tight feedback loop for improving AI models, and why?
Most teams use structured systems or direct user feedback to rapidly inform model improvements, though their specific methods—ranging from rubrics to live data—vary widely.
12 answers
Answers


Tulsee Doshi
Sr. Director & Product Lead, GenAI & Gemini Models · Google
Rubrics from User Feedback
Tulsee Doshi says her team looks at customer and user feedback to define what good looks like for Gemini, then translates that into rubrics to evaluate the model and turn into data for reward modeling.
Lenny's Podcast · Watch on YouTube

Laura Modiano
Founder Experience · Openai
Structured Feedback System
Laura Modiano says her team captures feedback in a structured system of record, organizing anything written or from calls so it can be routed to the right teams and keep everyone moving in the same direction.
The European VC · Watch on YouTube

Jayesh Govindarajan
Executive Vice President, Salesforce AI · Salesforce
Post to Slack Channel
Jayesh Govindarajan says every forward deployed engineer in his org posts to one Slack channel, tagging each post as a paper cut or feature request, so PMs can match them against the roadmap and learn directly from the field to improve the product.
Product Growth · Product Growth · Read

Emily Man
Partner · Primary Venture Partners
Break Down Jobs to KPIs
Emily Man describes loops as breaking the jobs to be done into discrete component parts aligned to business metrics and KPIs, so the system you build keeps getting better over time.
Run the Numbers with CJ Gustafson · Watch on YouTube

Sam Stephenson
Founder · Granola
Ship Early Internally
Sam Stephenson says his team ships things internally as quickly as they can to small groups of friends and baby users very early, though they hold a high bar for whether it actually works for people before putting it out.
Peter Yang · Watch on YouTube

Karri Saarinen
CEO, Cofounder · Linear
Winner Loop for Bug Investigation
Karri Saarinen says his team uses the "winner loop" to investigate bugs, connecting to data, Sentry and other tools to find the bug's source and write a fix, which saves engineers time so they only verify results.
Lenny's Podcast · Watch on YouTubeAndrew Somervell
Founder (Product + Technology) · Hamster
HTML Briefs for Feedback
Andrew Somervell says his team builds an HTML brief to present information densely so a colleague can quickly react and give feedback, though he notes the workflow of pushing it through GitHub or Slack adds friction.
Supra Insider · Watch on YouTube
Jonathan Evens
Product Lead · Google Deepmind
Collect User Interaction Signals
Jonathan Evens says his team collects user interaction signals such as ratings, clicks, thumbs up and full listens, calling these signals extremely valuable for the reinforcement learning from human feedback loop.
Mind the Product · Watch on YouTube

Hassan El Mghari
Director of Developer Experience · Together Ai
Automated Agent Evals
Hassan El Mghari says his team writes tasks representing real usage with each PM and runs agent evals that spin up a sandbox, launch Claude Code, and grade the result; the transcript doesn't show him giving a reason for this practice beyond describing the loop.
Product Growth · Product Growth · Read

Blaine Billingsley
Member of Design Staff · Openai
Run Many Evaluations
Blaine Billingsley says a huge part of his job, and everyone's, is running evaluations—running the thing many times and judging whether the results are good, heuristically or by rubric—which he calls a new and strange part of the work.
Mind the Product · Watch on YouTube

Sonya Huang
Partner · Sequoia Capital
Live Customer Data Feedback
Sonya Huang describes setting up a system so live customer data creates a feedback loop where the model improves with every customer interaction, as the final step in the journey she outlines.
Sequoia Capital · Watch on YouTube

Shaun Modi
CEO · Capitol
One‑Week Release Cycles
Shaun Modi says his team is on one-week release cycles.
FinStrat Management, Inc. · Watch on YouTube
More Generative AI questions
View all →- What design choice improves AI agent privacy?5 answers
- What architectural choices enable reliable agentic AI systems?8 answers
- What responsibilities should AI creators have for their models?12 answers
- What architecture optimizes compute for AI workloads?13 answers
- How do you balance AI customization and scalability?8 answers



















