Which metrics do you rely on to measure the business impact of an AI initiative, and why?

Most answers focus on user engagement, ROI, and efficiency metrics, though some emphasize workflow improvements, cost savings, or direct business outcomes like revenue per employee.

11 answers

Answers

  • Marissa Mayer

    CEO & Founder · Dazzle Ai

    Retention

    Marissa Mayer says retention over time is probably the most important metric, and that the ratio of daily to longer-term active users matters because useful technologies get used nearly every day.

    Semafor · Watch on YouTube
  • Patrick Moorhead

    Founder, CEO and Chief Analyst · Moor Insights & Strategy

    ROI

    Patrick Moorhead says he looks for proof of life in end users, meaning enterprises seeing big ROI swings, because the only way to do this is to reduce expenses or increase top-line revenue.

    The answer starts at 11:51.

    The AI Forecast: Data and AI in the Cloud Era · Listen on Apple Podcasts
  • Sagnik Nandy

    Chief Technology Officer · Docusign

    Cost and Token Efficiency

    Sagnik Nandy says his team measures cost per token and tokens per task, saying both are very important because they focus on ROI and sustainability, and reducing tokens per task and cost per token buys more headroom to try things.

    The answer starts at 36:23.

    The Engineering Leadership Podcast · Listen on Apple Podcasts
  • Brian McMullin

    Senior Vice President and Head of Product · Network Solutions

    Retention

    Brian McMullin says he relies on retention, though he calls it a lagging indicator and says frequency and breadth of product usage are predictive of customers sticking around.

    The answer starts at 26:56.

    LaunchPod · Listen on Apple Podcasts
  • Jonathan Evens

    Product Lead · Google Deepmind

    Side-by-Side Win Rates

    Jonathan Evens says he looks for side-by-side win rates, to continuously evaluate model outputs and see whether they are better than a previous version, a competing product, or a baseline.

    Mind the Product · Watch on YouTube
  • Supratik Shankar, CFA

    Building Dview · Dview

    Time Saved

    Supratik Shankar says AI automation will free up bandwidth and save time for existing teams to focus on other tasks, which he says creates meaningful impact for the organization.

    Analytics Insight · Analytics Insight · Listen at 22:53
  • Jeff Westcott

    Director, Practice Technology, Innovation & AI · Akin Gump Strauss Hauer & Feld LLP

    Adoption and Usage

    Jeff Westcott says he relies on adoption and usage metrics, such as whether people come back and use a tool daily or weekly and how much time they spend, because adoption is a good benchmark.

    ILTA Voices · ILTA Voices · Listen at 31:33
  • mike yu

    cofounder, ceo · vesta

    Human Cost to Originate

    Mike Yu says the numbers they look at are the fully loaded human cost to originate, since AI agents are meant to directly replace operational labor; he adds head count, loans per underwriter monthly, and cycle time.

    Chrisman Commentary · Watch on YouTube
  • Pete Johnson

    Field CTO, Artificial Intelligence · MongoDB

    Workflow Performance Metrics

    Pete Johnson says he relies on existing workflow metrics—like those used to bonus and evaluate employees—because when AI is added to a workflow and those metrics jump, he can attribute the jump to the agent and compute ROI.

    Super Data Science: ML & AI Podcast with Jon Krohn · Watch on YouTube
  • Srinivasan Raghavan

    Chief Product Officer · Freshworks

    Usage and Satisfaction

    Srinivasan Raghavan says he relies on a combination of usage metrics and satisfaction metrics, measured partly through in-product surveys of 5–10% of users monthly, because these signals make the functionality more effective.

    DataCamp · Watch on YouTube
  • Neil Patel

    Co-Founder · Np Digital

    Revenue per Employee

    Neil Patel says he finds measuring revenue per employee a healthy way to look at it as of today, citing someone whose revenue per employee rose from $600,000 to $1.2 million as showing efficiencies.

    Marketing School - Daily Marketing Tips · Watch on YouTube