What is one practice or design choice you have found most important for advancing reinforcement learning beyond traditional methods?

The answers highlight diverse approaches—ranging from novel reward signals and data quality to architectural changes and hardware choices—showing no single consensus but a shared focus on moving beyond standard RL techniques.

7 answers

Answers

  • Emma Brunskill

    Associate Professor (tenured), Computer Science Department · Stanford University

    Preference-Based Reward Signals

    Emma Brunskill says preference-based reward signals are really important and very powerful, and that she will cover them in her class.

    The answer starts at 8:47.

    Stanford Online: University Courses & Lectures · Listen on Apple Podcasts
  • Diogo Almeida

    Software Engineer · Google

    RLCD

    Diogo Almeida says his team had to create a new type of training they call RLCD, because no one had done anything to optimize for machine native intelligence.

    The MAD Podcast with Matt Turck · Watch on YouTube
  • Nathan Lambert

    Founder · Interconnects Ai

    Buying High-Quality Data

    Nathan Lambert says buying high-quality data is key, arguing a lot of frontier progress in domains that aren't fully verifiable comes from buying high-quality data and giving models worlds to act in.

    a16z · Watch on YouTube
  • Lasse Espeholt

    Co-Founder · Ineffable Intelligence

    Using GPUs for RL

    Lasse Espeholt says using GPUs for RL, citing his work building an RL agent that utilized GPUs and could scale linearly with compute.

    The answer starts at 58:28.

    Training Data · Listen on Apple Podcasts
  • Max Rumpf

    Founder · Sid.Ai

    Sub-Agent for Searching

    Max Rumpf recommends using a sub-agent for searching, because it keeps bad results out of the main agent's context window so the main agent sees more good stuff and is more likely to be correct.

    AI Engineer · Watch on YouTube
  • Richard Socher

    CEO · Recursive

    Reward Engineering

    Richard Socher says reward engineering is one of the most crucial bits, especially in order to avoid reward hacking, because as the AI gets better it gets better at finding weird special cases and counterexamples.

    Latent Space · Watch on YouTube
  • Jerry Tworek

    Chief Executive Officer · Core Automation

    Combine Pretraining and RL

    Jerry Tworek says the practice is to combine pretraining and RL together, claiming that is where an order of magnitude improvement would come from.

    Training Data · Training Data · Listen at 27:55

More Data Science & Machine Learning questions

View all →