What is one practice or design choice you have found most important for advancing reinforcement learning beyond traditional methods?
The answers highlight diverse approaches—ranging from novel reward signals and data quality to architectural changes and hardware choices—showing no single consensus but a shared focus on moving beyond standard RL techniques.
7 answers
Answers


Emma Brunskill
Associate Professor (tenured), Computer Science Department · Stanford University
Preference-Based Reward Signals
Emma Brunskill says preference-based reward signals are really important and very powerful, and that she will cover them in her class.
The answer starts at 8:47.
Stanford Online: University Courses & Lectures · Listen on Apple Podcasts
Diogo Almeida
Software Engineer · Google
RLCD
Diogo Almeida says his team had to create a new type of training they call RLCD, because no one had done anything to optimize for machine native intelligence.
The MAD Podcast with Matt Turck · Watch on YouTube

Nathan Lambert
Founder · Interconnects Ai
Buying High-Quality Data
Nathan Lambert says buying high-quality data is key, arguing a lot of frontier progress in domains that aren't fully verifiable comes from buying high-quality data and giving models worlds to act in.
a16z · Watch on YouTube

Lasse Espeholt
Co-Founder · Ineffable Intelligence
Using GPUs for RL
Lasse Espeholt says using GPUs for RL, citing his work building an RL agent that utilized GPUs and could scale linearly with compute.
The answer starts at 58:28.
Training Data · Listen on Apple Podcasts
Max Rumpf
Founder · Sid.Ai
Sub-Agent for Searching
Max Rumpf recommends using a sub-agent for searching, because it keeps bad results out of the main agent's context window so the main agent sees more good stuff and is more likely to be correct.
AI Engineer · Watch on YouTube
Richard Socher
CEO · Recursive
Reward Engineering
Richard Socher says reward engineering is one of the most crucial bits, especially in order to avoid reward hacking, because as the AI gets better it gets better at finding weird special cases and counterexamples.
Latent Space · Watch on YouTube

Jerry Tworek
Chief Executive Officer · Core Automation
Combine Pretraining and RL
Jerry Tworek says the practice is to combine pretraining and RL together, claiming that is where an order of magnitude improvement would come from.
Training Data · Training Data · Listen at 27:55
More Data Science & Machine Learning questions
View all →- Why does AI need continual learning beyond static training?6 answers
- What is the biggest technical limitation of LLMs?7 answers
- What practice best gives AI models the right context?9 answers
- What is key to building effective domain-specific AI models?9 answers
- What matters most when evaluating autonomous AI agents?6 answers


















