What practice advances reinforcement learning beyond tradition?

Diogo Almeida

Software Engineer · Google

RLCD

Diogo Almeida says his team had to create a new type of training they call RLCD, because no one had done anything to optimize for machine native intelligence.

Nathan Lambert

Founder · Interconnects Ai

Buying High-Quality Data

Nathan Lambert says buying high-quality data is key, arguing a lot of frontier progress in domains that aren't fully verifiable comes from buying high-quality data and giving models worlds to act in.

Anastasis Germanidis

Co-Founder & CTO · Runway

Learned World Models

Anastasis Germanidis says Learned World Models are the more scalable path, because the model learns the rules of the world from observation instead of having them hand-programmed, with fidelity improving given more data and compute.

Lasse Espeholt

Co-Founder · Ineffable Intelligence

Using GPUs for RL

Lasse Espeholt says using GPUs for RL, citing his work building an RL agent that utilized GPUs and could scale linearly with compute.

The answer starts at 58:28.

MR

Max Rumpf

Founder · Sid.Ai

Sub-Agent for Searching

Max Rumpf recommends using a sub-agent for searching, because it keeps bad results out of the main agent's context window so the main agent sees more good stuff and is more likely to be correct.

RS

Richard Socher

CEO · Recursive

Reward Engineering

Richard Socher says reward engineering is one of the most crucial bits, especially in order to avoid reward hacking, because as the AI gets better it gets better at finding weird special cases and counterexamples.

Jerry Tworek

VP of Research · Openai

Combine Pretraining and RL

Jerry Tworek says the practice is to combine pretraining and RL together, claiming that is where an order of magnitude improvement would come from.