LLMs get smarter and 15x faster: multi-stage RL training meets speculative decoding in 2026 breakthroughs

Early research in late 2025 established that reinforcement learning (RL) enhances language model reasoning only when pre-training leaves sufficient capacity and RL targets tasks at the model’s edge of competence, with at least 1% exposure during pre-training necessary for RL to generalize effectively and yield up to 60% improvements in pass@128 performance. Introducing a mid-training stage between pre-training and RL further boosted out-of-distribution reasoning by 10.8% over RL alone, highlighting the value of multi-stage training pipelines that scaffold learning progressively rather than relying solely on RL fine-tuning.

By mid-2026, the ExpRL method emerged as a pioneering approach to mid-training reinforcement learning, leveraging human-written question-answer data as reward scaffolds rather than imitation targets. This innovation allowed reinforcement of partial progress and intermediate reasoning steps through LLM-judged dense rewards, outperforming supervised fine-tuning, sparse-reward GRPO, and self-distillation on challenging math reasoning tasks. ExpRL’s automated, scalable design addresses a critical bottleneck in multi-stage training by eliminating the need for manual skill specification and shows promise beyond math domains, potentially enabling broader mixed-domain reasoning improvements.

Scaling reinforcement learning for large language models proved challenging initially due to infrastructure demands and the difficulty of credit assignment across long agentic rollouts, where feedback is only available at the end, complicating attribution of successful actions. However, as models reached a certain scale with strong world knowledge priors, RL began to work more reliably, especially when combined with human feedback (RHF) to mitigate reward hacking—a technique adapted early from game environments by pioneers like DeepMind and OpenAI. Among RL algorithms, the open source community has largely converged on GRPO for its simplicity and scalability, reflecting a broader trend that simpler, scalable methods tend to outperform more complex alternatives.

By mid-2026, reinforcement learning (RL) integration in large language model (LLM) training had matured significantly, evolving from the foundational InstructGPT three-step RLHF approach to sophisticated multi-stage RLHF pipelines exemplified by Llama 2 and Llama 3. These newer recipes replaced traditional online RL with iterative distillation, reward model filtering, and multiple training rounds, markedly boosting training efficiency and model capabilities. The DeepSeek series further illustrates this trajectory, advancing from simple SFT plus GRPO RL in late 2024 to complex multi-specialist RL and distillation workflows by early 2026, consolidating over ten domain experts via multi-objective policy distillation (MOPD) to enhance reasoning and domain expertise.

Scaling RL for LLMs has confronted persistent challenges, notably the high computational cost of sampling numerous answers and the thorny credit assignment problem during long rollouts where correctness is only ascertainable at the end. This complexity has driven a community-wide preference for simpler yet scalable methods such as GRPO, which has become a favored open-source standard due to its balance of effectiveness and computational tractability. Hugging Face’s NeMo-RL combined with vLLM exemplifies practical progress in rollout efficiency, achieving up to 2.5x faster RL training through speculative decoding acceleration, directly addressing the bottleneck of rollout speed.

Recent innovations have pushed RL beyond mere reward optimization towards nuanced reasoning and verifiability, as seen in Tülu 3’s introduction of RL with verifiable rewards (RLVR) and DeepSeek R1’s integration of reasoning-focused RL alongside GRPO. The ExpRL framework further advances this evolution by employing human-written question-answer pairs as reward scaffolds rather than imitation targets, enabling reinforcement of intermediate reasoning steps and partial progress. This approach outperforms traditional supervised fine-tuning and sparse-reward RL on challenging math reasoning tasks and shows promise for broader domain applicability, signaling a shift towards RL as a central, sophisticated driver in LLM training pipelines.

The viability of RL for large language models only crystallized once models reached a critical scale with robust world knowledge priors, overturning earlier skepticism about RL’s finickiness. This milestone, noted in mid-2026 analyses, underscores the interplay between model scale and RL effectiveness, suggesting that the maturation of LLM architectures and their embedded priors was a prerequisite for RL’s successful integration and scalability in training regimes.

Speculative decoding emerged as a groundbreaking technique to accelerate large language model inference by leveraging a smaller, faster draft model to propose multiple tokens ahead, which a larger target model then verifies in parallel. This approach, described as a 'lossless technique' by reasoning LLMs themselves, reliably achieves two to three times speedups without sacrificing output quality, effectively doubling inference efficiency and becoming an industry standard adopted by major players like Google, Anthropic, and Meta. As of early 2026, frameworks such as Hugging Face Transformers and vLLM have integrated speculative decoding, making it accessible for production use across a variety of models including Qwen3 and Llama 3.1.

The evolution of speculative decoding techniques has seen significant refinement from basic autoregressive drafting to sophisticated methods like Speculative Speculative Decoding (SSD), DFlash, EAGLE, and DDTree. SSD introduced the concept of running drafts ahead and verifying them in parallel, addressing algorithmic challenges to push inference speed further. DFlash innovated by replacing the autoregressive drafter with a lightweight block diffusion model that drafts entire token blocks in parallel, achieving up to an 8.5× speedup on benchmarks such as HumanEval without quality loss. Meanwhile, EAGLE3, applied to Gemma 4, diverges by training a draft head that fuses multi-level features from the target model for direct token prediction, showing promising acceptance lengths that hint at meaningful acceleration potential. DDTree builds upon DFlash by constructing a draft tree enabling simultaneous verification of multiple token continuations, surpassing previous limitations and outperforming strong autoregressive drafters like EAGLE-3.

Beyond raw speed gains, speculative decoding has transformed inference from a mere cost factor into a critical capability defining the peak intelligence and performance of language models. As one presenter noted in mid-2026, the tokens-per-second rate enabled by these techniques directly correlates with the 'peak intelligence' a model can deliver, underscoring the strategic importance of inference acceleration. This shift has spurred a wave of innovation within Google and across the AI industry, inspiring new efficiency improvements and integration into real-world products like Google Search and Google Cloud, where TPU implementations have dramatically improved latency and throughput overnight.

By early 2026, Google Research's speculative decoding innovation had become a cornerstone in industrial large language model inference, running natively on TPUs across Google Search and Google Cloud. This technique, which leverages a faster draft model to propose outputs verified in parallel by a larger model, effectively doubled computational efficiency and led to immediate, tangible improvements in product latency and throughput. As one analysis put it, much of the industry's LLM computation now operates as if it had twice the available compute, underscoring speculative decoding's transformative impact on production-scale AI workloads.

In a striking demonstration of real-world performance breakthroughs, Xiaomi and TileRT unveiled MiMo-V2.5-Pro-UltraSpeed in mid-2026, a serving mode for their trillion-parameter flagship model that achieves over 1,000 tokens per second on a standard 8-GPU commodity node—without any custom hardware. This 15x speedup over contemporaries like GPT-5.5 and Claude Opus was made possible by an ingenious synergy of FP4 quantization applied selectively to expert layers and DFlash speculative decoding, a co-designed system-model approach that reduces memory and bandwidth bottlenecks while enabling multi-token proposal and verification in a single pass.

This leap in inference speed has unlocked new application scenarios with stringent latency demands, such as fraud detection, trading signal generation, and real-time agent loops, where traditional models running at 60–70 tokens per second fall short. Xiaomi's decision to launch a limited API trial in June 2026, priced at three times the standard MiMo rates but delivering roughly ten times the generation speed, signals a confident industrial adoption and readiness for real-world deployment. By prioritizing enterprise and professional developers, Xiaomi is catalyzing a shift toward ultra-fast, scalable language model services that can meet the demands of latency-critical applications.

By mid-2026, innovations like EfficientRollout have demonstrated the tangible benefits of integrating training and inference techniques, notably through self-speculative decoding which accelerates reinforcement learning rollout speeds by up to 19.6% without sacrificing model quality. This approach exemplifies a broader trend towards blending algorithmic strategies with architectural enhancements to optimize both the training efficiency and inference responsiveness of scalable language models, signaling a promising direction for future research and deployment.