AI gets its bearings: new models close the gap in 3d spatial reasoning and action

By mid-2026, research from the SpatialAct benchmark has illuminated a critical disconnect in current vision-language models (VLMs): while these agents can handle isolated spatial reasoning tasks with some proficiency, they falter significantly when required to maintain coherent spatial understanding and execute reliable actions across multi-turn interactions in dynamic 3D environments. This reasoning-to-action gap highlights that VLMs struggle not just with planning but with sustaining accurate spatial beliefs as the environment evolves, a challenge that humans navigate with ease.

A deeper challenge uncovered by SpatialAct is the VLM agents’ inability to robustly track spatial states when their own actions induce changes in the environment, even when complexities of low-level control are abstracted away. This suggests that the core issue lies not in motor execution but in the models’ internal representation and updating of spatial information, underscoring a fundamental limitation in current architectures’ capacity to integrate perception, memory, and action in a cohesive loop.

By mid-2026, the GeoVR framework emerged as a groundbreaking approach to instilling intrinsic 3D spatial awareness in multimodal large language models (MLLMs) using only 2D video sequences. It achieves this by embedding explicit geometric and physical constraints into the model’s internal representations, leveraging a multi-objective learning strategy that distills geometry knowledge from pre-trained 3D foundation models. This strategy targets four complementary geometric objectives—estimating inter-frame camera poses, regressing dense depth maps, predicting metric scale factors, and extracting multi-scale 3D features—thereby reshaping how MLLMs comprehend spatial environments.

GeoVR’s innovative integration of physical and geometric constraints not only enhances 3D spatial consistency but also propels spatial reasoning capabilities to new heights. Extensive benchmarking demonstrates that GeoVR sets a new state-of-the-art standard on spatial reasoning tasks, marking a paradigm shift in how foundation models acquire spatial intelligence. This advancement underscores the potential of embedding explicit geometric priors to overcome traditional limitations of 2D-based models, effectively bridging the gap toward more robust and intrinsically spatially aware AI systems.

By mid-2026, GeneralVLA-2 marked a significant leap in robotic vision-language planning through its GeoFuse-MV3D method, which integrates geometry-prior-guided multi-view 3D reconstruction to stabilize object shapes. This approach uniquely verifies external geometry cues using input-view masks and applies soft visual-hull support combined with axis-wise refinement, fusing geometry while preserving appearance. These innovations yielded measurable improvements over the MV-SAM3D baseline, including a 2.20% reduction in Chamfer Distance and a 2.02% decrease in LPIPS, alongside increases in PSNR and SSIM by 2.36% and 1.03%, respectively, collectively enhancing spatial reasoning precision.

Complementing these reconstruction advances, GeneralVLA-2 also upgraded its KnowledgeBank into a governed long-term memory system that meticulously manages quality, confidence, lifecycle, verifier, and conflict metadata. This governance framework significantly sharpened memory retrieval precision and spatial reasoning accuracy, improving Terminal-Bench success rates by 4.53% and SWE-Bench resolve rates by 3.73%, while simultaneously reducing ambiguity scores by nearly 5%. This shift towards explicit memory control not only bolsters geometric relevance but also empowers robots with more reliable and contextually aware vision-language planning capabilities.

By mid-2026, foundational research such as "Watch, Remember, Reason" laid the groundwork for closed-loop contextual reasoning in video understanding by framing it through human-like faculties: watching, remembering, and reasoning. This unified framework not only addressed the challenges of long-horizon multimodal video tasks by emphasizing memory modeling and streaming context preservation but also demonstrated versatility across diverse domains including egocentric, sports, instructional, medical, and narrative videos, underscoring the critical role of multimodal alignment and evidence-grounded reasoning in advancing MLLMs.

Building on these principles, InternVideo3 introduced a sophisticated Multimodal Contextual Reasoning (MCR) framework that operationalizes video understanding as an iterative, closed-loop process of evidence accumulation and verification within a dynamically evolving shared context. This approach integrates observations, instructions, reasoning traces, tool actions, and memory to enable robust, visually grounded agency over long video sequences, marking a significant evolution in how open multimodal models adapt to complex, temporally extended tasks.

To tackle the computational challenges inherent in long-horizon multimodal video tasks, InternVideo3 employs Multimodal Multi-head Latent Attention (M²LA), a novel token-preserving reparameterization that compresses KV-cache states without sacrificing the full token stream. Coupled with a rigorous staged training regimen—including continued pretraining, supervised fine-tuning, rule-based reinforcement learning, and on-policy distillation—this design enables InternVideo3 to achieve state-of-the-art performance on benchmarks like Video-MME, MLVU, and EgoSchema, showcasing its capacity for evidence-grounded video agent behavior and efficient context handling at scale.

The Reason then Re-reason (ReRe) framework, introduced in mid-2026, revolutionizes spatial reasoning by enabling models to revisit and refine their spatial hypotheses through cross-view synthesis. By generating complementary novel views—particularly elevated and oblique perspectives—via a Geometry-to-Video pipeline, ReRe allows vision-language models (VLMs) to verify or revise their initial inferences in a training-free, inference-time process. This approach significantly elevates the performance of open-source multimodal large language models (MLLMs), enabling them to rival proprietary state-of-the-art counterparts on benchmarks like VSI-Bench and STI-Bench.

Building on the concept of dynamic spatial reasoning, the S-Agent paradigm transforms spatial perception from isolated frame-level predictions into a holistic, spatio-temporal evidence accumulation process. By casting the VLM as a semantic planner that orchestrates a hierarchy of spatial tools—ranging from 2D object grounding to 3D geometric lifting—S-Agent fosters continuous multi-view and video understanding. This scene-centric approach, augmented with temporal memory mechanisms such as Scene Memory and Agent Memory, enables the model to maintain evolving scene states and integrate reasoning context across frames and steps, thereby substantially enhancing spatial intelligence.

S-Agent’s impact extends beyond conceptual innovation, delivering practical performance gains across both open-source and closed-source VLMs without requiring additional training. Moreover, leveraging supervised fine-tuning on a vast spatial trajectory dataset (S-300K) generated by S-Agent itself, the resultant S-Agent-8B model surpasses similar-scale baselines and achieves parity with advanced closed-source giants like GPT-5.4 and Gemini 3. This breakthrough underscores the paradigm’s potential to democratize high-level spatial reasoning capabilities in vision-language systems.