Open-source models supercharge physical AI’s real-world leap

Tech Xplore ↗

The gist

Open-source, multimodal AI models like NVIDIA's Cosmos 3 are redefining what robots can do in the real world, slashing development time and pushing success rates to new heights.

What to know

  • Physical AI powered by vision-language-action and predictive world models now enables robots to plan, simulate, and execute multi-step tasks with up to 95% reliability.
  • NVIDIA’s June 2026 launch of Cosmos 3—a 20-trillion-token, open-source foundation model—has turbocharged innovation across robotics, vehicles, and beyond.
  • Startups and industry giants alike are leveraging physics-informed AI and scalable simulation to bring embodied intelligence from the lab to industrial floors, sports arenas, and autonomous fleets.

Architectures Reshape Robot Reasoning

Physical AI is leaping forward as hybrid models fuse perception, language, and predictive planning, enabling robots to interpret human intent and simulate actions before execution.

The foundational architecture of physical AI has coalesced around vision-language-action (VLA) models that unify perception, language understanding, and control into cohesive systems. Early breakthroughs, such as Sereact’s Cortex VLA and DeepMind’s SIMA-2, exemplify this integration by moving beyond isolated modules to architectures that jointly train or strategically combine pretrained vision-language backbones with action interfaces. These VLAs leverage massive pretrained vision-language models to improve generalization and robustness in real-world tasks, enabling robots to interpret unstructured human language and sensory inputs into executable actions, a leap from earlier robotics models that lacked reasoning capabilities.

A pivotal architectural innovation in embodied AI is the emergence of predictive world models that extend beyond reactive control to simulate and evaluate multiple possible futures before acting. Models like Odyssey 2, Dreamer v4, Genie-3, and NVIDIA’s DreamZero integrate perception, reasoning, and control by predicting future world states conditioned on agent actions, enabling long-horizon planning and 3D persistent environments. This shift from direct perception-to-action mappings to latent-space planning and simulation—exemplified by Sereact’s Cortex 2.0 and AMI Labs’ JEPA-based models—grounds AI in physical dynamics, improving robustness and enabling zero-shot generalization across tasks and embodiments.

Hybrid architectures that integrate both intuitive, fast system-1 reasoning and deliberate, symbolic system-2 planning represent a new frontier in physical AI design. The PAN world model, for instance, combines a visual-language encoder, a custom-trained large language model with an expanded vocabulary, and a diffusion-based decoder to enable both muscle-memory-like actions and abstract long-term planning. This modular approach contrasts with off-the-shelf LLM usage, allowing iterative improvement and holistic tuning, and reflects a broader trend toward architectures that balance learned abstractions with explicit planning layers to handle complex, multi-step embodied tasks efficiently.

The integration of multimodal data from diverse viewpoints—combining first-person visual and hand movement data with third-person whole-body and object interaction videos—has enabled foundational advances in embodied AI’s understanding of action sequences and purposes. Projects like K-ExaOne, led by LG AI Research Institute with contributions from Superb AI, exemplify this by constructing ultra-large physical AI foundation models that synthesize perception, reasoning, and control. This architectural innovation allows AI to progress from mere scene recognition to grasping complex, temporally extended actions, supporting scalable learning and generalization in real-world robotic systems.

Sources

Open Models Ignite Robotics Race

Open-source multimodal foundation models and global coalitions are democratizing robotics AI, slashing training times and empowering developers to customize and deploy physical intelligence at unprecedented scale.

By early 2026, the physical AI landscape witnessed a transformative acceleration driven by open robotics foundation models trained on massive multimodal datasets, such as tens of millions of hours of robotics and driving data freely accessible via platforms like Hugging Face. Advances in GPU compute efficiency drastically cut training times from years to weeks, enabling rapid iteration and deployment of models that encapsulate real-world physics and object manipulation, while physics-aware synthetic data generation bridged the sim-to-real gap to enhance robot performance in complex environments.

NVIDIA’s June 2026 launch of Cosmos 3 marked a watershed moment in open ecosystem development for physical AI, introducing a 20-trillion-token multimodal foundation model that unifies language, vision, audio, and action prediction within a novel mixture-of-transformers architecture. This open-source release, complete with weights, training scripts, and datasets under the Linux Foundation’s OpenMDW 1.1 license, empowers developers to customize and fine-tune models tailored to their unique robotic platforms and environments, addressing the critical need for adaptable AI in diverse real-world applications.

The formation of the Cosmos Coalition, led by NVIDIA and including partners like Agile Robots, Runway, and LTX, exemplifies a collaborative open ecosystem that accelerates physical AI innovation by pooling resources, datasets, and evaluation methods. This coalition not only fosters global cooperation—recently expanding to Japan to support sectors from logistics to healthcare—but also champions open, composable AI models that developers can run and fine-tune locally, contrasting sharply with competitors who restrict access behind costly APIs and cloud services.

NVIDIA’s open ecosystem strategy extends beyond models to include comprehensive software tools like Omniverse and OpenUSD, which streamline the creation of simulation-ready environments and management of complex 3D assets. By integrating scene understanding, synthetic data generation, and action prediction into a scalable model family ranging from 4 billion to 64 billion parameters, NVIDIA enables efficient adaptation across hardware platforms—from RTX GPUs to Jetson devices—thus addressing the high costs and challenges of collecting diverse real-world data, especially for rare or edge-case scenarios.

Sources
IBM TechnologyGlobeNewswire - Industry News on TechnologyLatent.SpaceTheAIGRIDNVIDIA BlogIB

Industry Deploys Embodied Intelligence

From AI-driven fabrication to autonomous fleets, embodied world models and scalable simulation are transforming sectors as startups and giants alike operationalize physical AI in real-world environments.

Industry adoption of physical AI and embodied world models is rapidly expanding across diverse sectors, from robotics and manufacturing to autonomous vehicles and simulation. Early breakthroughs like MIT’s AI-driven robotic assembly system, which allows users to design and fabricate complex objects from simple text prompts with over 90% user preference, demonstrate tangible impacts in democratizing fabrication and accelerating prototyping workflows. Meanwhile, startups such as General Intuition are pioneering foundation models trained on vast video game datasets to enable spatial-temporal reasoning that transfers efficiently to real-world robotics, achieving remarkable data efficiency with minimal real-world fine-tuning. This convergence of scalable simulation, real-world data grounding, and multimodal AI is driving specialized applications that range from industrial product design at PhysicsX to autonomous trucks and mining equipment powered by Applied Intuition’s Dana platform, underscoring a broad and growing ecosystem of physical AI deployment.

The integration of advanced world models and vision-language-action (VLA) architectures is becoming foundational for enabling robots and autonomous systems to perceive, reason, and act effectively in complex, dynamic environments. Companies like Sereact with Cortex 2.0 leverage imagined futures in visual latent space to proactively avoid unstable trajectories, achieving a 95% success rate on multi-step manipulation tasks without human intervention. Similarly, DeepRoute.ai’s 40-billion-parameter VLA foundation model exemplifies the scale and ambition in autonomous driving, while LG AI Research’s consortium, including Superb AI, advances ultra-large VLA models for industrial robotics and mobility. These models improve generalization across varying conditions, though industry debates continue around optimal training strategies—whether to rely on large-scale diverse data or task-specific datasets—to maximize real-world effectiveness.

Open ecosystems and platform approaches are accelerating physical AI adoption by lowering barriers to development and deployment across industries. Nvidia’s Cosmos 3 foundation model, adopted by leaders like LG, Samsung, and Mercedes-Benz, reduces training time and enables synthetic data generation, addressing critical data scarcity challenges. The launch of the Cosmos Coalition signals a strategic push toward collaborative model development. Similarly, Applied Intuition’s Dana platform integrates agentic AI into the software lifecycle of software-defined vehicles and industrial machines, compressing workflows from months to days and supporting safety-critical applications in trucking, mining, and defense. This shift from isolated model development to integrated engineering platforms emphasizes continuous learning and operationalization as the next competitive moat in physical AI.

Physical AI’s reach is extending beyond traditional robotics into novel domains such as sports entertainment and virtual production, showcasing its versatility and real-world impact. Peripheral Labs applies autonomous vehicle world models to create fully navigable 3D NBA game replays, transforming passive viewing into interactive spatial experiences. Meanwhile, Fei-Fei Li’s World Labs leverages spatial intelligence and large world models to generate interactive 3D replicas for virtual film production, architectural design, and robotics training, supported by scalable simulation pipelines that align digital twins closely with real environments. These specialized applications highlight how embodied AI is not only revolutionizing manufacturing and mobility but also enabling new forms of content creation and immersive experiences.

Sources
Tech XploreTBPNEquityRTAI Podcast Summaries from Transcripted.ai (VIDEO)Scaling Europe

Venture Capital Backs World Models

A surge of investment is fueling a diverse ecosystem of startups building physical AI, as investors bet big on embodied intelligence and continuous learning platforms over pure language models.

By mid-2026, venture capital investment in startups developing world models for physical AI surged dramatically, with over $3 billion raised in the first half of the year alone. Industry luminaries like Fei-Fei Li’s World Labs and Yann LeCun’s AMI Labs each secured approximately $1 billion, reflecting a strategic pivot from language models to embodied intelligence that enables AI systems to understand and interact with physical environments. This influx of capital underscores growing investor conviction in the commercial potential of physical AI as a transformative frontier.

The competitive landscape is rapidly diversifying as startups pursue varied approaches to world models, ranging from Niantic Spatial’s massive geospatial datasets derived from billions of images to General Intuition’s proprietary trillion-action-token dataset sourced from video game environments. This diversity extends to application domains as well, with companies like Overworld creating interactive video game worlds, Causal Labs focusing on weather prediction AI, and Extropic developing specialized hardware, illustrating a broadening ecosystem fueled by both innovation and strategic investor backing from firms like Valor Equity Partners and Point72 Ventures.

Investors and entrepreneurs alike are embracing a paradigm shift that values not just larger AI models but faster learning systems and continuous deployment infrastructures as the next competitive moat in physical AI. Applied Intuition’s Dana platform exemplifies this trend by accelerating development workflows twentyfold, enabling rapid iteration and real-world data feedback loops that compound value beyond static model improvements. This evolution signals a maturing market where engineering platforms and operational scalability are becoming as critical as foundational model capabilities.

The escalating demand for physical AI solutions across industrial sectors is driving startups like PhysicsX to scale globally and refine vertical-specific modules, while also confirming that larger, data-rich models yield superior generalization and zero-shot performance. Concurrently, companies such as General Intuition are leveraging fresh capital to enhance robotic embodiments through investments in compute infrastructure and talent, positioning themselves to meet the surging market appetite and solidify their foothold in this trillion-dollar opportunity. This dynamic illustrates how funding trends are tightly coupled with rapid organizational growth and technological sophistication.

Sources

Physics Becomes a Core Modality

Breakthroughs in physics-informed AI—like V-JEPA and GeoPT—are enabling machines to learn and generalize real-world dynamics from video, accelerating industrial simulation and robotic reasoning.

By early 2026, Yann LeCun's V-JEPA model marked a pivotal shift in physics-informed AI by teaching machines to predict missing video content through abstract conceptual understanding rather than pixel-level reconstruction. This approach enabled AI to grasp fundamental physical principles such as momentum and gravity, allowing robots to generalize tasks like liquid pouring from limited examples, thereby drastically reducing the need for extensive scenario-specific training. V-JEPA’s primitive world model capabilities extended physical AI beyond mere perception into higher-level spatial uncertainty modeling and planning, representing a foundational move toward AI systems that learn physics from videos rather than memorizing patterns.

Advancements in hybrid reasoning architectures and multimodal integration further propelled physical AI capabilities, exemplified by NVIDIA’s DreamZero and Cortex 2.0 systems. DreamZero’s 14-billion parameter World Action Model uniquely combined future state prediction with action generation, achieving over twice the task progress in unseen environments and adapting rapidly with minimal play data. Meanwhile, Cortex 2.0 innovated by imagining multiple future scene continuations in visual latent space, scoring them to guide proactive planning and robust execution, achieving a 95% success rate in complex multi-step tasks. This fusion of reactive control with foresight, grounded in real-world data rather than simulations, underscored a pragmatic shift toward transferable, efficient, and robust physical AI systems.

The emergence of physics as a core AI modality alongside text and pixels was crystallized by breakthroughs like MIT and Tsinghua’s GeoPT and the scaling efforts of PhysicsX. GeoPT demonstrated physics simulation up to four times faster and with 60% less data by leveraging synthetic dynamics of particle interactions, enabling more accurate and efficient modeling across industrial scenarios such as aerodynamics and fluid dynamics. Concurrently, PhysicsX established scaling laws showing that larger physics-informed models trained on extensive data sets achieve superior zero-shot generalization and are increasingly deployed via vertical modules tailored to specific industrial applications, from aerospace casting to aluminum smelting, accelerating product design and manufacturing.

Accelerated Understanding Inc. introduced a transformative nature-centric AI paradigm that moves beyond language-focused models by employing neural operators to simulate physical systems evolving across four-dimensional space-time. Their trillion-parameter models process unprecedented data scales—up to five trillion tokens in a single prompt—enabling unified physics simulation and reasoning across domains like chip design, robotics, weather prediction, and energy exploration. This approach promises to replace brittle, bespoke mathematical models with a general-purpose physics foundation model capable of reducing costly real-world experimentation and enhancing AI’s understanding and planning of complex physical phenomena.

Sources
TheAIGRIDInto AIAir Street PressMIT NewsTech XploreScaling Europe

Embodiment Drives AGI Ambitions

The path to Artificial General Intelligence is shifting toward embodied, multimodal systems that reason through vision, touch, and physical interaction—not just language alone.

By early 2026, the consensus among experts like Brett Acock of Figure Robotics is clear: achieving Artificial General Intelligence (AGI) demands embodied, multimodal systems that extend far beyond language models to include vision, audio, touch, and sophisticated physical reasoning. Figure Robotics’ Helix model exemplifies this trend by building hyper-accurate predictive world models that navigate real-world physics, suggesting that mastery of spatial and temporal reasoning in physical environments could become the true foundation of AGI, potentially leapfrogging pure language model approaches.

AMI Labs is pioneering this shift by developing JEPA-based world models that predict latent state representations and action-conditioned outcomes, focusing on abstract, compressed representations rather than raw sensory reconstruction. This approach directly tackles critical challenges such as hallucination and brittleness in AI systems, especially in safety-critical domains like robotics, healthcare, and industrial control, where persistent state tracking and causal understanding are paramount. The lab’s vision-heavy team and commitment to open research aim to foster a collaborative ecosystem that validates Yann LeCun’s 2022 JEPA framework beyond academia.

The PAN world model architecture further advances embodied AI by integrating dual reasoning modes—intuitive 'system one' and deliberate 'system two'—within a modular, hybrid framework that combines a self-developed large language model backbone with visual-language encoders and diffusion-based decoders. This design supports multimodal inputs and outputs while enabling holistic joint tuning akin to assembling aircraft parts, reflecting a strategic emphasis on self-sufficiency and internal retraining rather than reliance on external pre-trained models, thus promoting robustness and adaptability.

Despite these advances, significant challenges remain in embodied AI, particularly in achieving reliable long-horizon planning and action execution over extended timeframes essential for practical applications like robotics and autonomous driving. Experts like Eric Xing highlight the need for careful integration of data types, symbolic reasoning, and generative models to create steerable simulations that can faithfully replicate complex physical and social environments. These challenges underscore the importance of scalable infrastructure and open ecosystems to iterate on representations and reasoning strategies critical for the evolution toward AGI.

Sources
TheAIGRIDLatent.SpaceDataFramed

Get the stories behind the trends

Deep-dive reporting and the weekly brief, in your inbox.