Atlas AI sets new bar for 3d world modeling

Atlas pioneers a groundbreaking multimodal architecture that seamlessly integrates text, images, videos, camera poses, and 3D depth data into a single unified model, enabling both 3D reconstruction and generation within one framework. This holistic approach, championed by Fei-Fei Li and World Labs, marks a first in computer vision by combining pixel generation and reconstruction anchored on precise camera viewpoints, thus delivering high-fidelity spatial intelligence and spatiotemporal simulation capabilities.

Leveraging innovative 3D Gaussian splatting techniques, Atlas achieves hyper-realistic, traversable 3D scenes from as few as two to twenty-five ordinary photos, dramatically reducing traditional 3D acquisition workloads by two orders of magnitude. These transparent ellipsoid blobs enable efficient rendering and scalable camera translation, supporting complex effects like bullet time without the need for extensive multi-camera rigs, green screens, or expensive calibration—redefining cost-effective 3D scene generation.

Atlas’s new view prediction breakthrough fundamentally advances 3D reconstruction by spatially contextualizing pixel generation through native camera pose integration. This allows the model to generate smooth, consistent camera movements and fill in unseen scene areas, producing high-quality 1440p video clips from minimal input photos. By estimating geometry and viewpoint for every frame, Atlas transcends traditional next-frame prediction models, unlocking robust spatial intelligence critical for dynamic scene simulation and interactive applications.

The model’s ability to fuse diverse data modalities early in its pipeline enables it to stitch scattered, even unrelated, images into coherent, roamable 3D environments, effectively imagining spatial connections like corridors and rooms. This multimodal fusion not only enhances fidelity but also supports dynamic, editable 3D worlds that respond to user interactions, paving the way for immersive simulations and real-time robotics training with unprecedented scalability and precision.

Atlas by World Labs represents a foundational AI breakthrough by unifying 3D reconstruction and generation within a single architecture, a feat unprecedented in computer vision. This integration, anchored on viewpoint estimation, dissolves the traditional divide between generating imaginary scenes and reconstructing real environments, enabling the model to process text, images, videos, camera poses, and 3D depth maps natively from pre-training onward. As Fei-Fei Li explains, this fusion allows Atlas to simultaneously generate, reconstruct, and simulate worlds, marking a paradigm shift in multimodal world modeling.

At the core of Atlas lies the concept of 'new view prediction,' which elevates spatial intelligence beyond the capabilities of traditional video prediction models. Unlike LLMs that predict tokens or video models that forecast subsequent frames, Atlas predicts how a scene appears from novel spatial and temporal perspectives, effectively enabling the model to understand and simulate physical environments dynamically. This fundamental primitive, championed by Fei-Fei Li as on par with next token prediction, is poised to unlock advanced AI applications across robotics, architecture, and creative workflows by overcoming data constraints through editable and simulatable world representations.

Architecturally, Atlas transcends conventional approaches by embedding multimodal outputs within a persistent 3D coordinate space, allowing it to reason about physical geometry, structural depth, and object permanence. Functioning akin to a combined architectural engine and physics simulator, it constructs an internal 3D geometric mesh that supports precise manipulation of camera trajectories, lighting, and spatial movements while maintaining physical consistency. This novel design moves generative AI from passive 2D pixel synthesis to interactive 3D visual simulation, enabling real-time, pixel-perfect control and interactive world generation that extends across entertainment, robotics, and VR domains.

Atlas leverages the innovative use of Gaussian splats to capture and reconstruct real-world environments with exceptional visual fidelity, enabling robotic simulations that closely mirror reality. This technology encodes color information from multiple viewing angles, producing lifelike renderings of complex surfaces like reflective water or glass at high frame rates on standard Nvidia hardware. However, scaling these detailed 4D reconstructions from single rooms to enterprise-scale environments spanning tens of square kilometers, such as the 50 square kilometers of Rancho Cordova, remains a significant challenge for practical robotic training applications.

The integration of precise 3D reconstructions with collision meshes within simulators like Nvidia's Isaac enhances the physical realism of robotic interactions, addressing a critical gap where robots trained solely in simulation often falter in real-world deployment due to discrepancies in environmental accuracy. Atlas's ability to maintain persistent 3D state and spatial context mirrors human spatial reasoning and environment development over time, which is vital for bridging the divide between simulated and real environments and improving training outcomes.

Atlas's support for dynamic elements marks a transformative advance over previous static models like Marble World, with pretraining data capturing latent dynamics such as moving water waves and vehicles. This dynamic modeling allows the system to factor out transient elements to produce accurate static reconstructions, thereby enhancing simulation fidelity and robustness in robotics training scenarios where environmental changes are inevitable.

GE-Act 2.0, a world-action model pretrained on an unprecedented 30,000 hours of manipulation data, significantly boosts zero-shot control success rates—rising from 17.1% to 44.1% on G1-OP tasks and from 13.4% to 31.1% on G2-90D—demonstrating broad generalization across nearly all skill groups. By jointly training a control-oriented autoencoder, a single-step visual planner, and an inverse dynamics model with knowledge-aligned selective optimization, GE-Act 2.0 effectively bridges the real-sim gap, ensuring behaviorally compatible future state predictions. Its robust grounding of object attributes and ability to follow explicit instructions even against prior behaviors underscore its transformative potential for realistic robotic manipulation.

Atlas revolutionizes creative workflows by enabling consistent and physically coherent 3D view synthesis from minimal inputs, such as a few phone camera shots, allowing creators to generate complex effects like bullet-time without elaborate rigs. This capability stems from its multimodal autoregressive diffusion transformer architecture, which integrates text, images, video, camera poses, and depth to construct persistent 3D geometric scenes that maintain spatial relationships, lighting, and camera parameters, effectively functioning as both an architectural engine and physics simulator.

By automating the traditionally labor-intensive 3D design and revision process, Atlas empowers creatives to focus on higher-level innovation, providing intuitive spatial control and multimodal input integration that supports building persistent asset collections and spatial contexts. This statefulness aligns with how designers conceptualize environments over time, offering a stable foundation that addresses prior challenges of unstable multi-view image generation and enabling more fluid, iterative creative exploration.

Atlas’s real-time interactive video generation capabilities, exemplified by Runway’s GWM World 2 and Solaris platforms, allow users to engage dynamically with continuously generated 720p video streams and audio that respond to inputs such as camera movement and object manipulation. This shift toward interfaces generated on the fly not only transforms creative production but also redefines digital engagement and learning by enabling open-ended, steerable simulations and enterprise AI orchestration through integrations like CrewAI Flows with Gmail, Slack, and Salesforce.

Beyond media and entertainment, Atlas’s ability to replicate and pre-imagine real-world spaces is already impacting practical fields such as architecture, construction, and event booth design, where virtual spatial asset creation accelerates planning and visualization. This broad applicability underscores Atlas’s transformative potential to streamline workflows across diverse industries by providing a unified platform for intuitive, high-fidelity 3D world modeling and simulation.