Databricks doubles down on real-time AI: lakehouse//RT unifies data, slashes latency, and empowers non-tech users

Databricks’ LTAP and Lakehouse//RT architectures represent a paradigm shift by unifying transactional (OLTP) and analytical (OLAP) workloads on a single governed data copy stored in open formats like Delta Lake and Apache Iceberg. This consolidation eliminates the brittle and complex Change Data Capture (CDC) pipelines, which Databricks VP Shanku Niyogi famously dubbed “continuous data corruption,” thereby streamlining data management and governance under Unity Catalog. By enabling multiple specialized engines—such as Lakebase for PostgreSQL transactions and Lakehouse for analytics—to operate independently yet cohesively on the same data layer, Databricks overcomes the longstanding challenge of resource contention seen in previous HTAP attempts, allowing each workload to scale optimally with millisecond latency.

Powered by the new Reyden execution engine, Lakehouse//RT delivers real-time analytics directly on Delta Lake without requiring separate serving databases or additional storage layers, achieving query latencies as low as 10 milliseconds and supporting up to 12,000 concurrent queries with consistent low latency. Preview customers have reported performance improvements up to 16× faster than legacy real-time serving layers, enabling instant, pipeline-free queries that collapse traditional data silos and accelerate AI-driven decision-making and agentic workflows. This breakthrough not only simplifies architecture by removing duplicate storage and synchronization pipelines but also maintains robust governance, fundamentally transforming how enterprises handle high-concurrency, low-latency applications in 2026.

Databricks’ unified real-time data architecture addresses the explosive growth of AI agent workloads, which already generate approximately 80% of new databases on Lakebase, by eliminating the traditional assumption that transactional and analytical systems must be physically separated. Through a dual-layer storage approach—where Lakebase manages transactional data in a pageserver format locally and propagates it to object storage for analytical queries—Databricks maintains conceptual unification while pragmatically handling the technical complexity of two data representations. This innovation enables seamless, high-velocity experimentation with logical isolated data copies, reducing costly physical duplication and operational overhead, and setting a new standard for scalable, AI-driven data infrastructure.

The core innovation behind Databricks’ LTAP and Lakehouse//RT architectures lies in overcoming the fundamental performance dichotomy between row-oriented transactional workloads and columnar analytical queries within a shared storage system. By representing transactional tables in Iceberg format and enabling analytic compute directly on fresh, up-to-date data, Databricks eradicates the fragility and operational headaches of maintaining separate databases and ETL pipelines. This breakthrough not only delivers millisecond data freshness critical for AI-driven workflows and business intelligence but also simplifies data pipelines, reduces operational costs, and redefines enterprise data platform design amid rapidly evolving AI demands.

Databricks' AI agent Genie has fundamentally transformed enterprise analytics by empowering non-technical users across diverse teams—from shop floor owners to physical security and real estate staff—to independently perform complex, real-time data queries in plain language. This democratization of data access slashes response times from weeks to minutes, as users no longer need to rely on data scientists or write SQL queries themselves, dramatically accelerating decision-making and expanding the scope of data-driven questions. As one user noted, "I can just ask a question and get an answer," highlighting how Genie eliminates communication latency and disruption within organizations.

Genie's success hinges on foundational advances in product development and data modeling, including the creation of robust ontologies that enable AI agents to compute quantitative, data-grounded answers rather than merely reciting existing dashboards. This shift redefines business intelligence by replacing traditional dashboard reliance with conversational, agentic analytics workflows capable of answering complex questions like revenue forecasting or geographic sales analysis. As one analyst put it, "Genie is the way to actually answer business intelligence," marking a new era where AI agents become the primary interrogators of enterprise databases.

The rise of AI agents like Genie as primary database users has driven a surge in self-service analytics that demands revolutionary advances in inference cluster management. Databricks co-founder Reynold Xin highlights how fleetwide autoscaling of AI database fleets is powering real-time analytics and agentic workflows with unprecedented efficiency and cost control, effectively doubling developer productivity amid soaring AI operational costs. This autoscaling capability supports the high query concurrency generated by agentic analytics, enabling enterprises to scale AI-driven decision-making without prohibitive expense.

Databricks' introduction of logical isolated data copies has revolutionized experiment analytics by enabling AI agents to conduct high-velocity experimentation without the overhead and cost of physical data duplication. This innovation facilitates seamless, real-time data collaboration and agentic analytics workflows, further empowering AI agents like Genie to become the primary database users and dramatically enhancing developer productivity and enterprise decision-making. This breakthrough ushers in a new era of agile, scalable AI-driven analytics that supports rapid iteration and collaboration across teams.

Databricks' strategic enterprise positioning is anchored in its deliberate platform consolidation, which integrates relational features, conversational analytics, and agent orchestration directly into the lakehouse architecture, thereby minimizing dependence on disparate point solutions. This consolidation is exemplified by the Agent Bricks framework, which unifies models, enterprise data context, and tool execution into a single governed data asset, enhancing security and reducing latency—a critical advantage for enterprises seeking streamlined, secure AI-driven workflows. By emphasizing open-source protocols and unified data governance, Databricks not only standardizes AI toolset usage but also strengthens its competitive moat against proprietary ecosystems, aligning with enterprise demands to avoid vendor lock-in and simplify data management at scale.

Databricks sustains its innovation velocity through a culture of rapid prototyping and customer-centric development that empowers employees to build and iterate products within weeks without bureaucratic bottlenecks. This startup-like agility is bolstered by a core team of top talent operating with autonomy, enabling the company to tailor solutions closely with customers—often starting with bespoke implementations before scaling them broadly. Such an approach not only accelerates time-to-value but also ensures that platform enhancements directly address real-world enterprise challenges, as reflected in their targeted product launches for security and marketing teams that leverage seamless data integration to simplify AI agent deployment.

The platform ecosystem’s strength is further amplified by Databricks’ commitment to open data formats—beginning with Parquet and evolving through Delta and Iceberg—which facilitates broad enterprise adoption and downstream consumption while enabling large-scale batch processing alongside real-time AI workflows. This open foundation contrasts sharply with competitors like Snowflake, who focus on proprietary, smaller-scale solutions, and it empowers enterprises to unify data governance and management under a single, coherent system. Ubisoft’s migration to Databricks, leveraging Unity Catalog for governed data access and Lakeflow for rapid pipeline orchestration, illustrates how this ecosystem democratizes data management and accelerates developer agility by enabling teams to deploy and connect applications in hours rather than weeks.

Databricks’ vision for AI-driven enterprise platforms hinges on unifying diverse data streams—historical, real-time, and predictive—within an agent control plane that embeds governance and contextual awareness, enabling AI agents to continuously improve through implicit semantic learning tailored to enterprise-specific needs. This approach transforms AI agents from static skill marketplaces into dynamic collaborators that enhance decision-making and automation across business functions. By empowering both sales and engineering teams to build custom AI-powered applications, Databricks blurs traditional roles and accelerates customer value delivery, exemplifying a platform ecosystem designed not just for data management but for end-to-end AI-driven business problem solving.