Traditional data warehouses were designed in the 1980s for a specific kind of data problem: structured, predictable information generated by a small number of internal systems. They did that job reasonably well for decades, but the data problem has changed entirely and the architecture has not kept pace.
Organizations now handle structured data from transactions and databases, semi-structured data from APIs and IoT sensors, and unstructured data from emails, documents, images, and video. Gartner estimates that around 80% of enterprise data today is unstructured, a category that traditional warehouses were not built to handle. According to Dremio’s State of the Data Lakehouse survey, 86% of organizations plan to unify all their analytics data, something that is structurally impossible within a traditional warehouse architecture.
Lakehouse architecture has emerged as a direct response to these constraints.
The Hidden Costs of Running Two Separate Systems
For years, the standard answer to the limitations of data warehouses was to run a data lake alongside them. Data scientists got the flexibility of raw object storage such as S3 or ADLS for experimentation and machine learning. Business teams got the structured, SQL-optimized environment of a warehouse for reporting and dashboards. It sounded reasonable, but the operational reality was considerably messier.
Data engineers ended up spending most of their time building pipelines to move data between the two systems. The same data was stored twice, doubling storage costs. Because moving data takes time, business dashboards often reflected yesterday’s numbers rather than current ones. Without strict governance, the data lake gradually became what practitioners call a data swamp, meaning a repository where data exists but cannot reliably be found or trusted. Scaling a traditional warehouse requires either expensive hardware upgrades for on-premises systems or premium pricing tiers for cloud warehouses, often alongside complex migration projects.
What Lakehouse Architecture Actually Solves
A data lakehouse combines the reliability, governance, and query performance of a warehouse with low-cost cloud object storage, removing the need to maintain two separate systems while preserving the flexibility that made data lakes attractive in the first place.
The technical foundation that makes this possible is open table formats, primarily Apache Iceberg and Delta Lake. These formats bring ACID (Atomicity, Consistency, Isolation, Durability) transactions, schema evolution, and time travel capabilities directly to data lake storage. According to a survey of 500 data leaders by Dremio, 39% of organizations use Delta Lake and 31% use Apache Iceberg. Apache Iceberg has been gaining ground steadily, with major cloud providers including AWS, Google Cloud, and Microsoft Azure all announcing support for the format.
What these open formats deliver practically is vendor independence. Organizations are no longer locked into proprietary systems, and data can be queried by different compute engines without being moved or reformatted.
The architectural change this enables is significant. Traditional data architecture required data to flow through an ETL pipeline before it could be analyzed. In a lakehouse, data is stored once and queried directly by analytics tools, AI workloads, and business intelligence platforms simultaneously. Data duplication disappears, ETL complexity drops substantially, and both data scientists and business analysts work from the same source of truth.
The Rigidity Problem and the Six-Month Trap
Beyond AI, traditional warehouse architecture creates a specific operational problem that affects every analytics project. The schema must be defined upfront, and changing how customer data is structured triggers a migration project.
One industry professional described the pattern clearly. A project started in Q3 requires requirements gathered in Q4, data in the warehouse by Q1, and the first report by Q2. Six months have passed, the business has changed, new data sources exist, and the report reflects a snapshot of requirements that no longer match current needs. The warehouse was built for a specific point in time and cannot iterate without significant rework.
Lakehouse architecture uses a schema-on-read approach, storing data first and defining structure when it is queried rather than before it is ingested. New data sources can be integrated in hours, and teams can start working with data immediately, refine their questions as they go, and adjust structure based on what they learn rather than what they predicted they would need at the outset.
Why AI Workloads Changed the Calculation
The flexibility that lakehouses provide for unstructured data and rapid experimentation matters beyond standard analytics, and it is one reason AI adoption has accelerated the migration timeline considerably. In the Dremio survey, 81% of organizations reported using data lakehouses to support data scientists building and improving AI models, and the reasons are grounded in how AI workloads are structured.
Training machine learning models requires massive datasets, multiple data formats, rapid experimentation across different data combinations, and cost-effective compute scaling. Traditional warehouses were not designed for this kind of workload. A data scientist working in a traditional warehouse environment spends disproportionate time cleaning data exported from one system, reformatting it for another, and waiting for pipelines to complete before running the next experiment.
In a lakehouse environment, data scientists access clean, reliable data directly from the same storage layer used by business analysts. They can experiment with different subsets and formats without triggering pipeline work, and compute scales independently of storage to handle training workloads without the cost spike that equivalent work would generate in a warehouse environment.
The Cost Case for Migration
One of the most frequently cited reasons for migration is cost savings. According to the Dremio survey, 56% of companies expect to save more than 50% on analytics costs by moving to data lakehouses, and for large enterprises with more than 10,000 employees, nearly 30% expect savings greater than 75%. These figures reflect what organizations report after making the transition, based on responses from 500 enterprise IT and data professionals. The savings come from several places at once. Eliminating data duplication reduces storage costs directly. Separating compute and storage in cloud-native architectures means organizations pay for compute only when queries are running, rather than maintaining expensive dedicated infrastructure. Self-service analytics capabilities reduce dependence on specialized DBA expertise for routine tasks. Reducing ETL pipeline maintenance also frees engineering capacity for work that generates insight rather than moving data between systems.
Conclusion
According to the Dremio survey, 65% of companies are now running the majority of their analytics on lakehouses, and 70% predict that more than half of all analytics will run on lakehouse platforms within three years. Notably, 42% of those migrations came directly from cloud data warehouses, indicating the shift is not limited to organizations still running legacy on-premises systems.
For organizations still evaluating the move, a practical starting point is auditing where data duplication, ETL bottlenecks, and schema rigidity are creating the most friction. From there, a contained proof of concept on a single data domain using an open table format like Apache Iceberg, with governance requirements defined upfront, gives teams real performance and cost data before committing to a broader migration.
Frequently Asked Questions
What is lakehouse architecture?
Lakehouse architecture combines the scalability of a data lake with the governance and performance of a data warehouse, allowing teams to work with structured, semi-structured, and unstructured data in one unified environment.
How is a lakehouse different from a traditional data warehouse?
Traditional warehouses require predefined schemas and mainly support structured data. A lakehouse stores data more flexibly and enables analytics, reporting, and AI workloads from the same platform.
Why is lakehouse architecture important for AI?
AI workloads need large, varied datasets and fast experimentation. Lakehouses reduce data movement, support multiple data formats, and let compute scale independently from storage.
What are open table formats in a lakehouse?
Open table formats such as Apache Iceberg and Delta Lake add reliability features including transactions, schema evolution, and shared access across multiple analytics engines.
How can organizations start adopting lakehouse architecture?
Most organizations begin with a small proof of concept, compare cost and performance outcomes, and establish governance requirements before expanding adoption across teams.
