# Lakehouse Compare > Ten ways to run Spark batch work on a lakehouse, Iceberg, Delta Lake and Hudi side by side, and a Compute Bill priced from published AWS prices. Every page has a markdown twin at its address plus index.md (or send Accept: text/markdown). Every fact carries its source and the date it was read (10 Nov 2026). Datasets: https://datalakehousecompare.com/datasets/ (CSV and JSON, CC BY 4.0). ## Home - [Lakehouse platforms compared, with the bill for the same work on each](https://datalakehousecompare.com/index.md): Ten ways to run Spark batch work on a lakehouse, Iceberg, Delta Lake and Hudi side by side, and a Compute Bill priced from published AWS prices. ## Asset - [The Lakehouse Compute Bill](https://datalakehousecompare.com/compute-bill/index.md): What one Spark batch job costs on 12 ways of running it on AWS, from published prices, every assumption stated. ## Method - [How the Compute Bill is built](https://datalakehousecompare.com/compute-bill/method/index.md): The reference workload, the price sources, the arithmetic and what the bill leaves out. - [How Lakehouse Compare works](https://datalakehousecompare.com/method/index.md): Lakehouse Compare lists ten lakehouse platforms and three table formats and puts none of them in an order of merit. ## Index - [The platforms, A to Z](https://datalakehousecompare.com/platforms/index.md): 10 ways to run Spark batch work on a lakehouse, each with the same fields. - [Any two platforms side by side](https://datalakehousecompare.com/compare/index.md): 45 side-by-side pages: two platforms, the same rows, every cell sourced. - [Open table formats: Iceberg, Delta Lake, Hudi](https://datalakehousecompare.com/formats/index.md): Apache Iceberg, Delta Lake and Apache Hudi in the same eight rows, each with its source. - [Guides](https://datalakehousecompare.com/guides/index.md): 10 guides for the engineer who owns the Spark bill, each with its sources. - [Glossary](https://datalakehousecompare.com/glossary/index.md): 35 terms, each with a sourced example. - [News](https://datalakehousecompare.com/news/index.md): 15 dated items from primary sources, newest first. - [Situations](https://datalakehousecompare.com/situations/index.md): Five buyer situations, what to check in each and which published facts answer it. ## Profile - [Amazon EMR plus Athena on Iceberg: Spark and SQL engines on AWS](https://datalakehousecompare.com/platforms/aws-emr-athena-iceberg/index.md): Spark on EMR, SQL and Spark on Athena, and Iceberg tables in Glue or S3 Tables, priced per use on AWS and priced here in us-east-1. - [Databricks: Managed lakehouse built around Delta Lake](https://datalakehousecompare.com/platforms/databricks/index.md): Managed Spark with the Photon engine and serverless jobs, Delta Lake tables, and Iceberg reads of Delta data through UniForm, on AWS, Azure and Google Cloud. - [Dremio: Lakehouse with its own SQL engine](https://datalakehousecompare.com/platforms/dremio/index.md): Arrow-based SQL engine with automatic Iceberg table optimization and a Polaris-based Open Catalog, billed in DCUs on AWS, owned by SAP since July 2026. - [DualBird: FPGA Spark plugin for AWS F2](https://datalakehousecompare.com/platforms/dualbird/index.md): Spark plugin for AWS EC2 F2 FPGA instances, with a DualBird-run Iceberg compaction benchmark; it is an accelerator for EMR and EKS, not a lakehouse. - [Google BigQuery and BigLake: Serverless Google Cloud data warehouse](https://datalakehousecompare.com/platforms/bigquery-biglake/index.md): Serverless warehouse with Iceberg managed tables, BigLake external tables, Spark stored procedures and BigQuery Omni for S3 data. - [Microsoft Fabric: SaaS analytics on Azure with Spark](https://datalakehousecompare.com/platforms/microsoft-fabric/index.md): Azure SaaS analytics with Spark, OneLake storage and capacity pricing; S3 shortcuts read AWS data in place, read-only. - [Onehouse: Managed lakehouse in your cloud account](https://datalakehousecompare.com/platforms/onehouse/index.md): Quanton engine for Spark and SQL jobs in your own AWS or GCP account, with managed table services for Hudi, Iceberg and Delta Lake. - [Open stack on EMR and EKS: Self-run Spark with open table formats](https://datalakehousecompare.com/platforms/open-stack-emr-eks/index.md): Spark on EMR or EKS with open table formats: unchanged Spark code, with table maintenance, tuning and operations owned by the team. - [Snowflake: Cloud data platform with PySpark support](https://datalakehousecompare.com/platforms/snowflake/index.md): PySpark DataFrame code on Snowflake warehouses via Snowpark Connect, with Iceberg tables in customer-managed storage, on AWS, Azure and Google Cloud. - [Starburst: Managed Trino service for Iceberg](https://datalakehousecompare.com/platforms/starburst/index.md): Managed Trino lakehouse service that writes Iceberg tables, runs Iceberg maintenance commands and bills in credits, with prices by AWS region. ## Compare - [Amazon EMR plus Athena on Iceberg and Databricks side by side](https://datalakehousecompare.com/compare/aws-emr-athena-iceberg-vs-databricks/index.md): Amazon EMR plus Athena on Iceberg and Databricks: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Amazon EMR plus Athena on Iceberg and Dremio side by side](https://datalakehousecompare.com/compare/aws-emr-athena-iceberg-vs-dremio/index.md): Amazon EMR plus Athena on Iceberg and Dremio: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Amazon EMR plus Athena on Iceberg and DualBird side by side](https://datalakehousecompare.com/compare/aws-emr-athena-iceberg-vs-dualbird/index.md): Amazon EMR plus Athena on Iceberg and DualBird: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Amazon EMR plus Athena on Iceberg and Google BigQuery and BigLake side by side](https://datalakehousecompare.com/compare/aws-emr-athena-iceberg-vs-bigquery-biglake/index.md): Amazon EMR plus Athena on Iceberg and Google BigQuery and BigLake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Amazon EMR plus Athena on Iceberg and Microsoft Fabric side by side](https://datalakehousecompare.com/compare/aws-emr-athena-iceberg-vs-microsoft-fabric/index.md): Amazon EMR plus Athena on Iceberg and Microsoft Fabric: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Amazon EMR plus Athena on Iceberg and Onehouse side by side](https://datalakehousecompare.com/compare/aws-emr-athena-iceberg-vs-onehouse/index.md): Amazon EMR plus Athena on Iceberg and Onehouse: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Amazon EMR plus Athena on Iceberg and Open stack on EMR and EKS side by side](https://datalakehousecompare.com/compare/aws-emr-athena-iceberg-vs-open-stack-emr-eks/index.md): Amazon EMR plus Athena on Iceberg and Open stack on EMR and EKS: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Amazon EMR plus Athena on Iceberg and Snowflake side by side](https://datalakehousecompare.com/compare/aws-emr-athena-iceberg-vs-snowflake/index.md): Amazon EMR plus Athena on Iceberg and Snowflake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Amazon EMR plus Athena on Iceberg and Starburst side by side](https://datalakehousecompare.com/compare/aws-emr-athena-iceberg-vs-starburst/index.md): Amazon EMR plus Athena on Iceberg and Starburst: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Databricks and Dremio side by side](https://datalakehousecompare.com/compare/databricks-vs-dremio/index.md): Databricks and Dremio: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Databricks and DualBird side by side](https://datalakehousecompare.com/compare/databricks-vs-dualbird/index.md): Databricks and DualBird: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Databricks and Google BigQuery and BigLake side by side](https://datalakehousecompare.com/compare/bigquery-biglake-vs-databricks/index.md): Databricks and Google BigQuery and BigLake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Databricks and Microsoft Fabric side by side](https://datalakehousecompare.com/compare/databricks-vs-microsoft-fabric/index.md): Databricks and Microsoft Fabric: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Databricks and Onehouse side by side](https://datalakehousecompare.com/compare/databricks-vs-onehouse/index.md): Databricks and Onehouse: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Databricks and Open stack on EMR and EKS side by side](https://datalakehousecompare.com/compare/databricks-vs-open-stack-emr-eks/index.md): Databricks and Open stack on EMR and EKS: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Databricks and Snowflake side by side](https://datalakehousecompare.com/compare/databricks-vs-snowflake/index.md): Databricks and Snowflake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Databricks and Starburst side by side](https://datalakehousecompare.com/compare/databricks-vs-starburst/index.md): Databricks and Starburst: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Dremio and DualBird side by side](https://datalakehousecompare.com/compare/dremio-vs-dualbird/index.md): Dremio and DualBird: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Dremio and Google BigQuery and BigLake side by side](https://datalakehousecompare.com/compare/bigquery-biglake-vs-dremio/index.md): Dremio and Google BigQuery and BigLake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Dremio and Microsoft Fabric side by side](https://datalakehousecompare.com/compare/dremio-vs-microsoft-fabric/index.md): Dremio and Microsoft Fabric: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Dremio and Onehouse side by side](https://datalakehousecompare.com/compare/dremio-vs-onehouse/index.md): Dremio and Onehouse: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Dremio and Open stack on EMR and EKS side by side](https://datalakehousecompare.com/compare/dremio-vs-open-stack-emr-eks/index.md): Dremio and Open stack on EMR and EKS: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Dremio and Snowflake side by side](https://datalakehousecompare.com/compare/dremio-vs-snowflake/index.md): Dremio and Snowflake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Dremio and Starburst side by side](https://datalakehousecompare.com/compare/dremio-vs-starburst/index.md): Dremio and Starburst: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [DualBird and Google BigQuery and BigLake side by side](https://datalakehousecompare.com/compare/bigquery-biglake-vs-dualbird/index.md): DualBird and Google BigQuery and BigLake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [DualBird and Microsoft Fabric side by side](https://datalakehousecompare.com/compare/dualbird-vs-microsoft-fabric/index.md): DualBird and Microsoft Fabric: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [DualBird and Onehouse side by side](https://datalakehousecompare.com/compare/dualbird-vs-onehouse/index.md): DualBird and Onehouse: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [DualBird and Open stack on EMR and EKS side by side](https://datalakehousecompare.com/compare/dualbird-vs-open-stack-emr-eks/index.md): DualBird and Open stack on EMR and EKS: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [DualBird and Snowflake side by side](https://datalakehousecompare.com/compare/dualbird-vs-snowflake/index.md): DualBird and Snowflake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [DualBird and Starburst side by side](https://datalakehousecompare.com/compare/dualbird-vs-starburst/index.md): DualBird and Starburst: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Google BigQuery and BigLake and Microsoft Fabric side by side](https://datalakehousecompare.com/compare/bigquery-biglake-vs-microsoft-fabric/index.md): Google BigQuery and BigLake and Microsoft Fabric: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Google BigQuery and BigLake and Onehouse side by side](https://datalakehousecompare.com/compare/bigquery-biglake-vs-onehouse/index.md): Google BigQuery and BigLake and Onehouse: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Google BigQuery and BigLake and Open stack on EMR and EKS side by side](https://datalakehousecompare.com/compare/bigquery-biglake-vs-open-stack-emr-eks/index.md): Google BigQuery and BigLake and Open stack on EMR and EKS: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Google BigQuery and BigLake and Snowflake side by side](https://datalakehousecompare.com/compare/bigquery-biglake-vs-snowflake/index.md): Google BigQuery and BigLake and Snowflake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Google BigQuery and BigLake and Starburst side by side](https://datalakehousecompare.com/compare/bigquery-biglake-vs-starburst/index.md): Google BigQuery and BigLake and Starburst: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Microsoft Fabric and Onehouse side by side](https://datalakehousecompare.com/compare/microsoft-fabric-vs-onehouse/index.md): Microsoft Fabric and Onehouse: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Microsoft Fabric and Open stack on EMR and EKS side by side](https://datalakehousecompare.com/compare/microsoft-fabric-vs-open-stack-emr-eks/index.md): Microsoft Fabric and Open stack on EMR and EKS: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Microsoft Fabric and Snowflake side by side](https://datalakehousecompare.com/compare/microsoft-fabric-vs-snowflake/index.md): Microsoft Fabric and Snowflake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Microsoft Fabric and Starburst side by side](https://datalakehousecompare.com/compare/microsoft-fabric-vs-starburst/index.md): Microsoft Fabric and Starburst: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Onehouse and Open stack on EMR and EKS side by side](https://datalakehousecompare.com/compare/onehouse-vs-open-stack-emr-eks/index.md): Onehouse and Open stack on EMR and EKS: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Onehouse and Snowflake side by side](https://datalakehousecompare.com/compare/onehouse-vs-snowflake/index.md): Onehouse and Snowflake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Onehouse and Starburst side by side](https://datalakehousecompare.com/compare/onehouse-vs-starburst/index.md): Onehouse and Starburst: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Open stack on EMR and EKS and Snowflake side by side](https://datalakehousecompare.com/compare/open-stack-emr-eks-vs-snowflake/index.md): Open stack on EMR and EKS and Snowflake: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Open stack on EMR and EKS and Starburst side by side](https://datalakehousecompare.com/compare/open-stack-emr-eks-vs-starburst/index.md): Open stack on EMR and EKS and Starburst: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Snowflake and Starburst side by side](https://datalakehousecompare.com/compare/snowflake-vs-starburst/index.md): Snowflake and Starburst: published prices and results, code changes, table formats, where each runs, in the same rows with sources. - [Delta Lake and Apache Hudi side by side](https://datalakehousecompare.com/formats/delta-lake-vs-hudi/index.md): Delta Lake and Apache Hudi in the same eight rows: releases, engines, compaction, partitioning, row-level changes, catalogs. - [Delta Lake and Apache Iceberg side by side](https://datalakehousecompare.com/formats/delta-lake-vs-iceberg/index.md): Delta Lake and Apache Iceberg in the same eight rows: releases, engines, compaction, partitioning, row-level changes, catalogs. - [Apache Hudi and Apache Iceberg side by side](https://datalakehousecompare.com/formats/hudi-vs-iceberg/index.md): Apache Hudi and Apache Iceberg in the same eight rows: releases, engines, compaction, partitioning, row-level changes, catalogs. ## Format - [Apache Hudi: the published facts](https://datalakehousecompare.com/formats/hudi/index.md): Open lakehouse platform and table format under the Apache Software Foundation, with Merge-on-Read compaction, clustering and indexes; release 1.2.1 dated 2026-09-24. - [Apache Iceberg: the published facts](https://datalakehousecompare.com/formats/iceberg/index.md): Open table format under the Apache Software Foundation, with hidden partitioning, in-place evolution and a Spark compaction action; release 1.12.0 dated 2026-09-29. - [Delta Lake: the published facts](https://datalakehousecompare.com/formats/delta-lake/index.md): Open storage framework under the Linux Foundation, with OPTIMIZE compaction, liquid clustering, deletion vectors and UniForm; release 4.4.1 dated 2026-10-05. ## Guide - [Iceberg compaction: rewrite_data_files, strategies, platforms](https://datalakehousecompare.com/maintenance/iceberg-compaction/index.md): How Iceberg compaction works: the rewrite_data_files procedure, binpack, sort and z-order strategies, default thresholds, and which platforms run it for you. - [Iceberg snapshot expiry and orphan file removal](https://datalakehousecompare.com/maintenance/snapshot-expiry-and-orphan-files/index.md): How Iceberg snapshot expiry and orphan file removal work, with defaults, risks and the managed versions in S3 Tables, Glue, Athena, Snowflake and Starburst. - [Delta Lake OPTIMIZE and VACUUM: defaults and platform automation](https://datalakehousecompare.com/maintenance/delta-optimize-and-vacuum/index.md): Delta Lake OPTIMIZE, Z-ordering, auto compaction and VACUUM: documented defaults, safety checks, and how Databricks and Fabric run them. - [Apache Hudi table services: compaction, clustering, cleaning](https://datalakehousecompare.com/maintenance/hudi-table-services/index.md): Apache Hudi table services explained: compaction triggers and strategies, clustering, cleaning policies and timeline archival, with documented defaults. - [Which platforms automate Iceberg compaction](https://datalakehousecompare.com/maintenance/compaction-on-each-platform/index.md): Which platforms automate Iceberg table compaction: S3 Tables, Glue, Databricks, Dremio, Onehouse, Snowflake, BigQuery, with billing, plus DualBird's own benchmark. - [What is a data lakehouse: definition, origins, 2021 paper](https://datalakehousecompare.com/guides/what-is-a-data-lakehouse/index.md): Definition of a data lakehouse, where the term came from (the CIDR 2021 paper), and how warehouse, lake and lakehouse differ, with each claim tied to its source. - [Databricks vs Snowflake: published facts side by side](https://datalakehousecompare.com/guides/databricks-vs-snowflake-facts/index.md): Documented facts for Databricks and Snowflake side by side: Spark code, billing units, table formats, maintenance and regions, with dates and sources. - [Iceberg vs Delta Lake: documented facts, no pick](https://datalakehousecompare.com/guides/iceberg-vs-delta-lake/index.md): Apache Iceberg and Delta Lake side by side: latest releases, spec features, compaction defaults, Spark compatibility and cross-reading, all from project docs. - [EMR cost optimization for an Iceberg lakehouse](https://datalakehousecompare.com/guides/emr-cost-optimization-lakehouse/index.md): The EMR cost levers for Iceberg tables: instance choice, Spot, Graviton, managed scaling and compaction cost, with AWS list prices read 2026-11-10. - [Choosing a table format: what to check for Iceberg, Delta, Hudi](https://datalakehousecompare.com/guides/choosing-a-table-format/index.md): Checks before choosing Iceberg, Delta Lake or Hudi: writers, maintenance, platform support and exit cost, each with published facts and sources. - [The open stack on AWS: Spark, table formats and catalogs](https://datalakehousecompare.com/guides/open-stack-on-aws/index.md): The parts of an open lakehouse stack on AWS: EMR and EKS Spark, Iceberg, Delta and Hudi bundles, the Glue Data Catalog, S3 Tables and what you operate. - [Leaving Databricks or Snowflake: what moves and what does not](https://datalakehousecompare.com/guides/leaving-a-managed-platform/index.md): What moves when leaving Databricks or Snowflake for an open stack: table files, Spark code, maintenance, catalogs and billing, with documentation facts. - [Lakehouse pricing models: DBUs, credits, slots, CUs and EMR uplift](https://datalakehousecompare.com/guides/lakehouse-pricing-models/index.md): How each lakehouse platform bills (DBU, credit, slot, capacity unit, DCU, OCU, EC2 plus EMR uplift) with the dollar rates that could be read on 2026-11-10. - [Table catalogs for lakehouse engines: Glue, Unity, Polaris, Nessie](https://datalakehousecompare.com/guides/lakehouse-catalogs-for-tables/index.md): Table catalogs for Spark and other engines: AWS Glue Data Catalog, S3 Tables, Unity Catalog open source, Apache Polaris and Nessie, with documented interfaces. - [Where Spark batch work runs on each lakehouse platform](https://datalakehousecompare.com/guides/batch-spark-on-a-lakehouse/index.md): Where Spark batch ETL runs on AWS, BigQuery, Databricks, Dremio, Fabric, Onehouse, Snowflake and Starburst, with versions, billing units and limits. ## About - [About Lakehouse Compare](https://datalakehousecompare.com/about/index.md): Lakehouse Compare is a reference publication for teams choosing or re-choosing a lakehouse platform. ## Policy - [Editorial standards](https://datalakehousecompare.com/editorial-policy/index.md): Pages on Lakehouse Compare are written from primary sources: vendor documentation, price lists, project release pages and published benchmarks. - [Corrections](https://datalakehousecompare.com/corrections/index.md): Send a correction to editor@datalakehousecompare.com with the page address, the sentence that is wrong and a link to the source that shows the right fact. - [Contact](https://datalakehousecompare.com/contact/index.md): Write to editor@datalakehousecompare.com for corrections, questions about a source, a missing platform or an out-of-date price. - [Privacy](https://datalakehousecompare.com/privacy/index.md): Lakehouse Compare sets no cookies and loads no analytics script. - [Terms of use](https://datalakehousecompare.com/terms/index.md): Lakehouse Compare is a reference publication. ## Glossary - [Data lakehouse: definition and example](https://datalakehousecompare.com/glossary/data-lakehouse/index.md): A data lakehouse is a data management system that combines the benefits of a data lake and a data warehouse, per Databricks's documentation. - [Open table format: definition and example](https://datalakehousecompare.com/glossary/open-table-format/index.md): An open table format is a published specification for tracking which data files make up a table, along with its schema and partitioning, so that more than one engine - [Apache Iceberg: definition and example](https://datalakehousecompare.com/glossary/apache-iceberg/index.md): Apache Iceberg is an open table format for huge analytic tables, specified in a versioned document. - [Delta Lake: definition and example](https://datalakehousecompare.com/glossary/delta-lake/index.md): Delta Lake is an open source storage layer that adds ACID transactions, scalable metadata handling and time travel to data on S3, ADLS, GCS and HDFS. - [Apache Hudi: definition and example](https://datalakehousecompare.com/glossary/apache-hudi/index.md): Apache Hudi, pronounced hoodie, is an open table format and lakehouse platform built for high-performance writes on incremental pipelines. - [Apache XTable: definition and example](https://datalakehousecompare.com/glossary/apache-xtable/index.md): Apache XTable, formerly OneTable and still incubating at the Apache Software Foundation, translates table metadata between Hudi, Delta Lake and Iceberg without being - [Delta UniForm: definition and example](https://datalakehousecompare.com/glossary/delta-uniform/index.md): Delta Universal Format, or UniForm, lets a Delta Lake table generate Iceberg metadata so Iceberg clients can read it. - [Manifest file: definition and example](https://datalakehousecompare.com/glossary/manifest-file/index.md): In Apache Iceberg, a manifest is a file with one row per data file, holding the file's partition values and column metrics. - [Snapshot: definition and example](https://datalakehousecompare.com/glossary/snapshot/index.md): A snapshot is the state of an Iceberg table at one point in time, giving the full set of data files in the table. - [Time travel: definition and example](https://datalakehousecompare.com/glossary/time-travel/index.md): Time travel is querying a table as it was at an earlier version or timestamp. - [Schema evolution: definition and example](https://datalakehousecompare.com/glossary/schema-evolution/index.md): Schema evolution is changing a table's columns without rewriting its data. - [Partition evolution: definition and example](https://datalakehousecompare.com/glossary/partition-evolution/index.md): Partition evolution changes how an existing table is partitioned without rewriting old data. - [Hidden partitioning: definition and example](https://datalakehousecompare.com/glossary/hidden-partitioning/index.md): Hidden partitioning is an Iceberg feature where the table produces partition values by applying a transform, such as day(event_time), to a column. - [Copy-on-write: definition and example](https://datalakehousecompare.com/glossary/copy-on-write/index.md): Copy-on-write is a table type where an update or delete rewrites the whole base file that holds the changed rows. - [Merge-on-read: definition and example](https://datalakehousecompare.com/glossary/merge-on-read/index.md): Merge-on-read is a table type where updates and deletes go into small log files and are merged with base files when a query runs or when compaction rewrites them. - [Deletion vectors: definition and example](https://datalakehousecompare.com/glossary/deletion-vectors/index.md): Deletion vectors mark rows as deleted or modified in table metadata instead of rewriting the Parquet file that holds them. - [Compaction: definition and example](https://datalakehousecompare.com/glossary/compaction/index.md): Compaction rewrites many small data files into fewer large ones so queries open fewer files and manifests stay small. - [Small files problem: definition and example](https://datalakehousecompare.com/glossary/small-files-problem/index.md): The small files problem is the slowdown and metadata growth that follow when a table holds many data files far below the size an engine reads efficiently. - [Z-ordering: definition and example](https://datalakehousecompare.com/glossary/z-ordering/index.md): Z-ordering is a technique that colocates related values from several columns in the same set of files, so data skipping can ignore more files when a query filters on - [Liquid clustering: definition and example](https://datalakehousecompare.com/glossary/liquid-clustering/index.md): Liquid clustering is a Databricks data layout feature that replaces table partitioning and Z-ordering. - [Snapshot expiration: definition and example](https://datalakehousecompare.com/glossary/snapshot-expiration/index.md): Snapshot expiration removes old snapshots from an Iceberg table and deletes data files only those snapshots reference. - [Orphan files: definition and example](https://datalakehousecompare.com/glossary/orphan-files/index.md): Orphan files are files under a table's location that no table metadata references, usually left behind by failed tasks or jobs. - [REST catalog: definition and example](https://datalakehousecompare.com/glossary/rest-catalog/index.md): The Iceberg REST catalog is a common HTTP API, defined with an OpenAPI specification, for managing table metadata and catalog operations. - [AWS Glue Data Catalog: definition and example](https://datalakehousecompare.com/glossary/aws-glue-data-catalog/index.md): The AWS Glue Data Catalog is a centralized metadata repository on AWS that stores table locations, schemas and runtime metrics for your data sets. - [Amazon S3 Tables: definition and example](https://datalakehousecompare.com/glossary/amazon-s3-tables/index.md): Amazon S3 Tables is S3 storage built for tabular data. - [Apache Polaris: definition and example](https://datalakehousecompare.com/glossary/apache-polaris/index.md): Apache Polaris is an open source catalog for Apache Iceberg that implements Iceberg's REST API. - [Project Nessie: definition and example](https://datalakehousecompare.com/glossary/project-nessie/index.md): Project Nessie is a transactional catalog for data lakes with Git-like semantics. - [Unity Catalog: definition and example](https://datalakehousecompare.com/glossary/unity-catalog/index.md): Unity Catalog is the governance layer built into Databricks. - [DBU: definition and example](https://datalakehousecompare.com/glossary/dbu/index.md): A Databricks Unit, or DBU, is a normalized unit of processing power on the Databricks platform used for measurement and pricing. - [Snowflake credit: definition and example](https://datalakehousecompare.com/glossary/snowflake-credit/index.md): A Snowflake credit is the unit in which Snowflake meters compute: virtual warehouses, serverless features, compute pools and cloud services all consume credits. - [BigQuery slot: definition and example](https://datalakehousecompare.com/glossary/bigquery-slot/index.md): A BigQuery slot is a virtual compute unit BigQuery uses to run SQL queries, Python code and other jobs. - [Fabric capacity unit: definition and example](https://datalakehousecompare.com/glossary/fabric-capacity-unit/index.md): A capacity unit, or CU, measures compute power in Microsoft Fabric. - [OneLake shortcut: definition and example](https://datalakehousecompare.com/glossary/onelake-shortcut/index.md): A OneLake shortcut is a pointer in Microsoft Fabric to data stored elsewhere, so Fabric engines read it in place instead of copying it. - [Medallion architecture: definition and example](https://datalakehousecompare.com/glossary/medallion-architecture/index.md): A medallion architecture is a data design pattern that organizes lakehouse tables in three layers, bronze for raw data, silver for validated data and gold for enrich - [ACID transactions on object storage: definition and example](https://datalakehousecompare.com/glossary/acid-transactions/index.md): ACID means atomicity, consistency, isolation and durability. ## News - [Apache Iceberg 1.12.0 released, drops Spark 3.4 support](https://datalakehousecompare.com/news/iceberg-1-12-0-released/index.md): Apache Iceberg 1.12.0 was published on GitHub on 2026-09-30. - [Delta Lake 4.4.0 adds Apache Spark 4.2 support and identity columns](https://datalakehousecompare.com/news/delta-lake-4-4-0-released/index.md): Delta Lake 4.4.0 was released on 2026-08-20. - [Delta Lake 4.4.1 patch fixes S3A fast listing snapshots](https://datalakehousecompare.com/news/delta-lake-4-4-1-patch/index.md): Delta Lake 4.4.1 was published on 2026-10-05 as a patch on 4.4.0. - [Apache Hudi 1.2.1 moves Trino connector, raises heartbeat tolerance](https://datalakehousecompare.com/news/hudi-1-2-1-trino-packaging/index.md): Apache Hudi 1.2.1 was published on 2026-09-24. - [Apache XTable 0.4.0 adds Paimon and Parquet sources, Delta Kernel target](https://datalakehousecompare.com/news/xtable-0-4-0-incubating/index.md): Apache XTable 0.4.0-incubating was published on 2026-08-24. - [Apache Polaris 1.8.0 released with concurrent-commit retry fix](https://datalakehousecompare.com/news/polaris-1-8-0-released/index.md): Apache Polaris 1.8.0 was published on 2026-09-28. - [Unity Catalog OSS 0.6.0 adds Spark 4.2 connector and SQL views](https://datalakehousecompare.com/news/unity-catalog-0-6-0-released/index.md): Unity Catalog 0.6.0 was published on 2026-08-20. - [Unity Catalog OSS 0.7.0 enables native Iceberg tables by default](https://datalakehousecompare.com/news/unity-catalog-0-7-0-iceberg-rest/index.md): Unity Catalog 0.7.0 was published on 2026-10-13. - [Amazon S3 Tables adds managed migration from Hive to Iceberg](https://datalakehousecompare.com/news/s3-tables-hive-to-iceberg-migration/index.md): AWS announced on 2026-11-04 that Amazon S3 Tables can manage migration of Apache Hive tables to Apache Iceberg. - [AWS Glue Data Catalog supports vended credentials for federated Iceberg REST catalogs](https://datalakehousecompare.com/news/glue-catalog-vended-credentials-iceberg-rest/index.md): AWS announced on 2026-11-06 that Glue Data Catalog supports an external catalog's vended credentials in Iceberg REST catalog federation. - [Amazon Athena system tables store usage data as Iceberg tables on S3 Tables](https://datalakehousecompare.com/news/athena-system-tables-on-s3-tables/index.md): AWS announced on 2026-11-09 that Amazon Athena system tables capture usage data within minutes of query completion. - [Snowflake makes in-place Iceberg v2 to v3 upgrade generally available](https://datalakehousecompare.com/news/snowflake-iceberg-v3-upgrade-ga/index.md): Snowflake's release notes dated 2026-10-16 say in-place upgrade of Apache Iceberg tables from v2 to v3 is generally available. - [Snowflake opens externally managed Iceberg tables to outside engines via Horizon](https://datalakehousecompare.com/news/snowflake-horizon-external-iceberg-ga/index.md): Snowflake's release notes dated 2026-10-29 say access to externally managed Iceberg tables through Snowflake Horizon Catalog is generally available. - [Databricks: multi-table transactions generally available for managed Delta and Iceberg](https://datalakehousecompare.com/news/databricks-transactions-delta-iceberg-ga/index.md): Databricks's release notes dated 2026-10-14 say transactions are generally available for Unity Catalog managed Delta Lake and managed Apache Iceberg tables. - [BigQuery continuous queries can write to Iceberg managed tables](https://datalakehousecompare.com/news/bigquery-continuous-queries-to-iceberg/index.md): Google's BigQuery release notes dated 2026-09-28 say output rows from continuous queries can be written directly into Apache Iceberg managed tables with an INSERT st ## Faq - [Frequently asked questions](https://datalakehousecompare.com/faq/index.md): 30 questions with sourced answers. ## Situation - [Spark bill doubled on EMR or EKS](https://datalakehousecompare.com/situations/bill-doubled/index.md): What to check when an EMR or EKS Spark bill doubled: billing units, cost claims, code changes, settings that move spend, and region, with published facts. - [Nightly pipelines finish after the morning deadline](https://datalakehousecompare.com/situations/late-pipelines/index.md): For nightly pipelines that miss the morning: published runtimes, who ran them, whether a reader can repeat them, failure behavior on large jobs, and Spark versions. - [AI and feature pipelines wait on data preparation](https://datalakehousecompare.com/situations/ai-data-prep/index.md): For AI training and feature pipelines that wait on data prep: table formats, unchanged Spark code, UDF and fallback behavior, and where data has to live. - [Leaving Databricks or Snowflake for an open stack](https://datalakehousecompare.com/situations/leaving-managed/index.md): For a team weighing a move from Databricks or Snowflake to an open stack: what you own afterward, shared table formats, Spark changes and cloud coverage. - [Large Iceberg tables with daily compaction cost](https://datalakehousecompare.com/situations/iceberg-compaction/index.md): For teams paying for daily Iceberg compaction: who runs it, the published unit prices, strategies, delete handling and the benchmarks that exist. ## Datasets - [Datasets](https://datalakehousecompare.com/datasets/index.md): The site's datasets as CSV and JSON, edition 10 Nov 2026.