{
 "name": "Table format facts",
 "edition": "2026-11-10",
 "read_date": "2026-11-10",
 "source": "https://datalakehousecompare.com/datasets/",
 "license": "CC BY 4.0",
 "cite": "Lakehouse Compare, \"Lakehouse Compute Bill\", edition 10 Nov 2026, https://datalakehousecompare.com/datasets/ (CC BY 4.0).",
 "description": "Apache Iceberg, Delta Lake and Apache Hudi in the same eight rows, each with its source.",
 "rows": [
  {
   "format": "Apache Iceberg",
   "aspect": "Origin and governance",
   "fact": "Apache Iceberg is an Apache Software Foundation project, per its site footer, with its community run through dev, commits and issues mailing lists and a private PMC list.",
   "source_url": "https://iceberg.apache.org/community/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Iceberg",
   "aspect": "Latest release read",
   "fact": "Apache Iceberg 1.12.0, which the project releases page says was released on September 29, 2026 (GitHub lists the release on 2026-09-30). It removes Spark 3.4 support and Flink 2.0 support. The previous minor release, 1.11.0, was listed on GitHub on 2026-05-20.",
   "source_url": "https://iceberg.apache.org/releases/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Iceberg",
   "aspect": "Engines that write it",
   "fact": "The Iceberg home page names Spark, Trino, Flink, Presto, Hive and Impala as engines that work with the same tables at the same time, and shows MERGE INTO as a SQL command for writes.",
   "source_url": "https://iceberg.apache.org/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Iceberg",
   "aspect": "Compaction",
   "fact": "Iceberg compacts data files in parallel with Spark through the rewriteDataFiles action (the documentation example sets target-file-size-bytes to 500 MB). Rewriting manifests is also documented, and the jobs are run by the user.",
   "source_url": "https://iceberg.apache.org/docs/latest/maintenance/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Iceberg",
   "aspect": "Partitioning and clustering",
   "fact": "Iceberg uses hidden partitioning: it produces partition values by transforming a column (for example day(event_time)), and consumers need no extra filters. Partition layout and sort order can evolve in place without rewriting table data.",
   "source_url": "https://iceberg.apache.org/docs/latest/partitioning/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Iceberg",
   "aspect": "Row-level updates and deletes",
   "fact": "Iceberg documents write.delete.mode, write.update.mode and write.merge.mode, each defaulting to copy-on-write, with merge-on-read available for table format v2 and above. Spark supports MERGE INTO, UPDATE and DELETE FROM.",
   "source_url": "https://iceberg.apache.org/docs/latest/configuration/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Iceberg",
   "aspect": "Catalogs",
   "fact": "Spark catalog configuration documents Hive metastore, REST and directory-based Hadoop catalog types, and the documentation also lists AWS Glue, DynamoDB, JDBC and Nessie catalogs and a REST catalog protocol.",
   "source_url": "https://iceberg.apache.org/docs/latest/spark-configuration/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Iceberg",
   "aspect": "Interoperability",
   "fact": "The multi-engine support page lists Spark, Flink, Hive, Trino, Presto, Dremio, Amazon Athena, Amazon EMR, Amazon Redshift, Google BigQuery, Snowflake, Impala and others, and the documentation includes migration guides for Hive and Delta Lake tables.",
   "source_url": "https://iceberg.apache.org/multi-engine-support/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Delta Lake",
   "aspect": "Origin and governance",
   "fact": "Delta Lake says it is an independent open-source project not controlled by any single company, and that it joined the Delta Lake Project, a sub-project of the Linux Foundation Projects, in 2019. The site footer reads Delta Lake, a series of LF Projects, LLC.",
   "source_url": "https://delta.io/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Delta Lake",
   "aspect": "Latest release read",
   "fact": "Delta Lake 4.4.1, listed on GitHub on 05 Oct 2026 as a patch release on Delta Lake 4.4.0 (20 Aug 2026). It is built for Apache Spark 4.2.0, 4.1.0 and 4.0.1. The delta.io home page announces 4.4.0 on Spark 4.2.0.",
   "source_url": "https://github.com/delta-io/delta/releases/tag/v4.4.1",
   "read_date": "2026-11-10"
  },
  {
   "format": "Delta Lake",
   "aspect": "Engines that write it",
   "fact": "The Delta Lake home page names Spark, PrestoDB, Flink, Trino, Hive, Snowflake, Google BigQuery, Athena, Redshift, Databricks and Azure Fabric as compute engines, with APIs for Scala, Java, Rust and Python. It does not separate read from write support on that page.",
   "source_url": "https://delta.io/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Delta Lake",
   "aspect": "Compaction",
   "fact": "Delta Lake documents compaction (bin-packing) with OPTIMIZE, available from Delta Lake 1.2.0, with an optional WHERE predicate on partitions. The same page lists auto compaction, Z-ordering, data skipping and optimized write.",
   "source_url": "https://docs.delta.io/latest/optimizations-oss.html",
   "read_date": "2026-11-10"
  },
  {
   "format": "Delta Lake",
   "aspect": "Partitioning and clustering",
   "fact": "Delta Lake documents liquid clustering (CLUSTER BY), available from 3.1.0, which lets clustering columns be redefined without rewriting existing data. Clustering is not compatible with partitioning or ZORDER, and OPTIMIZE clusters data incrementally.",
   "source_url": "https://docs.delta.io/latest/delta-clustering.html",
   "read_date": "2026-11-10"
  },
  {
   "format": "Delta Lake",
   "aspect": "Row-level updates and deletes",
   "fact": "Delta Lake supports DELETE, UPDATE and MERGE INTO statements. Deletion vectors, available from 2.3.0 and labeled experimental on the page, mark rows as removed without rewriting the Parquet file until a rewrite such as OPTIMIZE applies them.",
   "source_url": "https://docs.delta.io/latest/delta-deletion-vectors.html",
   "read_date": "2026-11-10"
  },
  {
   "format": "Delta Lake",
   "aspect": "Catalogs",
   "fact": "Delta Lake 4.4.0, dated 20 Aug 2026, adds Unity Catalog Delta API integration for catalog-managed tables in Delta Kernel and the experimental Flink connector, and generated columns for tables created through Unity Catalog.",
   "source_url": "https://github.com/delta-io/delta/releases/tag/v4.4.0",
   "read_date": "2026-11-10"
  },
  {
   "format": "Delta Lake",
   "aspect": "Interoperability",
   "fact": "Delta Universal Format (UniForm) generates Iceberg metadata asynchronously, so Iceberg clients read Delta tables as if they were Iceberg, and the page says the same for Hudi clients. A single copy of the data files is kept.",
   "source_url": "https://docs.delta.io/latest/delta-uniform.html",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Hudi",
   "aspect": "Origin and governance",
   "fact": "Apache Hudi is a trademark of the Apache Software Foundation, per its site footer, and its home page describes it as an open data lakehouse platform built on an open table format.",
   "source_url": "https://hudi.apache.org/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Hudi",
   "aspect": "Latest release read",
   "fact": "Apache Hudi 1.2.1, listed on GitHub on 24 Sep 2026, keeps table version 9 as in 1.2.0 (23 May 2026). A 1.1.2 patch release on the older 1.1 line was listed on 17 Oct 2026. Hudi 1.2.0 added VECTOR, VARIANT and BLOB types and Lance file format support.",
   "source_url": "https://github.com/apache/hudi/releases/tag/release-1.2.1",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Hudi",
   "aspect": "Engines that write it",
   "fact": "The Hudi home page lists Apache Spark, Apache Flink, Databricks, AWS EMR, Azure HDInsight, Onehouse, Ray and Daft under data processing, and Kafka, Debezium and Flink CDC under data streaming.",
   "source_url": "https://hudi.apache.org/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Hudi",
   "aspect": "Compaction",
   "fact": "Compaction in Hudi applies only to Merge-on-Read tables and merges row-based log files into the columnar base file to produce a new base file. It has two steps, scheduling and execution. Clustering is a separate service that rewrites small files into larger ones.",
   "source_url": "https://hudi.apache.org/docs/compaction",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Hudi",
   "aspect": "Partitioning and clustering",
   "fact": "Hudi clustering batches small files into larger files and can cluster data by sort key for locality, using a replace commit. A hidden-partitioning feature like the one in Iceberg: none found on 2026-11-10 in the pages read.",
   "source_url": "https://hudi.apache.org/docs/clustering",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Hudi",
   "aspect": "Row-level updates and deletes",
   "fact": "Hudi is designed for mutable change streams: Merge-on-Read tables write updates and deletes to log files and merge them with base files during queries, and Copy-on-Write rewrites base files. The Record Level Index, added in 0.14.0, speeds key lookups.",
   "source_url": "https://hudi.apache.org/docs/table_types",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Hudi",
   "aspect": "Catalogs",
   "fact": "The Hudi home page lists AWS Glue Data Catalog, Google BigQuery, Apache Hive Metastore and DataHub as catalogs, with Apache XTable (incubating) for sync. The documentation describes syncing table metadata to a Hive metastore.",
   "source_url": "https://hudi.apache.org/",
   "read_date": "2026-11-10"
  },
  {
   "format": "Apache Hudi",
   "aspect": "Interoperability",
   "fact": "The Hudi home page lists Apache XTable (incubating) for metadata sync, and queries through Presto, Trino, Hive, AWS Athena, Google BigQuery, Apache Doris, StarRocks and Apache Impala. Delta UniForm documentation says Hudi clients can read Delta tables.",
   "source_url": "https://hudi.apache.org/",
   "read_date": "2026-11-10"
  }
 ]
}