Skip to content
Lakehouse Compare

Table maintenance, read 10 Nov 2026

Compaction on each platform: what is automated, what is billed

Table formatsOpen the Compute Bill

The table

Vendors alphabetical, AWS services first. Facts read 2026-11-10 from each vendor's documentation. This table lists automation and does not compare quality or cost.
PlatformAutomatic compactionHow it is billed or triggeredSource
Amazon AthenaNo: manual OPTIMIZE ... REWRITE DATA USING BIN_PACK, plus VACUUMYou issue the statementsAthena docs
Amazon S3 TablesYes, on by default for every table; 512 MB default target (64 to 512 MB); auto, binpack, sort, z-order$0.005 per GB plus $0.002 per 1,000 objects; sort or z-order $0.01 per GB plus $0.002 per 1,000 objects (us-east-1)S3 docs and price list
AWS Glue Data CatalogYes, but off by default; enabled per table with an IAM role; starts above 100 files each under 75% of target$0.44 per DPU-hour (us-east-1); Parquet tables onlyGlue docs and price list
BigQueryYes, for Iceberg managed tables: compaction, clustering and garbage collectionDCUs billed per second; load and export use Enterprise pay-as-you-go slotsGoogle Cloud docs
DatabricksYes, predictive optimization on Unity Catalog managed Delta and Iceberg tables; on by defaultServerless jobs SKU; does not run ZORDERDatabricks docs
DremioYes, for Iceberg tables in the Open Catalog, on a dedicated engine it configures; OPTIMIZE TABLE for manual runsEngines bill DCUs; Dremio says it prefers automatic over manual runsDremio docs
OnehouseVendor claims automated asynchronous table servicesBilled in OCUs; no dollar rate foundOnehouse docs
SnowflakeYes for Snowflake-managed Iceberg tables, on by default; none for externally managed tablesBillable credits; bundled when only Snowflake writes to Snowflake storage; Automatic Clustering billed separatelySnowflake docs
Starburst GalaxyNo: ALTER TABLE ... EXECUTE optimize; files under 100 MB by default are mergedYou issue the command; managed ingest table maintenance 0.30 credits per hourStarburst docs

Open-source engines and EMR

Open-source Iceberg, Delta Lake and Hudi leave compaction to a job you schedule, with settings such as Delta auto compaction (128 MB target) and Hudi async compaction as partial exceptions. Amazon EMR runs those jobs as Spark on EC2, EKS or Serverless and does not schedule them. The format pages are Iceberg compaction, Delta OPTIMIZE and VACUUM and Hudi table services.

Microsoft Fabric documents OPTIMIZE and an auto compaction setting for Delta tables; managed Iceberg compaction for Fabric: none found on 2026-11-10 in the pages read, and Fabric does not run inside AWS.

What the managed options leave to you

Snapshot expiry and orphan file removal are separate services on some platforms and none were found on others in the pages read; see snapshot expiry and orphan files. S3 Tables snapshot management fails on tables with user-defined tags or branches. Glue deletes only orphans created after the optimizer was created. Snowflake says it does not support orphan file deletion for Snowflake-managed Iceberg tables. Databricks predictive optimization skips tables outside Unity Catalog management.

DualBird's own compaction benchmark

DualBird, a Spark plugin vendor, published a sort-based Iceberg compaction benchmark in a blog post dated June 11, 2026, and a white paper landing page dated July 1, 2026. DualBird ran it, and no third-party run was found on 2026-11-10. The dataset is synthetic: about 370 million rows, 64 mixed-type columns, ZSTD level 3, 100 GB. The baselines are vanilla Spark and an unnamed C++ accelerated Spark.

DualBird says it measured roughly 12 to 20 times faster single-task execution, 55% to 85% lower EC2 cost, and a compaction cost of about $2 per TB against about $6 per TB rising past $13 per TB on CPU clusters. It attributes part of the saving to F2 local NVMe replacing EBS volumes. The DualBird home page shows a tile reading X15-X20 for what appears to be the same test, so the two published ranges differ. The white paper with the method is behind a sign-up form, and the blog gives no instance counts, Spark version, configuration or scripts.

DualBird is not a lakehouse. It speeds up Spark on EMR and EKS; a team on a fully managed platform cannot add it. It documents no Spark version list and no table-format support list; Iceberg appears only in its own compaction benchmark.

Other maintenance pages

Sources

  1. Athena Iceberg optimization, read 10 Nov 2026
  2. Amazon S3 Tables maintenance, read 10 Nov 2026
  3. AWS price list, Amazon S3, us-east-1, read 10 Nov 2026
  4. AWS Glue compaction optimization, read 10 Nov 2026
  5. AWS price list, AWS Glue, us-east-1, read 10 Nov 2026
  6. BigQuery Iceberg tables, read 10 Nov 2026
  7. Databricks predictive optimization, read 10 Nov 2026
  8. Dremio Cloud table optimization, read 10 Nov 2026
  9. Snowflake Iceberg table management, read 10 Nov 2026
  10. Snowflake Iceberg storage, read 10 Nov 2026
  11. Starburst Galaxy Iceberg, read 10 Nov 2026
  12. Starburst Galaxy pricing details, read 10 Nov 2026
  13. Onehouse about, read 10 Nov 2026
  14. Fabric table compaction, read 10 Nov 2026
  15. DualBird, Iceberg compaction benchmark blog, 2026-06-11, read 10 Nov 2026
  16. DualBird, compaction white paper landing page, read 10 Nov 2026
  17. DualBird home page, read 10 Nov 2026

Last reviewed 10 Nov 2026.