Skip to content
Lakehouse Compare

Situation, read 10 Nov 2026

Large Iceberg tables and daily compaction cost: what to check on a lakehouse platform

All situationsOpen the Compute Bill

6 things to check, with the published facts that answer each

Who runs compaction on each platform: the platform or your team?

Compaction that runs by default costs money you did not schedule. Compaction you schedule costs engineering time.

Options in alphabetical order. Vendor statements are the vendor's own.
OptionPublished factSource, read
Amazon EMR plus Athena on IcebergS3 Tables compaction is on by default for all tables with a 512 MB default target file size (configurable 64 to 512 MB); Glue Iceberg compaction is disabled by default for new tables, and Athena compaction is manual.AWS documentation, 10 Nov 2026
DremioDremio runs automatic optimization on a dedicated engine it configures, and the same operations can be run by hand with OPTIMIZE TABLE.Dremio documentation, 10 Nov 2026
Google BigQuery and BigLakeBigQuery Iceberg managed tables run automatic storage optimization, so compaction needs no job configuration on Google Cloud.Google Cloud documentation, 10 Nov 2026
OnehouseOnehouse lists a Table Optimizer among its product claims.Onehouse page, 10 Nov 2026
Open stack on EMR and EKSIceberg documentation shows compaction as the Spark rewriteDataFiles action, with target-file-size-bytes of 500 MB in its example, run as a job you schedule.Apache Iceberg documentation, 10 Nov 2026
StarburstStarburst Galaxy compacts with ALTER TABLE ... EXECUTE optimize; files under file_size_threshold (default 100MB) are merged.Starburst documentation, 10 Nov 2026

What is the published unit price for compaction?

A per-unit price lets you multiply by your own daily volume. Unit prices differ in what they count: gigabytes, objects, DPU-hours or credits.

Options in alphabetical order. Vendor statements are the vendor's own.
OptionPublished factSource, read
Amazon EMR plus Athena on IcebergS3 Tables in us-east-1: compaction is $0.005 per GB processed plus $0.002 per 1,000 objects; sort or Z-order compaction is $0.01 per GB plus $0.002 per 1,000 objects.AWS price list, 10 Nov 2026
DremioDremio bills compute in DCUs only while engines are running; billable usage is DCUs for compute, LLM tokens for AI features and storage.Dremio documentation, 10 Nov 2026
DualBirdDualBird states about $2 per TB compacted for itself against about $6 per TB rising past $13 per TB for CPU clusters; DualBird ran the test and part of the saving is attributed to F2 local NVMe.DualBird page, 10 Nov 2026
Google BigQuery and BigLakeBigQuery storage optimization is a separate billed item; the dollar rate for it: none found on 2026-11-10 in the pricing page text read.Google Cloud documentation, 10 Nov 2026
OnehouseOnehouse bills in OCUs (1 OCU is a 4-core instance for 1 hour in your account); no dollar rate per OCU was found on 2026-11-10.Onehouse documentation, 10 Nov 2026
StarburstStarburst Galaxy lists table maintenance and data transforms at 0.30 credits per hour and file ingest at 0.60 credits per hour.Starburst page, 10 Nov 2026

Which Glue and EMR line items apply when your own Spark job compacts?

Self-run compaction is billed as ordinary compute plus storage requests. The AWS price list gives the rates.

Options in alphabetical order. Vendor statements are the vendor's own.
OptionPublished factSource, read
Amazon EMR plus Athena on IcebergGlue Iceberg compaction costs $0.44 per DPU-hour in us-east-1; Glue 6.0+ ETL jobs cost $0.308 per DPU-hour and older Glue ETL $0.44.AWS price list, 10 Nov 2026
Open stack on EMR and EKSEMR Serverless bills vCPU-hours and GB-hours of workers; the us-east-1 rates in AWS's pricing example are $0.052624 and $0.0057785.AWS page, 10 Nov 2026

Which compaction and layout strategies does each platform document?

Bin-pack, sort and Z-order rewrite files differently and cost differently. Check which one the price covers.

Options in alphabetical order. Vendor statements are the vendor's own.
OptionPublished factSource, read
Amazon EMR plus Athena on IcebergAthena documents OPTIMIZE ... REWRITE DATA USING BIN_PACK; S3 Tables prices sort and Z-order compaction separately from standard compaction.AWS documentation, 10 Nov 2026
DremioDremio documents data file compaction, clustering, partition evolution and manifest rewriting for Iceberg tables in its Open Catalog.Dremio documentation, 10 Nov 2026
DualBirdDualBird's published benchmark is sort-based Iceberg compaction on a synthetic table of about 370 million rows and 64 columns with ZSTD level 3.DualBird page, 10 Nov 2026
Google BigQuery and BigLakeBigQuery Iceberg managed tables document automatic compaction, clustering and garbage collection.Google Cloud documentation, 10 Nov 2026
Open stack on EMR and EKSIceberg documents rewriteDataFiles and manifest rewriting as optional maintenance; Hudi merge-on-read tables need compaction to bound read latency, and Delta Lake documents OPTIMIZE.Apache Iceberg documentation, 10 Nov 2026

How are row-level deletes handled during compaction?

Tables with frequent deletes carry delete files. Compaction either applies them or leaves them for readers.

Options in alphabetical order. Vendor statements are the vendor's own.
OptionPublished factSource, read
Amazon EMR plus Athena on IcebergS3 Tables compaction also applies row-level deletes.AWS documentation, 10 Nov 2026
DremioDremio handles row-level deletes for Iceberg tables with position deletes on v2 and deletion vectors on v3.Dremio documentation, 10 Nov 2026

What has been benchmarked for compaction, and by whom?

Most published results are SQL query or ETL tests, and the one compaction benchmark found is run by its vendor.

Options in alphabetical order. Vendor statements are the vendor's own.
OptionPublished factSource, read
DremioDremio's published benchmark is a 1 TB TPC-DS query test run by Dremio; it is not a compaction test.Dremio page, 10 Nov 2026
DualBirdDualBird's compaction white paper (July 1, 2026) restates the results and holds methodology, instance counts and Spark version behind a download form; the blog gives dataset shape and baselines.DualBird page, 10 Nov 2026
Open stack on EMR and EKSIceberg, Delta or Hudi compaction benchmarks for EMR or EKS, with time or cost: none found on 2026-11-10.Apache Iceberg documentation, 10 Nov 2026
StarburstA Starburst benchmark of compaction on Galaxy CPU clusters with data size and scripts: none found on 2026-11-10.Starburst page, 10 Nov 2026

Sources

  1. AWS documentation, read 10 Nov 2026
  2. Dremio documentation, read 10 Nov 2026
  3. Google Cloud documentation, read 10 Nov 2026
  4. Onehouse page, read 10 Nov 2026
  5. Apache Iceberg documentation, read 10 Nov 2026
  6. Starburst documentation, read 10 Nov 2026
  7. AWS price list, read 10 Nov 2026
  8. Dremio documentation, read 10 Nov 2026
  9. DualBird page, read 10 Nov 2026
  10. Onehouse documentation, read 10 Nov 2026
  11. Starburst page, read 10 Nov 2026
  12. AWS price list, read 10 Nov 2026
  13. AWS page, read 10 Nov 2026
  14. AWS documentation, read 10 Nov 2026
  15. Dremio page, read 10 Nov 2026
  16. DualBird page, read 10 Nov 2026
  17. Starburst page, read 10 Nov 2026

Last reviewed 10 Nov 2026.